2024-04-20T10:59:48.080 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T10:59:48.080 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} --owner scheduled_rishabh@teuthology --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510 --verbose -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/orig.config.yaml 2024-04-20T10:59:48.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:59:48.235 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} email: null first_in_suite: false job_id: '7664510' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync rbytes: false syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfKNWmY1p7I+5w9SQsoBxeMbi53C4WzMJAd5iPMU3nGtkDKLzJXcMd+t+aOubje+prEa4qlSSuU78oG3MEz3Hk= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLe3Yp05ftzTUgBpMnkeH8yPfHBMhIAj8/8X4xjdR34j8DUzO2SnPjp7/Gbrj/1bQ/wugJoBIG1AKjoi4t2waiA= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPmy24BxrHwI8DvhECBJ3moTKjDEk42Zdr0RHmtXht/3rarIOXde/b3HzsG3rLNrGPPsEVZEVzCGUX+A5FrFEqE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T10:59:48.235 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T10:59:48.236 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T10:59:48.236 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T10:59:48.238 INFO:teuthology.task.internal:Checking packages... 2024-04-20T10:59:48.262 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T10:59:48.262 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:59:48.262 INFO:teuthology.packaging:ref: None 2024-04-20T10:59:48.262 INFO:teuthology.packaging:tag: None 2024-04-20T10:59:48.262 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T10:59:48.262 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:59:48.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T10:59:48.480 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T10:59:48.482 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T10:59:48.490 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T10:59:48.490 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T10:59:48.512 INFO:teuthology.task.internal:Saving configuration 2024-04-20T10:59:48.535 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T10:59:48.545 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:59:48.566 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:54:39.544519', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfKNWmY1p7I+5w9SQsoBxeMbi53C4WzMJAd5iPMU3nGtkDKLzJXcMd+t+aOubje+prEa4qlSSuU78oG3MEz3Hk='} 2024-04-20T10:59:48.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:54:39.546207', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLe3Yp05ftzTUgBpMnkeH8yPfHBMhIAj8/8X4xjdR34j8DUzO2SnPjp7/Gbrj/1bQ/wugJoBIG1AKjoi4t2waiA='} 2024-04-20T10:59:48.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:54:39.545460', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPmy24BxrHwI8DvhECBJ3moTKjDEk42Zdr0RHmtXht/3rarIOXde/b3HzsG3rLNrGPPsEVZEVzCGUX+A5FrFEqE='} 2024-04-20T10:59:48.604 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T10:59:48.612 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-20T10:59:48.613 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-20T10:59:48.613 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-20T10:59:48.613 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T10:59:48.706 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc11828a160>, signals=[15]) 2024-04-20T10:59:48.706 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T10:59:48.713 INFO:teuthology.task.internal:Opening connections... 2024-04-20T10:59:48.713 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T10:59:48.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:59:48.796 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-04-20T10:59:48.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:59:48.872 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T10:59:48.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:59:48.945 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T10:59:48.953 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-20T10:59:48.972 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-20T10:59:48.972 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-20T10:59:49.031 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-20T10:59:49.031 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-20T10:59:49.031 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-20T10:59:49.031 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:59:49.031 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:59:49.032 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:59:49.033 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-20T10:59:49.052 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-04-20T10:59:49.069 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-04-20T10:59:49.070 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-20T10:59:49.126 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-04-20T10:59:49.126 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-04-20T10:59:49.126 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-04-20T10:59:49.126 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:59:49.127 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:59:49.128 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:59:49.128 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:59:49.128 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-20T10:59:49.143 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-20T10:59:49.162 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-20T10:59:49.163 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-20T10:59:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-20T10:59:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-20T10:59:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-20T10:59:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:59:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-20T10:59:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:59:49.220 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:59:49.220 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-20T10:59:49.245 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T10:59:49.261 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T10:59:49.269 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T10:59:49.269 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:59:49.273 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:59:49.276 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:59:49.292 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T10:59:49.300 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T10:59:49.300 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:59:49.330 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:59:49.332 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:59:49.443 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T10:59:49.512 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T10:59:49.529 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T10:59:49.530 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-20T10:59:49.530 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-20T10:59:49.530 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:59:49.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:59:49.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-20T10:59:49.805 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:59:49.805 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:59:49.806 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-04-20T10:59:49.824 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:59:49.825 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:59:49.825 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:59:49.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum upgrade -y linux-firmware 2024-04-20T10:59:52.190 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-20T10:59:54.575 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-20T10:59:59.342 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-04-20T11:00:01.160 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-20T11:00:02.127 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-20T11:00:07.251 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-20T11:00:09.346 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T11:00:09.347 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T11:00:09.348 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T11:00:09.465 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T11:00:09.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T11:00:09.712 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T11:00:09.712 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T11:00:09.714 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi044.front.sepia.ceph.com to complete... 2024-04-20T11:00:22.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20 11:00:22 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T11:00:22.067 INFO:teuthology.orchestra.run.smithi044.stderr:FINISHED --2024-04-20 11:00:22-- 2024-04-20T11:00:22.067 INFO:teuthology.orchestra.run.smithi044.stderr:Total wall clock time: 12s 2024-04-20T11:00:22.067 INFO:teuthology.orchestra.run.smithi044.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-20T11:00:22.070 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T11:00:22.070 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T11:00:24.261 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying... ######################################## 2024-04-20T11:00:24.389 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing... ######################################## 2024-04-20T11:00:24.516 INFO:teuthology.orchestra.run.smithi044.stdout:Updating / installing... 2024-04-20T11:00:32.817 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T11:01:13.026 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi044.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T11:01:13.027 DEBUG:teuthology.orchestra.run.smithi044:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T11:01:13.110 INFO:teuthology.orchestra.run.smithi044.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.110 INFO:teuthology.orchestra.run.smithi044.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.110 INFO:teuthology.orchestra.run.smithi044.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T11:01:13.111 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T11:01:13.112 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T11:01:13.113 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T11:01:13.114 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T11:01:13.114 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T11:01:13.114 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T11:01:13.114 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T11:01:13.114 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T11:01:13.114 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T11:01:13.115 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T11:01:13.116 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T11:01:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T11:01:13.118 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T11:01:13.119 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T11:01:13.120 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T11:01:13.121 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T11:01:13.122 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T11:01:13.123 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T11:01:13.124 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T11:01:13.125 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T11:01:13.126 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T11:01:13.127 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T11:01:13.128 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T11:01:13.129 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T11:01:13.130 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T11:01:13.131 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T11:01:13.132 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T11:01:13.133 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T11:01:13.134 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T11:01:13.135 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T11:01:13.136 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T11:01:13.137 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T11:01:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T11:01:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T11:01:13.140 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T11:01:13.141 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T11:01:13.142 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T11:01:13.143 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T11:01:13.144 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T11:01:13.145 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T11:01:13.146 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T11:01:13.147 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T11:01:13.148 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T11:01:13.149 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T11:01:13.150 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T11:01:13.151 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T11:01:13.152 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T11:01:13.153 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T11:01:13.154 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T11:01:13.155 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T11:01:13.156 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T11:01:13.157 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T11:01:13.158 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T11:01:13.159 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T11:01:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T11:01:13.161 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T11:01:13.162 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T11:01:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T11:01:13.164 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T11:01:13.165 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T11:01:13.166 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T11:01:13.167 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T11:01:13.168 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T11:01:13.169 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T11:01:13.170 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T11:01:13.171 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T11:01:13.172 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T11:01:13.173 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T11:01:13.174 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T11:01:13.175 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T11:01:13.176 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T11:01:13.177 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T11:01:13.178 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T11:01:13.179 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T11:01:13.180 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T11:01:13.181 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T11:01:13.182 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T11:01:13.183 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T11:01:13.184 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T11:01:13.185 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T11:01:13.186 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T11:01:13.187 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T11:01:13.188 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T11:01:13.189 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T11:01:13.190 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T11:01:13.191 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T11:01:13.192 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T11:01:13.193 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T11:01:13.194 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T11:01:13.195 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T11:01:13.196 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T11:01:13.197 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T11:01:13.198 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T11:01:13.199 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T11:01:13.200 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T11:01:13.201 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T11:01:13.202 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T11:01:13.203 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T11:01:13.204 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T11:01:13.205 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T11:01:13.206 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T11:01:13.207 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T11:01:13.208 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T11:01:13.209 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T11:01:13.210 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T11:01:13.211 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T11:01:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T11:01:13.213 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T11:01:13.214 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T11:01:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T11:01:13.216 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T11:01:13.217 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T11:01:13.218 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T11:01:13.219 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T11:01:13.220 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T11:01:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T11:01:13.222 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T11:01:13.223 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T11:01:13.224 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T11:01:13.225 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T11:01:13.226 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T11:01:13.227 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T11:01:13.228 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T11:01:13.229 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T11:01:13.230 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T11:01:13.231 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T11:01:13.232 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T11:01:13.233 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T11:01:13.234 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T11:01:13.235 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T11:01:13.236 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T11:01:13.237 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T11:01:13.238 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T11:01:13.239 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T11:01:13.240 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T11:01:13.241 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T11:01:13.242 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T11:01:13.243 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T11:01:13.244 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T11:01:13.245 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T11:01:13.246 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T11:01:13.247 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T11:01:13.248 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T11:01:13.249 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T11:01:13.250 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T11:01:13.255 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.256 DEBUG:teuthology.orchestra.run.smithi044:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T11:01:13.280 INFO:teuthology.orchestra.run.smithi044.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T11:01:13.280 INFO:teuthology.orchestra.run.smithi044.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T11:01:13.280 INFO:teuthology.orchestra.run.smithi044.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T11:01:13.280 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-04-20T11:01:13.280 INFO:teuthology.orchestra.run.smithi044.stdout:for file in vmlinuz System.map config; do 2024-04-20T11:01:13.280 INFO:teuthology.orchestra.run.smithi044.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout: fi 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:done 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.281 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-04-20T11:01:13.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -qi grub2-tools 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Name : grub2-tools 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Epoch : 1 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Version : 2.06 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Release : 68.el9 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Architecture: x86_64 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Group : Unspecified 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:Size : 8264254 2024-04-20T11:01:13.364 INFO:teuthology.orchestra.run.smithi044.stdout:License : GPLv3+ 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Packager : builder@centos.org 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Vendor : CentOS 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Summary : Support tools for GRUB. 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:Description : 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout:hardware devices. 2024-04-20T11:01:13.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:01:13.366 INFO:teuthology.orchestra.run.smithi044.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T11:01:13.366 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T11:01:13.366 INFO:teuthology.task.kernel:Updating grub on smithi044 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:13.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T11:01:13.918 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-04-20T11:01:15.245 INFO:teuthology.orchestra.run.smithi044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T11:01:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-04-20T11:01:15.274 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-04-20T11:01:15.316 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.LPZLXzSmtx 2024-04-20T11:01:15.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LPZLXzSmtx 2024-04-20T11:01:15.383 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.LPZLXzSmtx 2024-04-20T11:01:15.524 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.LPZLXzSmtx is 6KB 2024-04-20T11:01:15.572 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.LPZLXzSmtx 2024-04-20T11:01:15.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T11:01:15.649 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T11:01:15.649 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T11:01:15.649 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T11:01:15.649 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T11:01:15.651 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T11:01:15.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo shutdown -r now 2024-04-20T11:01:45.838 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-20T11:01:45.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T11:01:45.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:02:45.842 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T11:02:54.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T11:02:54.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:02:58.235 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-20T11:02:58.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T11:02:58.865 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T11:02:58.865 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-04-20T11:02:58.908 INFO:teuthology.orchestra.run.smithi044.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T11:02:58.909 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T11:02:58.909 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T11:02:58.909 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T11:02:59.913 DEBUG:teuthology.parallel:result is None 2024-04-20T11:02:59.913 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T11:02:59.922 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T11:02:59.923 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T11:02:59.927 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T11:02:59.930 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T11:02:59.965 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T11:02:59.974 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T11:02:59.984 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T11:02:59.984 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T11:03:00.010 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T11:03:00.013 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T11:03:00.100 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T11:03:00.109 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T11:03:00.109 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T11:03:00.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:00.125 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T11:03:00.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:00.142 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T11:03:00.149 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T11:03:00.168 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T11:03:00.198 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:03:00.231 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:03:00.235 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:03:00.265 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:03:00.274 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:03:00.318 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T11:03:00.323 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T11:03:00.331 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T11:03:00.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T11:03:00.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T11:03:00.338 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T11:03:00.409 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T11:03:00.418 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T11:03:00.419 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T11:03:00.423 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T11:03:00.426 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T11:03:00.467 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:03:00.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:03:00.601 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:03:00.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:03:00.754 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:03:00.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T11:03:00.846 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:03:00.891 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:03:00.947 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:03:01.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:03:01.076 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:03:01.076 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T11:03:01.142 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:03:01.187 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T11:03:01.242 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:03:01.330 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T11:03:01.368 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:03:01.368 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T11:03:01.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-20T11:03:01.440 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-04-20T11:03:01.443 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-04-20T11:03:01.483 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:03:01.509 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:03:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:03:02.020 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T11:03:02.030 INFO:teuthology.task.internal:Starting timer... 2024-04-20T11:03:02.030 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T11:03:02.047 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T11:03:02.061 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-20T11:03:02.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-20T11:03:02.169 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-20T11:03:02.175 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-04-20T11:03:02.245 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-04-20T11:03:02.248 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-04-20T11:03:02.313 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-04-20T11:03:02.315 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T11:03:02.315 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-20T11:03:02.364 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-20T11:03:02.365 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-04-20T11:03:02.385 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-04-20T11:03:02.385 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-04-20T11:03:02.405 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-04-20T11:03:02.406 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-04-20T11:03:02.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:03:02.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:02.476 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:03:02.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:02.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:03:02.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:02.535 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T11:03:02.535 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-20T11:03:02.679 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-04-20T11:03:02.713 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-04-20T11:03:02.739 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T11:03:02.750 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T11:03:02.767 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T11:03:02.768 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi078.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T11:05:22.018 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-04-20T11:05:22.019 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T11:05:22.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:05:22.131 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-20T11:05:22.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T11:05:22.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-20T11:05:22.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:05:22.369 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-04-20T11:05:22.450 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-20T11:05:22.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-20T11:05:22.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T11:05:22.525 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-20T11:05:22.615 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-20T11:05:22.615 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T11:05:22.627 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T11:05:22.627 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T11:05:22.628 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:05:22.636 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T11:05:22.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:05:22.638 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T11:05:22.638 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:05:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T11:05:22.681 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T11:05:22.687 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T11:05:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T11:05:22.704 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-04-20T11:05:22.715 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-04-20T11:05:22.726 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T11:05:22.727 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-04-20T11:05:22.729 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T11:05:22.738 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-04-20T11:05:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T11:05:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T11:05:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T11:05:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T11:05:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:05:22.804 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:05:22.804 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-20T11:05:22.804 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.804 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.804 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.804 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.825 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:05:22.829 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:05:22.829 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-20T11:05:22.829 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.829 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.829 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.829 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:22.845 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-20T11:05:22.879 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-20T11:05:22.923 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T11:05:22.968 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T11:05:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:05:23.126 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:05:23.126 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-20T11:05:23.127 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:23.127 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:23.127 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:23.127 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:05:23.129 INFO:teuthology.run_tasks:Running task install... 2024-04-20T11:05:23.137 DEBUG:teuthology.task.install:project ceph 2024-04-20T11:05:23.137 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T11:05:23.137 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T11:05:23.138 INFO:teuthology.task.install:Using flavor: default 2024-04-20T11:05:23.146 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T11:05:23.146 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T11:05:23.146 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T11:05:23.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:05:23.149 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T11:05:23.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:05:23.150 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T11:05:23.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:05:23.318 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T11:05:23.318 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T11:05:23.328 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T11:05:23.328 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T11:05:23.389 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T11:05:23.389 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T11:05:23.430 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T11:05:23.430 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:05:23.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T11:05:23.500 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T11:05:23.500 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:05:23.500 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T11:05:23.527 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T11:05:23.528 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T11:05:23.532 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T11:05:23.532 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T11:05:23.567 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T11:05:23.567 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:05:23.568 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T11:05:23.595 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T11:05:23.596 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T11:05:23.596 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T11:05:23.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T11:05:23.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T11:05:23.671 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T11:05:23.735 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-04-20T11:05:23.737 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-04-20T11:05:23.741 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T11:05:23.781 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T11:05:23.806 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-04-20T11:05:23.808 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-04-20T11:05:23.865 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-20T11:05:23.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-20T11:05:23.961 INFO:teuthology.orchestra.run.smithi165.stdout:48 files removed 2024-04-20T11:05:23.988 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T11:05:24.035 INFO:teuthology.orchestra.run.smithi078.stdout:48 files removed 2024-04-20T11:05:24.062 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T11:05:24.203 INFO:teuthology.orchestra.run.smithi044.stdout:48 files removed 2024-04-20T11:05:24.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T11:05:24.734 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 172 kB/s | 84 kB 00:00 2024-04-20T11:05:24.780 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 182 kB/s | 84 kB 00:00 2024-04-20T11:05:25.134 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-20T11:05:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-20T11:05:25.236 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 154 kB/s | 84 kB 00:00 2024-04-20T11:05:25.518 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-20T11:05:25.631 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-20T11:05:25.778 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-20T11:05:26.337 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-20T11:05:27.382 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-20T11:05:29.483 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-20T11:05:33.667 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-20T11:05:34.950 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 877 kB/s | 8.0 MB 00:09 2024-04-20T11:05:35.318 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-20T11:05:37.076 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-20T11:05:44.884 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 446 kB/s | 8.0 MB 00:18 2024-04-20T11:05:55.193 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-20T11:06:00.226 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-04-20T11:06:02.402 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:06:02.403 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:06:02.414 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T11:06:02.415 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T11:06:02.416 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T11:06:02.416 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T11:06:02.418 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T11:06:02.420 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T11:06:02.421 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T11:06:02.483 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T11:06:02.496 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T11:06:02.497 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T11:06:02.498 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T11:06:02.499 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T11:06:02.500 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T11:06:02.501 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T11:06:02.502 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T11:06:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T11:06:02.504 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T11:06:02.505 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T11:06:02.506 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T11:06:02.507 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-04-20T11:06:02.508 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 2 Packages 2024-04-20T11:06:02.509 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:06:02.509 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 218 M 2024-04-20T11:06:02.509 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-20T11:06:08.133 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 885 kB/s | 19 MB 00:22 2024-04-20T11:06:15.427 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-04-20T11:06:17.520 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 16 kB 00:00 2024-04-20T11:06:19.919 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 21 MB 00:02 2024-04-20T11:06:25.076 INFO:teuthology.orchestra.run.smithi078.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-20T11:06:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:(2/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 8.3 MB/s | 5.4 MB 00:00 2024-04-20T11:06:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:(3/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 2.3 MB/s | 893 kB 00:00 2024-04-20T11:06:25.476 INFO:teuthology.orchestra.run.smithi078.stdout:(4/150): ceph-immutable-object-cache-19.0.0-313 2.1 MB/s | 145 kB 00:00 2024-04-20T11:06:25.618 INFO:teuthology.orchestra.run.smithi078.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-20T11:06:25.902 INFO:teuthology.orchestra.run.smithi078.stdout:(6/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-04-20T11:06:26.010 INFO:teuthology.orchestra.run.smithi078.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-20T11:06:26.146 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-20T11:06:26.919 INFO:teuthology.orchestra.run.smithi078.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 12 MB/s | 11 MB 00:00 2024-04-20T11:06:26.986 INFO:teuthology.orchestra.run.smithi078.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 378 kB/s | 25 kB 00:00 2024-04-20T11:06:27.645 INFO:teuthology.orchestra.run.smithi078.stdout:(10/150): ceph-common-19.0.0-3133.ga8c84f59.el9 7.3 MB/s | 21 MB 00:02 2024-04-20T11:06:27.712 INFO:teuthology.orchestra.run.smithi078.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 3.3 MB/s | 225 kB 00:00 2024-04-20T11:06:27.812 INFO:teuthology.orchestra.run.smithi078.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 271 kB/s | 27 kB 00:00 2024-04-20T11:06:27.903 INFO:teuthology.orchestra.run.smithi078.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 346 kB/s | 32 kB 00:00 2024-04-20T11:06:28.071 INFO:teuthology.orchestra.run.smithi078.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 4.4 MB/s | 748 kB 00:00 2024-04-20T11:06:28.163 INFO:teuthology.orchestra.run.smithi078.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.7 MB/s | 163 kB 00:00 2024-04-20T11:06:28.263 INFO:teuthology.orchestra.run.smithi078.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-20T11:06:28.455 INFO:teuthology.orchestra.run.smithi078.stdout:(17/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 6.6 MB/s | 17 MB 00:02 2024-04-20T11:06:28.480 INFO:teuthology.orchestra.run.smithi078.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 2.3 MB/s | 501 kB 00:00 2024-04-20T11:06:28.760 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:06:28.762 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:06:28.784 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T11:06:28.785 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T11:06:28.787 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T11:06:28.788 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T11:06:28.791 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T11:06:28.794 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T11:06:28.796 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T11:06:28.872 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T11:06:28.892 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-20T11:06:28.892 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T11:06:28.893 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T11:06:28.894 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T11:06:28.895 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T11:06:28.896 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T11:06:28.897 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T11:06:28.898 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T11:06:28.899 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T11:06:28.900 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T11:06:28.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T11:06:28.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T11:06:28.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-20T11:06:28.904 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-04-20T11:06:28.905 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-04-20T11:06:28.905 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:06:28.905 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 218 M 2024-04-20T11:06:28.905 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T11:06:28.955 INFO:teuthology.orchestra.run.smithi078.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 95 kB/s | 45 kB 00:00 2024-04-20T11:06:29.039 INFO:teuthology.orchestra.run.smithi078.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 1.6 MB/s | 135 kB 00:00 2024-04-20T11:06:29.156 INFO:teuthology.orchestra.run.smithi078.stdout:(21/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-04-20T11:06:29.198 INFO:teuthology.orchestra.run.smithi078.stdout:(22/150): python3-cephfs-19.0.0-3133.ga8c84f59. 1.0 MB/s | 162 kB 00:00 2024-04-20T11:06:29.240 INFO:teuthology.orchestra.run.smithi078.stdout:(23/150): python3-rados-19.0.0-3133.ga8c84f59.e 3.8 MB/s | 321 kB 00:00 2024-04-20T11:06:29.281 INFO:teuthology.orchestra.run.smithi078.stdout:(24/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.5 MB/s | 299 kB 00:00 2024-04-20T11:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:(25/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.2 MB/s | 99 kB 00:00 2024-04-20T11:06:29.357 INFO:teuthology.orchestra.run.smithi078.stdout:(26/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-20T11:06:29.441 INFO:teuthology.orchestra.run.smithi078.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-20T11:06:29.525 INFO:teuthology.orchestra.run.smithi078.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 410 kB/s | 34 kB 00:00 2024-04-20T11:06:29.608 INFO:teuthology.orchestra.run.smithi078.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.7 MB/s | 142 kB 00:00 2024-04-20T11:06:29.717 INFO:teuthology.orchestra.run.smithi078.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 7.9 MB/s | 3.1 MB 00:00 2024-04-20T11:06:30.192 INFO:teuthology.orchestra.run.smithi078.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 6.1 MB/s | 3.5 MB 00:00 2024-04-20T11:06:30.198 INFO:teuthology.orchestra.run.smithi044.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-20T11:06:30.648 INFO:teuthology.orchestra.run.smithi044.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.9 MB/s | 893 kB 00:00 2024-04-20T11:06:30.743 INFO:teuthology.orchestra.run.smithi078.stdout:(32/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 13 MB/s | 49 MB 00:03 2024-04-20T11:06:30.748 INFO:teuthology.orchestra.run.smithi044.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 1.4 MB/s | 145 kB 00:00 2024-04-20T11:06:30.898 INFO:teuthology.orchestra.run.smithi044.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-04-20T11:06:30.967 INFO:teuthology.orchestra.run.smithi078.stdout:(33/150): ceph-mgr-modules-core-19.0.0-3133.ga8 335 kB/s | 249 kB 00:00 2024-04-20T11:06:30.967 INFO:teuthology.orchestra.run.smithi078.stdout:(34/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 227 kB/s | 49 kB 00:00 2024-04-20T11:06:31.060 INFO:teuthology.orchestra.run.smithi078.stdout:(35/150): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-04-20T11:06:31.085 INFO:teuthology.orchestra.run.smithi078.stdout:(36/150): ceph-prometheus-alerts-19.0.0-3133.ga 109 kB/s | 16 kB 00:00 2024-04-20T11:06:31.107 INFO:teuthology.orchestra.run.smithi044.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-04-20T11:06:31.111 INFO:teuthology.orchestra.run.smithi078.stdout:(37/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.8 MB/s | 273 kB 00:00 2024-04-20T11:06:31.144 INFO:teuthology.orchestra.run.smithi078.stdout:(38/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 311 kB/s | 18 kB 00:00 2024-04-20T11:06:31.228 INFO:teuthology.orchestra.run.smithi078.stdout:(39/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 4.5 MB/s | 763 kB 00:00 2024-04-20T11:06:31.253 INFO:teuthology.orchestra.run.smithi078.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 565 kB/s | 80 kB 00:00 2024-04-20T11:06:31.337 INFO:teuthology.orchestra.run.smithi078.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-20T11:06:31.341 INFO:teuthology.orchestra.run.smithi044.stdout:(6/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 4.0 MB/s | 2.4 MB 00:00 2024-04-20T11:06:31.379 INFO:teuthology.orchestra.run.smithi078.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-20T11:06:31.412 INFO:teuthology.orchestra.run.smithi078.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-20T11:06:31.438 INFO:teuthology.orchestra.run.smithi078.stdout:(44/150): libquadmath-11.4.1-3.el9.x86_64.rpm 934 kB/s | 195 kB 00:00 2024-04-20T11:06:31.463 INFO:teuthology.orchestra.run.smithi078.stdout:(45/150): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-20T11:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout:(46/150): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-20T11:06:31.513 INFO:teuthology.orchestra.run.smithi078.stdout:(47/150): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-20T11:06:31.589 INFO:teuthology.orchestra.run.smithi078.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-04-20T11:06:31.666 INFO:teuthology.orchestra.run.smithi044.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-04-20T11:06:31.772 INFO:teuthology.orchestra.run.smithi078.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 584 kB/s | 107 kB 00:00 2024-04-20T11:06:31.806 INFO:teuthology.orchestra.run.smithi078.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-20T11:06:31.823 INFO:teuthology.orchestra.run.smithi078.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-04-20T11:06:31.856 INFO:teuthology.orchestra.run.smithi078.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-04-20T11:06:31.882 INFO:teuthology.orchestra.run.smithi078.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 555 kB/s | 33 kB 00:00 2024-04-20T11:06:31.907 INFO:teuthology.orchestra.run.smithi078.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-04-20T11:06:31.941 INFO:teuthology.orchestra.run.smithi078.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-20T11:06:31.966 INFO:teuthology.orchestra.run.smithi078.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-20T11:06:32.000 INFO:teuthology.orchestra.run.smithi078.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-20T11:06:32.025 INFO:teuthology.orchestra.run.smithi078.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-20T11:06:32.100 INFO:teuthology.orchestra.run.smithi078.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-04-20T11:06:32.159 INFO:teuthology.orchestra.run.smithi078.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-04-20T11:06:32.201 INFO:teuthology.orchestra.run.smithi078.stdout:(61/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-04-20T11:06:32.226 INFO:teuthology.orchestra.run.smithi078.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-04-20T11:06:32.251 INFO:teuthology.orchestra.run.smithi078.stdout:(63/150): libtool-2.4.6-45.el9.x86_64.rpm 2.5 MB/s | 582 kB 00:00 2024-04-20T11:06:32.277 INFO:teuthology.orchestra.run.smithi078.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-20T11:06:32.302 INFO:teuthology.orchestra.run.smithi078.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-20T11:06:32.327 INFO:teuthology.orchestra.run.smithi078.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-04-20T11:06:32.503 INFO:teuthology.orchestra.run.smithi078.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-20T11:06:32.583 INFO:teuthology.orchestra.run.smithi044.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 12 MB/s | 11 MB 00:00 2024-04-20T11:06:32.661 INFO:teuthology.orchestra.run.smithi078.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-20T11:06:32.703 INFO:teuthology.orchestra.run.smithi078.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-04-20T11:06:32.728 INFO:teuthology.orchestra.run.smithi078.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-20T11:06:32.787 INFO:teuthology.orchestra.run.smithi078.stdout:(71/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 4.1 MB 00:00 2024-04-20T11:06:32.842 INFO:teuthology.orchestra.run.smithi044.stdout:(9/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 6.9 MB/s | 21 MB 00:03 2024-04-20T11:06:32.856 INFO:teuthology.orchestra.run.smithi078.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-20T11:06:32.858 INFO:teuthology.orchestra.run.smithi044.stdout:(10/150): ceph-selinux-19.0.0-3133.ga8c84f59.el 92 kB/s | 25 kB 00:00 2024-04-20T11:06:32.888 INFO:teuthology.orchestra.run.smithi078.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-04-20T11:06:32.942 INFO:teuthology.orchestra.run.smithi044.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.7 MB/s | 225 kB 00:00 2024-04-20T11:06:32.946 INFO:teuthology.orchestra.run.smithi078.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-20T11:06:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-20T11:06:33.025 INFO:teuthology.orchestra.run.smithi044.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 327 kB/s | 27 kB 00:00 2024-04-20T11:06:33.072 INFO:teuthology.orchestra.run.smithi078.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-04-20T11:06:33.130 INFO:teuthology.orchestra.run.smithi078.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-20T11:06:33.146 INFO:teuthology.orchestra.run.smithi044.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 267 kB/s | 32 kB 00:00 2024-04-20T11:06:33.197 INFO:teuthology.orchestra.run.smithi078.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-20T11:06:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:(79/150): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-20T11:06:33.276 INFO:teuthology.orchestra.run.smithi044.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 5.6 MB/s | 748 kB 00:00 2024-04-20T11:06:33.298 INFO:teuthology.orchestra.run.smithi078.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-04-20T11:06:33.377 INFO:teuthology.orchestra.run.smithi044.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.6 MB/s | 163 kB 00:00 2024-04-20T11:06:33.390 INFO:teuthology.orchestra.run.smithi078.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-20T11:06:33.415 INFO:teuthology.orchestra.run.smithi078.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-20T11:06:33.449 INFO:teuthology.orchestra.run.smithi078.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-20T11:06:33.474 INFO:teuthology.orchestra.run.smithi078.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-20T11:06:33.477 INFO:teuthology.orchestra.run.smithi044.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-20T11:06:33.508 INFO:teuthology.orchestra.run.smithi078.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-04-20T11:06:33.533 INFO:teuthology.orchestra.run.smithi078.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-20T11:06:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-20T11:06:33.592 INFO:teuthology.orchestra.run.smithi078.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-04-20T11:06:33.610 INFO:teuthology.orchestra.run.smithi044.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 3.7 MB/s | 501 kB 00:00 2024-04-20T11:06:33.659 INFO:teuthology.orchestra.run.smithi078.stdout:(89/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-20T11:06:33.677 INFO:teuthology.orchestra.run.smithi078.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 467 kB/s | 51 kB 00:00 2024-04-20T11:06:33.693 INFO:teuthology.orchestra.run.smithi078.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-04-20T11:06:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-04-20T11:06:33.760 INFO:teuthology.orchestra.run.smithi078.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-04-20T11:06:33.802 INFO:teuthology.orchestra.run.smithi078.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-20T11:06:33.827 INFO:teuthology.orchestra.run.smithi078.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-04-20T11:06:33.861 INFO:teuthology.orchestra.run.smithi078.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-20T11:06:33.919 INFO:teuthology.orchestra.run.smithi078.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-20T11:06:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 690 kB/s | 58 kB 00:00 2024-04-20T11:06:33.978 INFO:teuthology.orchestra.run.smithi078.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-20T11:06:34.004 INFO:teuthology.orchestra.run.smithi078.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-20T11:06:34.037 INFO:teuthology.orchestra.run.smithi078.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-04-20T11:06:34.112 INFO:teuthology.orchestra.run.smithi078.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-20T11:06:34.221 INFO:teuthology.orchestra.run.smithi078.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-20T11:06:34.380 INFO:teuthology.orchestra.run.smithi078.stdout:(104/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-20T11:06:34.394 INFO:teuthology.orchestra.run.smithi044.stdout:(18/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 6.9 MB/s | 5.4 MB 00:00 2024-04-20T11:06:34.488 INFO:teuthology.orchestra.run.smithi078.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-04-20T11:06:34.514 INFO:teuthology.orchestra.run.smithi078.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-20T11:06:34.539 INFO:teuthology.orchestra.run.smithi078.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 973 kB/s | 309 kB 00:00 2024-04-20T11:06:34.644 INFO:teuthology.orchestra.run.smithi044.stdout:(19/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 5.1 MB/s | 17 MB 00:03 2024-04-20T11:06:34.648 INFO:teuthology.orchestra.run.smithi078.stdout:(108/150): libarrow-9.0.0-10.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-20T11:06:34.669 INFO:teuthology.orchestra.run.smithi044.stdout:(20/150): python3-ceph-argparse-19.0.0-3133.ga8 164 kB/s | 45 kB 00:00 2024-04-20T11:06:34.673 INFO:teuthology.orchestra.run.smithi078.stdout:(109/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-04-20T11:06:34.698 INFO:teuthology.orchestra.run.smithi078.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 309 kB/s | 49 kB 00:00 2024-04-20T11:06:34.728 INFO:teuthology.orchestra.run.smithi044.stdout:(21/150): python3-ceph-common-19.0.0-3133.ga8c8 1.6 MB/s | 135 kB 00:00 2024-04-20T11:06:34.732 INFO:teuthology.orchestra.run.smithi078.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-04-20T11:06:34.757 INFO:teuthology.orchestra.run.smithi078.stdout:(112/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-20T11:06:34.762 INFO:teuthology.orchestra.run.smithi044.stdout:(22/150): python3-cephfs-19.0.0-3133.ga8c84f59. 1.7 MB/s | 162 kB 00:00 2024-04-20T11:06:34.782 INFO:teuthology.orchestra.run.smithi078.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-20T11:06:34.816 INFO:teuthology.orchestra.run.smithi078.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-20T11:06:34.841 INFO:teuthology.orchestra.run.smithi078.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 754 kB/s | 43 kB 00:00 2024-04-20T11:06:34.866 INFO:teuthology.orchestra.run.smithi078.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 389 kB/s | 32 kB 00:00 2024-04-20T11:06:34.870 INFO:teuthology.orchestra.run.smithi044.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 2.7 MB/s | 299 kB 00:00 2024-04-20T11:06:34.892 INFO:teuthology.orchestra.run.smithi078.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-20T11:06:34.917 INFO:teuthology.orchestra.run.smithi078.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-20T11:06:34.921 INFO:teuthology.orchestra.run.smithi044.stdout:(24/150): python3-rados-19.0.0-3133.ga8c84f59.e 1.6 MB/s | 321 kB 00:00 2024-04-20T11:06:34.942 INFO:teuthology.orchestra.run.smithi078.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-20T11:06:34.954 INFO:teuthology.orchestra.run.smithi044.stdout:(25/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.2 MB/s | 99 kB 00:00 2024-04-20T11:06:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-20T11:06:34.992 INFO:teuthology.orchestra.run.smithi078.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 715 kB/s | 35 kB 00:00 2024-04-20T11:06:35.004 INFO:teuthology.orchestra.run.smithi044.stdout:(26/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-20T11:06:35.018 INFO:teuthology.orchestra.run.smithi078.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.7 MB/s | 203 kB 00:00 2024-04-20T11:06:35.068 INFO:teuthology.orchestra.run.smithi078.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-20T11:06:35.085 INFO:teuthology.orchestra.run.smithi078.stdout:(124/150): python3-grpcio-tools-1.46.7-10.el9.x 2.2 MB/s | 144 kB 00:00 2024-04-20T11:06:35.118 INFO:teuthology.orchestra.run.smithi078.stdout:(125/150): python3-jaraco-8.2.1-3.el9.noarch.rp 109 kB/s | 11 kB 00:00 2024-04-20T11:06:35.121 INFO:teuthology.orchestra.run.smithi044.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.4 MB/s | 170 kB 00:00 2024-04-20T11:06:35.144 INFO:teuthology.orchestra.run.smithi078.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-20T11:06:35.169 INFO:teuthology.orchestra.run.smithi078.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-20T11:06:35.194 INFO:teuthology.orchestra.run.smithi078.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-20T11:06:35.219 INFO:teuthology.orchestra.run.smithi078.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-20T11:06:35.253 INFO:teuthology.orchestra.run.smithi078.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-04-20T11:06:35.263 INFO:teuthology.orchestra.run.smithi044.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 242 kB/s | 34 kB 00:00 2024-04-20T11:06:35.278 INFO:teuthology.orchestra.run.smithi078.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-04-20T11:06:35.304 INFO:teuthology.orchestra.run.smithi078.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 962 kB/s | 79 kB 00:00 2024-04-20T11:06:35.329 INFO:teuthology.orchestra.run.smithi078.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-20T11:06:35.354 INFO:teuthology.orchestra.run.smithi078.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-20T11:06:35.371 INFO:teuthology.orchestra.run.smithi078.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-20T11:06:35.396 INFO:teuthology.orchestra.run.smithi078.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-20T11:06:35.414 INFO:teuthology.orchestra.run.smithi044.stdout:(29/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 6.7 MB/s | 3.1 MB 00:00 2024-04-20T11:06:35.422 INFO:teuthology.orchestra.run.smithi078.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 566 kB/s | 28 kB 00:00 2024-04-20T11:06:35.438 INFO:teuthology.orchestra.run.smithi078.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2024-04-20T11:06:35.447 INFO:teuthology.orchestra.run.smithi044.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 778 kB/s | 142 kB 00:00 2024-04-20T11:06:35.464 INFO:teuthology.orchestra.run.smithi078.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-20T11:06:35.480 INFO:teuthology.orchestra.run.smithi078.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-20T11:06:35.497 INFO:teuthology.orchestra.run.smithi078.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 617 kB/s | 36 kB 00:00 2024-04-20T11:06:35.514 INFO:teuthology.orchestra.run.smithi078.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-20T11:06:35.548 INFO:teuthology.orchestra.run.smithi078.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-20T11:06:35.573 INFO:teuthology.orchestra.run.smithi078.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-20T11:06:35.599 INFO:teuthology.orchestra.run.smithi078.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-20T11:06:35.616 INFO:teuthology.orchestra.run.smithi078.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-04-20T11:06:35.641 INFO:teuthology.orchestra.run.smithi078.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-20T11:06:35.683 INFO:teuthology.orchestra.run.smithi078.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-20T11:06:35.873 INFO:teuthology.orchestra.run.smithi044.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 7.7 MB/s | 3.5 MB 00:00 2024-04-20T11:06:35.981 INFO:teuthology.orchestra.run.smithi044.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 2.3 MB/s | 249 kB 00:00 2024-04-20T11:06:36.106 INFO:teuthology.orchestra.run.smithi044.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 397 kB/s | 49 kB 00:00 2024-04-20T11:06:36.263 INFO:teuthology.orchestra.run.smithi044.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 106 kB/s | 16 kB 00:00 2024-04-20T11:06:36.291 INFO:teuthology.orchestra.run.smithi078.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-04-20T11:06:36.340 INFO:teuthology.orchestra.run.smithi044.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 3.5 MB/s | 273 kB 00:00 2024-04-20T11:06:36.457 INFO:teuthology.orchestra.run.smithi044.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 6.4 MB/s | 763 kB 00:00 2024-04-20T11:06:36.525 INFO:teuthology.orchestra.run.smithi078.stdout:(150/150): librados2-19.0.0-3133.ga8c84f59.el9. 3.7 MB/s | 3.4 MB 00:00 2024-04-20T11:06:36.529 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-20T11:06:36.530 INFO:teuthology.orchestra.run.smithi078.stdout:Total 6.4 MB/s | 218 MB 00:34 2024-04-20T11:06:36.541 INFO:teuthology.orchestra.run.smithi044.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 222 kB/s | 18 kB 00:00 2024-04-20T11:06:36.725 INFO:teuthology.orchestra.run.smithi044.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 441 kB/s | 80 kB 00:00 2024-04-20T11:06:37.008 INFO:teuthology.orchestra.run.smithi044.stdout:(39/150): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-20T11:06:37.059 INFO:teuthology.orchestra.run.smithi044.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-20T11:06:37.151 INFO:teuthology.orchestra.run.smithi044.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-20T11:06:37.284 INFO:teuthology.orchestra.run.smithi044.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.2 MB 00:00 2024-04-20T11:06:37.318 INFO:teuthology.orchestra.run.smithi044.stdout:(43/150): libquadmath-11.4.1-3.el9.x86_64.rpm 634 kB/s | 195 kB 00:00 2024-04-20T11:06:37.353 INFO:teuthology.orchestra.run.smithi044.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-20T11:06:37.385 INFO:teuthology.orchestra.run.smithi044.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-20T11:06:37.419 INFO:teuthology.orchestra.run.smithi044.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-20T11:06:37.452 INFO:teuthology.orchestra.run.smithi044.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-20T11:06:37.594 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-20T11:06:37.619 INFO:teuthology.orchestra.run.smithi044.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-20T11:06:37.678 INFO:teuthology.orchestra.run.smithi044.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-20T11:06:37.711 INFO:teuthology.orchestra.run.smithi044.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-20T11:06:37.719 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-20T11:06:37.719 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-20T11:06:37.753 INFO:teuthology.orchestra.run.smithi044.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-04-20T11:06:37.787 INFO:teuthology.orchestra.run.smithi044.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-20T11:06:37.829 INFO:teuthology.orchestra.run.smithi044.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 440 kB/s | 33 kB 00:00 2024-04-20T11:06:37.888 INFO:teuthology.orchestra.run.smithi044.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 317 kB/s | 18 kB 00:00 2024-04-20T11:06:37.946 INFO:teuthology.orchestra.run.smithi044.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 194 kB/s | 11 kB 00:00 2024-04-20T11:06:38.021 INFO:teuthology.orchestra.run.smithi044.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-20T11:06:38.105 INFO:teuthology.orchestra.run.smithi044.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-04-20T11:06:38.147 INFO:teuthology.orchestra.run.smithi044.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 363 kB/s | 45 kB 00:00 2024-04-20T11:06:38.197 INFO:teuthology.orchestra.run.smithi044.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-20T11:06:38.239 INFO:teuthology.orchestra.run.smithi044.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 6.3 MB/s | 582 kB 00:00 2024-04-20T11:06:38.282 INFO:teuthology.orchestra.run.smithi044.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 307 kB/s | 25 kB 00:00 2024-04-20T11:06:38.332 INFO:teuthology.orchestra.run.smithi044.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 519 kB/s | 47 kB 00:00 2024-04-20T11:06:38.374 INFO:teuthology.orchestra.run.smithi044.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-20T11:06:38.408 INFO:teuthology.orchestra.run.smithi044.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-20T11:06:38.474 INFO:teuthology.orchestra.run.smithi044.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 354 kB/s | 35 kB 00:00 2024-04-20T11:06:38.658 INFO:teuthology.orchestra.run.smithi044.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-20T11:06:38.742 INFO:teuthology.orchestra.run.smithi044.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 16 MB/s | 4.1 MB 00:00 2024-04-20T11:06:38.825 INFO:teuthology.orchestra.run.smithi044.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 9.6 MB/s | 1.6 MB 00:00 2024-04-20T11:06:38.876 INFO:teuthology.orchestra.run.smithi044.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-20T11:06:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-20T11:06:39.010 INFO:teuthology.orchestra.run.smithi044.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-20T11:06:39.051 INFO:teuthology.orchestra.run.smithi044.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-04-20T11:06:39.219 INFO:teuthology.orchestra.run.smithi044.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-04-20T11:06:39.269 INFO:teuthology.orchestra.run.smithi044.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-20T11:06:39.319 INFO:teuthology.orchestra.run.smithi044.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-20T11:06:39.394 INFO:teuthology.orchestra.run.smithi044.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-04-20T11:06:39.426 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-20T11:06:39.426 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-20T11:06:39.528 INFO:teuthology.orchestra.run.smithi044.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-20T11:06:39.570 INFO:teuthology.orchestra.run.smithi044.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 19 kB 00:00 2024-04-20T11:06:39.603 INFO:teuthology.orchestra.run.smithi044.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-20T11:06:39.645 INFO:teuthology.orchestra.run.smithi044.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 467 kB/s | 35 kB 00:00 2024-04-20T11:06:39.696 INFO:teuthology.orchestra.run.smithi044.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-20T11:06:39.746 INFO:teuthology.orchestra.run.smithi044.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-20T11:06:39.813 INFO:teuthology.orchestra.run.smithi044.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-20T11:06:39.872 INFO:teuthology.orchestra.run.smithi044.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 736 kB/s | 42 kB 00:00 2024-04-20T11:06:39.922 INFO:teuthology.orchestra.run.smithi044.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 5.4 MB/s | 269 kB 00:00 2024-04-20T11:06:40.039 INFO:teuthology.orchestra.run.smithi044.stdout:(86/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-20T11:06:40.072 INFO:teuthology.orchestra.run.smithi044.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-20T11:06:40.114 INFO:teuthology.orchestra.run.smithi044.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-20T11:06:40.156 INFO:teuthology.orchestra.run.smithi044.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 617 kB/s | 51 kB 00:00 2024-04-20T11:06:40.199 INFO:teuthology.orchestra.run.smithi044.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 647 kB/s | 54 kB 00:00 2024-04-20T11:06:40.274 INFO:teuthology.orchestra.run.smithi044.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 562 kB/s | 42 kB 00:00 2024-04-20T11:06:40.333 INFO:teuthology.orchestra.run.smithi044.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 768 kB/s | 44 kB 00:00 2024-04-20T11:06:40.383 INFO:teuthology.orchestra.run.smithi044.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-04-20T11:06:40.442 INFO:teuthology.orchestra.run.smithi044.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 624 kB/s | 36 kB 00:00 2024-04-20T11:06:40.500 INFO:teuthology.orchestra.run.smithi044.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 824 kB/s | 47 kB 00:00 2024-04-20T11:06:40.559 INFO:teuthology.orchestra.run.smithi044.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-20T11:06:40.617 INFO:teuthology.orchestra.run.smithi044.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-20T11:06:40.684 INFO:teuthology.orchestra.run.smithi044.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-20T11:06:40.743 INFO:teuthology.orchestra.run.smithi044.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-20T11:06:40.802 INFO:teuthology.orchestra.run.smithi044.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-04-20T11:06:41.161 INFO:teuthology.orchestra.run.smithi044.stdout:(101/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-20T11:06:41.194 INFO:teuthology.orchestra.run.smithi044.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 57 kB/s | 22 kB 00:00 2024-04-20T11:06:41.369 INFO:teuthology.orchestra.run.smithi044.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.1 MB/s | 863 kB 00:00 2024-04-20T11:06:41.494 INFO:teuthology.orchestra.run.smithi044.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-20T11:06:41.545 INFO:teuthology.orchestra.run.smithi044.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 396 kB/s | 19 kB 00:00 2024-04-20T11:06:41.637 INFO:teuthology.orchestra.run.smithi044.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-20T11:06:41.687 INFO:teuthology.orchestra.run.smithi044.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 529 kB/s | 26 kB 00:00 2024-04-20T11:06:41.737 INFO:teuthology.orchestra.run.smithi044.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 989 kB/s | 49 kB 00:00 2024-04-20T11:06:41.829 INFO:teuthology.orchestra.run.smithi044.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-20T11:06:41.879 INFO:teuthology.orchestra.run.smithi044.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2024-04-20T11:06:42.049 INFO:teuthology.orchestra.run.smithi044.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 691 kB/s | 151 kB 00:00 2024-04-20T11:06:42.113 INFO:teuthology.orchestra.run.smithi044.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-04-20T11:06:42.164 INFO:teuthology.orchestra.run.smithi044.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 4.7 MB/s | 548 kB 00:00 2024-04-20T11:06:42.180 INFO:teuthology.orchestra.run.smithi044.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 659 kB/s | 43 kB 00:00 2024-04-20T11:06:42.214 INFO:teuthology.orchestra.run.smithi044.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 653 kB/s | 32 kB 00:00 2024-04-20T11:06:42.231 INFO:teuthology.orchestra.run.smithi044.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 302 kB/s | 15 kB 00:00 2024-04-20T11:06:42.273 INFO:teuthology.orchestra.run.smithi044.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T11:06:42.315 INFO:teuthology.orchestra.run.smithi044.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-20T11:06:42.349 INFO:teuthology.orchestra.run.smithi044.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-20T11:06:42.366 INFO:teuthology.orchestra.run.smithi044.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 717 kB/s | 35 kB 00:00 2024-04-20T11:06:42.408 INFO:teuthology.orchestra.run.smithi044.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-20T11:06:42.467 INFO:teuthology.orchestra.run.smithi044.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.5 MB/s | 144 kB 00:00 2024-04-20T11:06:42.533 INFO:teuthology.orchestra.run.smithi044.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-20T11:06:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-20T11:06:42.584 INFO:teuthology.orchestra.run.smithi044.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 359 kB/s | 18 kB 00:00 2024-04-20T11:06:42.601 INFO:teuthology.orchestra.run.smithi044.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-04-20T11:06:42.635 INFO:teuthology.orchestra.run.smithi044.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2024-04-20T11:06:42.660 INFO:teuthology.orchestra.run.smithi044.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-04-20T11:06:42.727 INFO:teuthology.orchestra.run.smithi044.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-20T11:06:42.752 INFO:teuthology.orchestra.run.smithi044.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 507 kB/s | 46 kB 00:00 2024-04-20T11:06:42.786 INFO:teuthology.orchestra.run.smithi044.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-20T11:06:42.811 INFO:teuthology.orchestra.run.smithi044.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 997 kB/s | 58 kB 00:00 2024-04-20T11:06:42.853 INFO:teuthology.orchestra.run.smithi044.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-20T11:06:42.878 INFO:teuthology.orchestra.run.smithi044.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 248 kB/s | 16 kB 00:00 2024-04-20T11:06:42.931 INFO:teuthology.orchestra.run.smithi044.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-20T11:06:42.962 INFO:teuthology.orchestra.run.smithi044.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 341 kB/s | 28 kB 00:00 2024-04-20T11:06:42.987 INFO:teuthology.orchestra.run.smithi044.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 556 kB/s | 31 kB 00:00 2024-04-20T11:06:43.021 INFO:teuthology.orchestra.run.smithi044.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T11:06:43.046 INFO:teuthology.orchestra.run.smithi044.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T11:06:43.080 INFO:teuthology.orchestra.run.smithi044.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 621 kB/s | 36 kB 00:00 2024-04-20T11:06:43.105 INFO:teuthology.orchestra.run.smithi044.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 881 kB/s | 51 kB 00:00 2024-04-20T11:06:43.648 INFO:teuthology.orchestra.run.smithi044.stdout:(142/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 4.5 MB/s | 49 MB 00:10 2024-04-20T11:06:43.664 INFO:teuthology.orchestra.run.smithi044.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 161 kB/s | 90 kB 00:00 2024-04-20T11:06:43.697 INFO:teuthology.orchestra.run.smithi044.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 372 kB/s | 230 kB 00:00 2024-04-20T11:06:43.715 INFO:teuthology.orchestra.run.smithi044.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 403 kB/s | 20 kB 00:00 2024-04-20T11:06:43.756 INFO:teuthology.orchestra.run.smithi044.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-20T11:06:43.832 INFO:teuthology.orchestra.run.smithi044.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-20T11:06:44.016 INFO:teuthology.orchestra.run.smithi044.stdout:(148/150): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-04-20T11:06:44.299 INFO:teuthology.orchestra.run.smithi044.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-04-20T11:06:44.441 INFO:teuthology.orchestra.run.smithi044.stdout:(150/150): librados2-19.0.0-3133.ga8c84f59.el9. 5.0 MB/s | 3.4 MB 00:00 2024-04-20T11:06:44.451 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T11:06:44.452 INFO:teuthology.orchestra.run.smithi044.stdout:Total 14 MB/s | 218 MB 00:15 2024-04-20T11:06:45.212 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-20T11:06:45.294 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T11:06:45.319 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T11:06:45.362 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T11:06:45.692 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T11:06:45.795 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:06:45.916 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:06:45.955 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:06:46.037 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:06:46.079 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:06:46.109 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T11:06:46.135 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T11:06:46.285 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T11:06:46.341 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T11:06:46.369 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:06:46.473 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:06:46.503 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:06:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:06:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:06:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:06:46.851 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:06:46.888 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:06:46.994 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T11:06:47.092 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T11:06:47.129 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T11:06:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T11:06:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T11:06:47.263 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T11:06:47.280 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T11:06:47.306 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T11:06:47.339 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T11:06:47.367 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T11:06:47.436 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T11:06:47.489 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T11:06:47.512 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T11:06:47.512 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T11:06:47.519 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T11:06:47.555 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T11:06:47.632 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T11:06:47.666 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T11:06:47.709 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T11:06:47.738 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T11:06:47.810 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T11:06:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T11:06:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T11:06:48.071 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T11:06:48.122 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T11:06:48.661 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T11:06:48.755 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T11:06:48.782 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T11:06:48.825 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T11:06:48.863 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T11:06:48.896 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T11:06:48.951 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T11:06:49.024 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T11:06:49.055 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T11:06:49.091 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T11:06:49.138 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T11:06:49.184 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T11:06:49.227 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T11:06:49.268 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T11:06:49.409 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T11:06:49.449 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T11:06:49.479 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T11:06:49.521 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T11:06:49.616 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T11:06:49.654 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T11:06:49.686 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T11:06:49.712 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T11:06:49.736 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T11:06:49.812 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T11:06:49.948 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T11:06:49.985 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T11:06:50.013 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T11:06:50.048 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T11:06:50.092 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T11:06:50.130 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T11:06:50.161 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T11:06:50.191 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T11:06:50.225 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T11:06:50.234 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T11:06:50.234 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T11:06:50.259 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T11:06:50.301 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T11:06:50.327 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T11:06:50.415 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T11:06:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T11:06:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T11:06:50.536 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T11:06:50.637 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T11:06:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T11:06:51.431 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T11:06:51.479 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T11:06:51.513 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T11:06:51.614 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T11:06:51.663 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T11:06:51.751 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T11:06:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T11:06:51.959 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T11:06:51.988 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T11:06:52.067 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T11:06:52.721 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T11:06:52.829 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T11:06:54.078 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T11:06:54.177 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T11:06:54.204 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T11:06:54.237 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T11:06:54.551 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T11:06:54.600 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:06:54.691 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:06:54.722 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T11:06:54.758 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T11:06:55.241 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T11:06:55.270 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:06:55.325 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:06:55.355 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T11:06:56.924 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T11:06:57.001 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T11:06:57.038 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T11:06:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T11:06:57.621 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T11:06:57.689 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:06:57.728 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:06:57.736 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:06:57.965 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:06:57.996 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:06:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:06:58.234 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:06:58.271 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T11:06:58.279 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:06:58.307 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T11:06:58.323 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:06:58.412 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:06:58.412 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T11:06:58.412 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T11:06:58.412 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:06:58.441 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:06:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T11:06:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T11:06:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:06:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:06:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:06:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:06:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:06:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:06:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:06:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:07:00.186 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T11:07:00.312 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T11:07:00.365 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T11:07:00.414 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T11:07:00.572 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T11:07:00.634 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T11:07:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T11:07:00.759 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T11:07:00.796 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T11:07:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T11:07:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T11:07:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T11:07:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T11:07:01.200 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T11:07:01.241 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T11:07:01.295 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T11:07:01.349 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T11:07:01.479 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T11:07:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T11:07:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T11:07:01.868 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T11:07:01.940 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T11:07:02.694 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T11:07:02.800 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T11:07:02.838 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T11:07:02.889 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T11:07:02.935 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T11:07:02.985 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T11:07:03.084 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T11:07:03.196 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T11:07:03.239 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T11:07:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T11:07:03.359 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T11:07:03.403 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T11:07:03.468 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T11:07:03.535 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T11:07:03.762 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T11:07:03.819 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T11:07:03.857 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T11:07:03.917 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T11:07:04.100 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T11:07:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T11:07:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T11:07:04.255 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T11:07:04.287 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T11:07:04.496 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T11:07:04.696 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T11:07:04.743 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T11:07:04.784 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T11:07:04.836 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T11:07:04.954 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T11:07:05.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T11:07:05.085 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T11:07:05.132 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T11:07:05.187 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T11:07:05.237 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T11:07:05.314 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T11:07:05.368 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T11:07:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T11:07:05.589 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T11:07:05.633 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T11:07:05.707 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T11:07:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T11:07:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T11:07:06.978 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T11:07:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T11:07:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T11:07:07.247 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T11:07:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T11:07:07.435 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T11:07:07.488 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T11:07:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T11:07:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T11:07:07.903 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T11:07:08.866 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T11:07:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T11:07:09.776 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:07:09.776 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-04-20T11:07:09.777 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:09.954 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:07:09.970 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:07:09.970 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:09.970 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T11:07:09.970 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:10.769 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T11:07:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T11:07:10.908 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T11:07:10.946 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T11:07:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T11:07:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:07:11.474 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:07:11.499 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:07:11.500 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:11.500 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T11:07:11.500 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:07:11.500 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:07:11.500 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:11.561 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:07:11.594 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T11:07:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T11:07:11.725 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:07:11.747 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:07:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T11:07:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:07:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:07:11.748 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:12.192 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:07:12.214 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:07:12.215 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:12.215 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T11:07:12.215 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:07:12.215 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:07:12.215 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:12.281 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T11:07:12.314 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:07:12.331 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T11:07:12.387 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T11:07:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T11:07:12.476 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:07:12.508 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T11:07:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T11:07:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T11:07:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T11:07:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T11:07:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T11:07:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T11:07:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T11:07:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T11:07:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:07:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:07:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T11:07:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T11:07:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:07:14.701 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:07:14.736 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:07:15.664 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:07:15.704 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:07:15.705 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:07:15.744 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:07:15.845 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:07:16.032 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T11:07:16.063 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:07:16.086 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:07:16.086 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:16.086 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T11:07:16.086 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:07:16.086 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:07:16.086 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:16.130 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:07:16.141 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:07:16.240 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T11:07:16.287 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T11:07:16.377 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:07:16.424 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:07:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:07:16.590 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T11:07:16.591 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T11:07:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:16.634 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:07:17.220 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T11:07:17.258 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:07:17.280 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:07:17.280 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:17.280 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T11:07:17.281 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:07:17.281 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:07:17.281 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:07:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:07:17.348 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:17.348 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T11:07:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:17.393 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:07:17.415 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:07:17.415 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:17.416 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T11:07:17.416 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:07:17.416 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:07:17.416 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:17.727 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:07:17.750 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:07:17.750 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:17.750 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T11:07:17.750 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:07:17.750 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:07:17.750 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T11:07:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T11:07:22.438 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T11:07:22.470 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T11:07:22.498 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T11:07:22.679 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T11:07:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T11:07:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T11:07:23.321 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:07:23.364 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:07:23.384 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:07:24.160 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T11:07:24.260 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T11:07:24.301 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T11:07:24.382 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T11:07:24.415 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T11:07:24.415 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:07:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:07:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:07:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-20T11:07:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T11:07:30.524 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T11:07:30.525 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T11:07:30.526 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T11:07:30.526 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T11:07:30.526 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T11:07:30.526 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T11:07:30.526 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T11:07:30.526 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T11:07:30.537 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T11:07:30.538 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T11:07:30.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T11:07:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T11:07:30.541 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T11:07:30.542 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T11:07:30.543 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T11:07:30.544 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T11:07:30.545 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T11:07:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:07:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:07:30.689 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:30.689 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T11:07:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:32.409 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:07:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:07:32.495 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:32.495 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T11:07:32.495 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:07:32.496 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:07:32.496 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T11:07:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T11:07:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T11:07:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T11:07:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T11:07:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T11:07:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T11:07:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T11:07:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-20T11:07:32.685 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-20T11:07:32.759 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:07:32.841 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:07:32.841 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:32.841 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T11:07:32.841 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:07:32.841 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:07:32.842 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:33.023 DEBUG:teuthology.parallel:result is None 2024-04-20T11:07:33.351 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:07:33.425 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:07:33.425 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:33.425 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T11:07:33.425 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:07:33.425 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:07:33.425 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:33.578 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T11:07:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T11:07:33.731 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T11:07:34.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T11:07:34.083 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T11:07:34.175 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T11:07:34.313 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T11:07:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T11:07:34.551 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T11:07:34.615 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T11:07:34.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T11:07:35.501 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:07:35.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:07:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T11:07:35.722 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T11:07:35.750 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:07:37.122 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:07:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:07:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:07:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:07:38.521 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:07:38.721 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T11:07:38.754 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:07:38.826 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:07:38.826 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:38.826 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T11:07:38.826 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:07:38.826 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:07:38.826 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:38.897 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:07:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:07:39.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T11:07:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T11:07:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T11:07:40.266 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:07:40.337 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:07:40.337 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:40.337 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T11:07:40.337 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:07:40.337 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:07:40.337 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:40.392 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:07:40.468 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:07:40.468 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:40.468 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T11:07:40.469 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:40.543 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:07:40.624 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:07:40.624 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:40.624 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T11:07:40.624 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:07:40.624 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:07:40.624 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:07:41.066 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:07:41.066 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:07:41.066 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T11:07:41.066 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:07:41.066 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:07:41.067 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:46.384 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T11:07:46.439 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T11:07:46.508 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T11:07:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T11:07:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T11:07:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T11:07:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T11:07:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T11:07:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:07:48.330 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:07:48.387 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:07:49.786 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T11:07:49.886 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T11:07:49.964 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T11:07:50.140 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T11:07:50.193 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T11:07:50.193 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:07:50.319 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:07:50.319 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:07:50.857 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:07:50.858 INFO:teuthology.orchestra.run.smithi044.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T11:07:50.858 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:53.071 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 48 kB/s | 6.2 MB 02:12 2024-04-20T11:07:56.859 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T11:07:56.859 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T11:07:56.859 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T11:07:56.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T11:07:56.861 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T11:07:56.862 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T11:07:56.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T11:07:56.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T11:07:56.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T11:07:56.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T11:07:56.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T11:07:56.873 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T11:07:56.874 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T11:07:56.875 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T11:07:56.876 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T11:07:56.877 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T11:07:56.878 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T11:07:56.879 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T11:07:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T11:07:56.881 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T11:07:56.882 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T11:07:56.882 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T11:07:56.882 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T11:07:56.882 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T11:07:56.882 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T11:07:56.882 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T11:07:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T11:07:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T11:07:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T11:07:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T11:07:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T11:07:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T11:07:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T11:07:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T11:07:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T11:07:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T11:07:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:07:59.467 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T11:08:00.007 DEBUG:teuthology.parallel:result is None 2024-04-20T11:08:12.319 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 912 B/s | 16 kB 00:18 2024-04-20T11:08:13.237 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-20T11:08:18.212 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-20T11:08:20.388 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:08:20.388 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T11:08:20.399 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T11:08:20.399 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T11:08:20.400 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T11:08:20.401 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T11:08:20.403 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T11:08:20.405 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T11:08:20.405 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T11:08:20.465 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T11:08:20.489 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T11:08:20.490 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T11:08:20.491 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T11:08:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T11:08:20.493 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T11:08:20.494 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T11:08:20.495 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T11:08:20.496 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T11:08:20.497 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T11:08:20.498 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T11:08:20.499 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T11:08:20.500 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 2 Packages 2024-04-20T11:08:20.501 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:08:20.502 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 218 M 2024-04-20T11:08:20.502 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T11:08:21.604 INFO:teuthology.orchestra.run.smithi165.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-20T11:08:21.821 INFO:teuthology.orchestra.run.smithi165.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 4.0 MB/s | 893 kB 00:00 2024-04-20T11:08:21.871 INFO:teuthology.orchestra.run.smithi165.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 2.8 MB/s | 145 kB 00:00 2024-04-20T11:08:22.155 INFO:teuthology.orchestra.run.smithi165.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-04-20T11:08:22.355 INFO:teuthology.orchestra.run.smithi165.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-04-20T11:08:22.639 INFO:teuthology.orchestra.run.smithi165.stdout:(6/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 4.5 MB/s | 5.4 MB 00:01 2024-04-20T11:08:22.922 INFO:teuthology.orchestra.run.smithi165.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-04-20T11:08:23.414 INFO:teuthology.orchestra.run.smithi165.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 11 MB/s | 21 MB 00:01 2024-04-20T11:08:23.481 INFO:teuthology.orchestra.run.smithi165.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 379 kB/s | 25 kB 00:00 2024-04-20T11:08:24.140 INFO:teuthology.orchestra.run.smithi165.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 8.7 MB/s | 11 MB 00:01 2024-04-20T11:08:24.214 INFO:teuthology.orchestra.run.smithi165.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.9 MB/s | 225 kB 00:00 2024-04-20T11:08:24.290 INFO:teuthology.orchestra.run.smithi165.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 360 kB/s | 27 kB 00:00 2024-04-20T11:08:24.357 INFO:teuthology.orchestra.run.smithi165.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 476 kB/s | 32 kB 00:00 2024-04-20T11:08:24.499 INFO:teuthology.orchestra.run.smithi165.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 5.2 MB/s | 748 kB 00:00 2024-04-20T11:08:24.582 INFO:teuthology.orchestra.run.smithi165.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.9 MB/s | 163 kB 00:00 2024-04-20T11:08:24.665 INFO:teuthology.orchestra.run.smithi165.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 1.5 MB/s | 126 kB 00:00 2024-04-20T11:08:24.757 INFO:teuthology.orchestra.run.smithi165.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 5.3 MB/s | 501 kB 00:00 2024-04-20T11:08:25.133 INFO:teuthology.orchestra.run.smithi165.stdout:(18/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 6.7 MB/s | 17 MB 00:02 2024-04-20T11:08:25.225 INFO:teuthology.orchestra.run.smithi165.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 493 kB/s | 45 kB 00:00 2024-04-20T11:08:25.325 INFO:teuthology.orchestra.run.smithi165.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 1.3 MB/s | 135 kB 00:00 2024-04-20T11:08:25.392 INFO:teuthology.orchestra.run.smithi165.stdout:(21/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 8.5 MB/s | 5.4 MB 00:00 2024-04-20T11:08:25.417 INFO:teuthology.orchestra.run.smithi165.stdout:(22/150): python3-cephfs-19.0.0-3133.ga8c84f59. 1.7 MB/s | 162 kB 00:00 2024-04-20T11:08:25.468 INFO:teuthology.orchestra.run.smithi165.stdout:(23/150): python3-rados-19.0.0-3133.ga8c84f59.e 4.2 MB/s | 321 kB 00:00 2024-04-20T11:08:25.518 INFO:teuthology.orchestra.run.smithi165.stdout:(24/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 2.9 MB/s | 299 kB 00:00 2024-04-20T11:08:25.543 INFO:teuthology.orchestra.run.smithi165.stdout:(25/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 99 kB 00:00 2024-04-20T11:08:25.593 INFO:teuthology.orchestra.run.smithi165.stdout:(26/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-20T11:08:25.685 INFO:teuthology.orchestra.run.smithi165.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-20T11:08:25.777 INFO:teuthology.orchestra.run.smithi165.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 374 kB/s | 34 kB 00:00 2024-04-20T11:08:25.877 INFO:teuthology.orchestra.run.smithi165.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.4 MB/s | 142 kB 00:00 2024-04-20T11:08:25.928 INFO:teuthology.orchestra.run.smithi165.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 8.0 MB/s | 3.1 MB 00:00 2024-04-20T11:08:26.803 INFO:teuthology.orchestra.run.smithi165.stdout:(31/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 15 MB/s | 49 MB 00:03 2024-04-20T11:08:26.862 INFO:teuthology.orchestra.run.smithi165.stdout:(32/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 3.6 MB/s | 3.5 MB 00:00 2024-04-20T11:08:26.895 INFO:teuthology.orchestra.run.smithi165.stdout:(33/150): ceph-mgr-modules-core-19.0.0-3133.ga8 2.7 MB/s | 249 kB 00:00 2024-04-20T11:08:26.920 INFO:teuthology.orchestra.run.smithi165.stdout:(34/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 846 kB/s | 49 kB 00:00 2024-04-20T11:08:26.945 INFO:teuthology.orchestra.run.smithi165.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3133.ga 328 kB/s | 16 kB 00:00 2024-04-20T11:08:27.046 INFO:teuthology.orchestra.run.smithi165.stdout:(36/150): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-04-20T11:08:27.071 INFO:teuthology.orchestra.run.smithi165.stdout:(37/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.8 MB/s | 273 kB 00:00 2024-04-20T11:08:27.096 INFO:teuthology.orchestra.run.smithi165.stdout:(38/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 366 kB/s | 18 kB 00:00 2024-04-20T11:08:27.130 INFO:teuthology.orchestra.run.smithi165.stdout:(39/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 4.0 MB/s | 763 kB 00:00 2024-04-20T11:08:27.230 INFO:teuthology.orchestra.run.smithi165.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 506 kB/s | 80 kB 00:00 2024-04-20T11:08:27.272 INFO:teuthology.orchestra.run.smithi165.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-20T11:08:27.322 INFO:teuthology.orchestra.run.smithi165.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-20T11:08:27.364 INFO:teuthology.orchestra.run.smithi165.stdout:(43/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-20T11:08:27.389 INFO:teuthology.orchestra.run.smithi165.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-20T11:08:27.414 INFO:teuthology.orchestra.run.smithi165.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-20T11:08:27.440 INFO:teuthology.orchestra.run.smithi165.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-20T11:08:27.465 INFO:teuthology.orchestra.run.smithi165.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-20T11:08:27.499 INFO:teuthology.orchestra.run.smithi165.stdout:(48/150): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-20T11:08:27.615 INFO:teuthology.orchestra.run.smithi165.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-20T11:08:27.632 INFO:teuthology.orchestra.run.smithi165.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 801 kB/s | 107 kB 00:00 2024-04-20T11:08:27.658 INFO:teuthology.orchestra.run.smithi165.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-04-20T11:08:27.708 INFO:teuthology.orchestra.run.smithi165.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-20T11:08:27.733 INFO:teuthology.orchestra.run.smithi165.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-04-20T11:08:27.759 INFO:teuthology.orchestra.run.smithi165.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-20T11:08:27.784 INFO:teuthology.orchestra.run.smithi165.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 360 kB/s | 18 kB 00:00 2024-04-20T11:08:27.809 INFO:teuthology.orchestra.run.smithi165.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-20T11:08:27.834 INFO:teuthology.orchestra.run.smithi165.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-20T11:08:27.860 INFO:teuthology.orchestra.run.smithi165.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-20T11:08:27.893 INFO:teuthology.orchestra.run.smithi165.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-20T11:08:27.927 INFO:teuthology.orchestra.run.smithi165.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-20T11:08:27.952 INFO:teuthology.orchestra.run.smithi165.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-20T11:08:27.978 INFO:teuthology.orchestra.run.smithi165.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-20T11:08:28.003 INFO:teuthology.orchestra.run.smithi165.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-20T11:08:28.053 INFO:teuthology.orchestra.run.smithi165.stdout:(64/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-04-20T11:08:28.078 INFO:teuthology.orchestra.run.smithi165.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 35 kB 00:00 2024-04-20T11:08:28.103 INFO:teuthology.orchestra.run.smithi165.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-20T11:08:28.196 INFO:teuthology.orchestra.run.smithi165.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-20T11:08:28.237 INFO:teuthology.orchestra.run.smithi165.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-20T11:08:28.296 INFO:teuthology.orchestra.run.smithi165.stdout:(69/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 19 MB/s | 4.1 MB 00:00 2024-04-20T11:08:28.321 INFO:teuthology.orchestra.run.smithi165.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-20T11:08:28.354 INFO:teuthology.orchestra.run.smithi165.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-20T11:08:28.388 INFO:teuthology.orchestra.run.smithi165.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-20T11:08:28.413 INFO:teuthology.orchestra.run.smithi165.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T11:08:28.455 INFO:teuthology.orchestra.run.smithi165.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-20T11:08:28.531 INFO:teuthology.orchestra.run.smithi165.stdout:(75/150): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-20T11:08:28.556 INFO:teuthology.orchestra.run.smithi165.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-20T11:08:28.581 INFO:teuthology.orchestra.run.smithi165.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-04-20T11:08:28.606 INFO:teuthology.orchestra.run.smithi165.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-20T11:08:28.632 INFO:teuthology.orchestra.run.smithi165.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-20T11:08:28.657 INFO:teuthology.orchestra.run.smithi165.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-04-20T11:08:28.755 INFO:teuthology.orchestra.run.smithi165.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-20T11:08:28.782 INFO:teuthology.orchestra.run.smithi165.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-20T11:08:28.860 INFO:teuthology.orchestra.run.smithi165.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.1 MB/s | 222 kB 00:00 2024-04-20T11:08:28.883 INFO:teuthology.orchestra.run.smithi165.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-04-20T11:08:28.908 INFO:teuthology.orchestra.run.smithi165.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 892 kB/s | 42 kB 00:00 2024-04-20T11:08:28.934 INFO:teuthology.orchestra.run.smithi165.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-20T11:08:28.976 INFO:teuthology.orchestra.run.smithi165.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-20T11:08:29.001 INFO:teuthology.orchestra.run.smithi165.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-20T11:08:29.026 INFO:teuthology.orchestra.run.smithi165.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-20T11:08:29.051 INFO:teuthology.orchestra.run.smithi165.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-20T11:08:29.127 INFO:teuthology.orchestra.run.smithi165.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-04-20T11:08:29.152 INFO:teuthology.orchestra.run.smithi165.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-20T11:08:29.177 INFO:teuthology.orchestra.run.smithi165.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-04-20T11:08:29.202 INFO:teuthology.orchestra.run.smithi165.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-20T11:08:29.228 INFO:teuthology.orchestra.run.smithi165.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-20T11:08:29.253 INFO:teuthology.orchestra.run.smithi165.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-20T11:08:29.280 INFO:teuthology.orchestra.run.smithi165.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-20T11:08:29.303 INFO:teuthology.orchestra.run.smithi165.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-20T11:08:29.328 INFO:teuthology.orchestra.run.smithi165.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-20T11:08:29.354 INFO:teuthology.orchestra.run.smithi165.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-20T11:08:29.379 INFO:teuthology.orchestra.run.smithi165.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-20T11:08:29.654 INFO:teuthology.orchestra.run.smithi165.stdout:(102/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-20T11:08:29.721 INFO:teuthology.orchestra.run.smithi165.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 61 kB/s | 22 kB 00:00 2024-04-20T11:08:29.788 INFO:teuthology.orchestra.run.smithi165.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 4.1 MB/s | 548 kB 00:00 2024-04-20T11:08:29.822 INFO:teuthology.orchestra.run.smithi165.stdout:(105/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-20T11:08:29.847 INFO:teuthology.orchestra.run.smithi165.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-04-20T11:08:29.881 INFO:teuthology.orchestra.run.smithi165.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 783 kB/s | 26 kB 00:00 2024-04-20T11:08:29.906 INFO:teuthology.orchestra.run.smithi165.stdout:(108/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-20T11:08:29.931 INFO:teuthology.orchestra.run.smithi165.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-04-20T11:08:29.956 INFO:teuthology.orchestra.run.smithi165.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-20T11:08:29.990 INFO:teuthology.orchestra.run.smithi165.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-20T11:08:30.032 INFO:teuthology.orchestra.run.smithi165.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-04-20T11:08:30.065 INFO:teuthology.orchestra.run.smithi165.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-04-20T11:08:30.107 INFO:teuthology.orchestra.run.smithi165.stdout:(114/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 838 kB 00:00 2024-04-20T11:08:30.132 INFO:teuthology.orchestra.run.smithi165.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-20T11:08:30.158 INFO:teuthology.orchestra.run.smithi165.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-20T11:08:30.183 INFO:teuthology.orchestra.run.smithi165.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-20T11:08:30.217 INFO:teuthology.orchestra.run.smithi165.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-20T11:08:30.242 INFO:teuthology.orchestra.run.smithi165.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-04-20T11:08:30.276 INFO:teuthology.orchestra.run.smithi165.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 602 kB/s | 35 kB 00:00 2024-04-20T11:08:30.301 INFO:teuthology.orchestra.run.smithi165.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-20T11:08:30.368 INFO:teuthology.orchestra.run.smithi165.stdout:(122/150): libarrow-9.0.0-10.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-20T11:08:30.393 INFO:teuthology.orchestra.run.smithi165.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-20T11:08:30.410 INFO:teuthology.orchestra.run.smithi165.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-20T11:08:30.427 INFO:teuthology.orchestra.run.smithi165.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 530 kB/s | 18 kB 00:00 2024-04-20T11:08:30.444 INFO:teuthology.orchestra.run.smithi165.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 690 kB/s | 23 kB 00:00 2024-04-20T11:08:30.469 INFO:teuthology.orchestra.run.smithi165.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-04-20T11:08:30.494 INFO:teuthology.orchestra.run.smithi165.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-20T11:08:30.545 INFO:teuthology.orchestra.run.smithi165.stdout:(129/150): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-20T11:08:30.570 INFO:teuthology.orchestra.run.smithi165.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-20T11:08:30.603 INFO:teuthology.orchestra.run.smithi165.stdout:(131/150): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-20T11:08:30.628 INFO:teuthology.orchestra.run.smithi165.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-04-20T11:08:30.654 INFO:teuthology.orchestra.run.smithi165.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-20T11:08:30.687 INFO:teuthology.orchestra.run.smithi165.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-20T11:08:30.713 INFO:teuthology.orchestra.run.smithi165.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-20T11:08:30.738 INFO:teuthology.orchestra.run.smithi165.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-20T11:08:30.763 INFO:teuthology.orchestra.run.smithi165.stdout:(137/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-04-20T11:08:30.788 INFO:teuthology.orchestra.run.smithi165.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-20T11:08:30.813 INFO:teuthology.orchestra.run.smithi165.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-20T11:08:30.839 INFO:teuthology.orchestra.run.smithi165.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-04-20T11:08:30.864 INFO:teuthology.orchestra.run.smithi165.stdout:(141/150): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-20T11:08:30.889 INFO:teuthology.orchestra.run.smithi165.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-20T11:08:30.914 INFO:teuthology.orchestra.run.smithi165.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-20T11:08:30.940 INFO:teuthology.orchestra.run.smithi165.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-20T11:08:30.965 INFO:teuthology.orchestra.run.smithi165.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-20T11:08:30.990 INFO:teuthology.orchestra.run.smithi165.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-20T11:08:31.015 INFO:teuthology.orchestra.run.smithi165.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-20T11:08:31.099 INFO:teuthology.orchestra.run.smithi165.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-20T11:08:31.458 INFO:teuthology.orchestra.run.smithi165.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 7.3 MB/s | 3.4 MB 00:00 2024-04-20T11:08:31.591 INFO:teuthology.orchestra.run.smithi165.stdout:(150/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-20T11:08:31.596 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T11:08:31.596 INFO:teuthology.orchestra.run.smithi165.stdout:Total 20 MB/s | 218 MB 00:11 2024-04-20T11:08:32.635 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T11:08:32.760 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T11:08:32.761 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T11:08:34.476 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T11:08:34.476 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T11:08:40.268 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T11:08:40.353 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T11:08:40.387 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T11:08:40.430 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T11:08:40.811 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T11:08:40.871 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:08:40.991 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T11:08:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:08:41.113 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:08:41.155 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:08:41.192 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T11:08:41.218 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T11:08:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T11:08:41.441 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T11:08:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:08:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:08:41.586 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:08:41.634 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T11:08:41.654 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:08:41.677 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T11:08:41.971 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:08:42.003 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T11:08:42.118 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T11:08:42.209 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T11:08:42.244 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T11:08:42.280 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T11:08:42.354 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T11:08:42.404 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T11:08:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T11:08:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T11:08:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T11:08:42.609 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T11:08:42.671 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T11:08:42.702 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T11:08:42.745 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T11:08:42.822 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T11:08:42.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T11:08:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T11:08:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T11:08:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T11:08:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T11:08:43.117 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T11:08:43.255 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T11:08:43.321 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T11:08:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T11:08:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T11:08:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T11:08:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T11:08:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T11:08:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T11:08:44.157 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T11:08:44.228 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T11:08:44.259 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T11:08:44.295 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T11:08:44.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T11:08:44.378 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T11:08:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T11:08:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T11:08:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T11:08:44.660 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T11:08:44.690 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T11:08:44.732 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T11:08:44.827 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T11:08:44.865 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T11:08:44.905 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T11:08:44.931 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T11:08:44.955 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T11:08:45.031 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T11:08:45.157 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T11:08:45.195 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T11:08:45.223 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T11:08:45.258 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T11:08:45.302 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T11:08:45.339 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T11:08:45.379 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T11:08:45.409 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T11:08:45.443 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T11:08:45.477 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T11:08:45.518 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T11:08:45.553 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T11:08:45.641 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T11:08:45.691 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T11:08:45.722 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T11:08:45.769 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T11:08:45.869 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T11:08:45.925 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T11:08:46.665 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T11:08:46.712 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T11:08:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T11:08:46.846 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T11:08:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T11:08:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T11:08:47.031 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T11:08:47.201 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T11:08:47.241 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T11:08:47.315 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T11:08:47.972 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T11:08:48.086 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T11:08:49.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T11:08:49.640 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T11:08:49.668 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T11:08:49.701 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T11:08:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T11:08:50.047 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:08:50.138 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T11:08:50.168 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T11:08:50.204 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T11:08:50.686 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T11:08:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:08:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T11:08:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T11:08:53.087 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:08:53.122 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:08:53.668 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T11:08:53.708 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:08:53.798 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T11:08:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T11:08:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T11:08:53.799 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:08:53.826 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:09:05.159 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-04-20T11:09:05.160 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:05.335 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:09:05.353 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T11:09:05.353 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:05.353 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T11:09:05.353 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:06.846 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:09:06.872 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T11:09:06.872 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:06.872 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T11:09:06.872 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:09:06.873 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T11:09:06.873 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:09:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T11:09:07.120 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:07.120 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T11:09:07.120 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:09:07.120 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T11:09:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:09:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T11:09:07.587 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:07.588 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T11:09:07.588 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:09:07.588 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T11:09:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:07.694 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T11:09:07.750 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T11:09:07.813 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T11:09:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T11:09:08.048 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T11:09:08.124 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T11:09:08.220 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T11:09:08.351 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T11:09:08.398 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T11:09:08.429 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T11:09:08.465 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T11:09:08.833 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:09:08.863 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T11:09:08.896 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T11:09:08.982 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T11:09:09.011 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:09:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T11:09:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:09:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T11:09:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:09:11.182 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T11:09:11.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T11:09:11.382 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:09:11.406 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T11:09:11.406 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:11.406 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T11:09:11.406 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:09:11.406 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T11:09:11.406 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:11.457 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:09:11.469 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T11:09:11.566 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T11:09:11.615 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T11:09:12.534 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T11:09:12.568 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:09:12.591 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T11:09:12.591 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:12.591 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T11:09:12.591 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:09:12.591 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T11:09:12.592 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:12.636 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:09:12.658 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T11:09:12.658 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:12.658 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T11:09:12.658 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:12.703 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:09:12.725 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T11:09:12.725 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:12.726 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T11:09:12.726 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:09:12.726 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T11:09:12.726 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:13.028 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:09:13.052 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T11:09:13.052 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T11:09:13.052 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T11:09:13.052 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:09:13.052 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T11:09:13.052 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:17.681 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T11:09:17.731 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T11:09:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T11:09:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T11:09:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T11:09:18.022 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T11:09:18.060 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T11:09:18.090 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T11:09:18.722 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:09:18.762 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:09:18.783 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T11:09:19.573 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T11:09:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T11:09:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T11:09:19.796 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T11:09:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T11:09:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:09:19.929 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T11:09:19.929 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:09:25.695 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:09:25.695 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T11:09:25.695 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T11:09:25.695 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T11:09:25.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T11:09:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T11:09:25.698 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T11:09:25.698 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T11:09:25.709 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T11:09:25.710 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T11:09:25.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T11:09:25.712 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T11:09:25.713 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T11:09:25.714 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T11:09:25.715 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T11:09:25.716 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T11:09:25.717 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T11:09:25.718 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T11:09:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T11:09:25.720 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T11:09:27.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T11:09:27.761 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:27.761 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.762 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T11:09:27.764 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T11:09:27.765 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T11:09:27.766 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T11:09:27.767 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T11:09:27.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T11:09:27.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T11:09:27.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T11:09:27.771 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T11:09:27.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T11:09:27.773 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:09:27.774 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T11:09:28.147 DEBUG:teuthology.parallel:result is None 2024-04-20T11:09:28.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:09:28.300 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T11:09:28.377 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T11:09:28.378 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T11:09:28.378 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T11:09:28.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:09:28.522 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T11:09:28.549 INFO:teuthology.orchestra.run.smithi078.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T11:09:28.549 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T11:09:28.549 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T11:09:28.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:09:28.887 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T11:09:28.909 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T11:09:28.910 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T11:09:28.910 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T11:09:28.912 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T11:09:28.912 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:09:28.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T11:09:29.011 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:09:29.011 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T11:09:29.041 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:09:29.042 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T11:09:29.067 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T11:09:29.067 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:09:29.068 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T11:09:29.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T11:09:29.252 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:09:29.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T11:09:29.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T11:09:29.349 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:09:29.349 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T11:09:29.373 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T11:09:29.436 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T11:09:29.437 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:09:29.437 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T11:09:29.536 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T11:09:29.663 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:09:29.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T11:09:29.686 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T11:09:29.751 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:09:29.751 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T11:09:29.774 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T11:09:29.837 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T11:09:29.837 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:09:29.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T11:09:29.928 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T11:09:30.007 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:09:30.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T11:09:30.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T11:09:30.093 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:09:30.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T11:09:30.117 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T11:09:30.180 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-20T11:09:30.301 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2}} 2024-04-20T11:09:30.301 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:09:30.302 INFO:tasks.cephadm:Cluster fsid is 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:09:30.302 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-20T11:09:30.302 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.44', 'mon.b': '172.21.15.78', 'mon.c': '172.21.15.165'} 2024-04-20T11:09:30.303 INFO:tasks.cephadm:First mon is mon.a on smithi044 2024-04-20T11:09:30.303 INFO:tasks.cephadm:First mgr is x 2024-04-20T11:09:30.303 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-20T11:09:30.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-04-20T11:09:30.403 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-04-20T11:09:30.426 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-04-20T11:09:30.450 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-20T11:09:30.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:09:30.669 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-20T11:09:30.807 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:09:30.808 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-20T11:09:30.809 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:31.339 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 11:09 /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:31.339 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:32.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 11:09 /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:32.545 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:33.851 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 11:09 /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:33.851 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:33.915 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:33.933 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T11:09:33.968 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-20T11:09:33.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T11:09:33.975 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T11:09:33.978 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T11:09:35.430 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T11:09:36.053 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T11:09:36.100 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T11:10:04.782 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T11:10:04.782 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:10:04.783 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T11:10:04.783 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-04-20T11:10:04.783 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T11:10:04.783 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T11:10:04.783 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T11:10:04.901 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-20T11:10:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:10:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T11:10:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-04-20T11:10:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T11:10:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-20T11:10:04.902 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-20T11:10:16.916 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:10:16.916 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T11:10:16.916 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T11:10:16.916 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-04-20T11:10:16.916 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T11:10:16.916 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:10:16.917 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:10:16.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-04-20T11:10:17.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-04-20T11:10:17.095 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-04-20T11:10:17.128 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-04-20T11:10:17.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-04-20T11:10:17.248 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-04-20T11:10:17.277 INFO:tasks.cephadm:Writing seed config... 2024-04-20T11:10:17.278 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-20T11:10:17.278 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-20T11:10:17.278 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-20T11:10:17.278 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-20T11:10:17.278 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-20T11:10:17.279 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-20T11:10:17.280 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-20T11:10:17.281 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-20T11:10:17.282 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-20T11:10:17.282 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-20T11:10:17.282 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-20T11:10:17.282 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-20T11:10:17.282 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-20T11:10:17.283 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:10:17.283 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-20T11:10:17.343 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 759aac0c-ff06-11ee-bc92-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-20T11:10:17.343 DEBUG:teuthology.orchestra.run.smithi044:mon.a> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service 2024-04-20T11:10:17.391 DEBUG:teuthology.orchestra.run.smithi044:mgr.x> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x.service 2024-04-20T11:10:17.437 INFO:tasks.cephadm:Bootstrapping... 2024-04-20T11:10:17.437 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.44 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:10:17.709 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T11:10:17.709 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '759aac0c-ff06-11ee-bc92-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.44', '--skip-admin-label'] 2024-04-20T11:10:17.710 INFO:teuthology.orchestra.run.smithi044.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-20T11:10:17.710 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying podman|docker is present... 2024-04-20T11:10:17.800 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T11:10:17.800 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying lvm2 is present... 2024-04-20T11:10:17.800 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying time synchronization is in place... 2024-04-20T11:10:17.820 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T11:10:17.820 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T11:10:17.841 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T11:10:17.841 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-04-20T11:10:17.866 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2024-04-20T11:10:17.888 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2024-04-20T11:10:17.889 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2024-04-20T11:10:17.889 INFO:teuthology.orchestra.run.smithi044.stdout:Repeating the final host check... 2024-04-20T11:10:17.953 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T11:10:17.953 INFO:teuthology.orchestra.run.smithi044.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-20T11:10:17.953 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl is present 2024-04-20T11:10:17.953 INFO:teuthology.orchestra.run.smithi044.stdout:lvcreate is present 2024-04-20T11:10:17.968 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T11:10:17.968 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T11:10:17.982 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T11:10:17.982 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-04-20T11:10:17.998 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2024-04-20T11:10:18.012 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2024-04-20T11:10:18.012 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2024-04-20T11:10:18.013 INFO:teuthology.orchestra.run.smithi044.stdout:Host looks OK 2024-04-20T11:10:18.013 INFO:teuthology.orchestra.run.smithi044.stdout:Cluster fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:18.013 INFO:teuthology.orchestra.run.smithi044.stdout:Acquiring lock 139712938089632 on /run/cephadm/759aac0c-ff06-11ee-bc92-c7b262605968.lock 2024-04-20T11:10:18.013 INFO:teuthology.orchestra.run.smithi044.stdout:Lock 139712938089632 acquired on /run/cephadm/759aac0c-ff06-11ee-bc92-c7b262605968.lock 2024-04-20T11:10:18.014 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 3300 ... 2024-04-20T11:10:18.014 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 6789 ... 2024-04-20T11:10:18.014 INFO:teuthology.orchestra.run.smithi044.stdout:Base mon IP(s) is [172.21.15.44:3300, 172.21.15.44:6789], mon addrv is [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-04-20T11:10:18.022 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.44 metric 100 2024-04-20T11:10:18.023 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.44 metric 100 2024-04-20T11:10:18.030 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-20T11:10:18.030 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-20T11:10:18.030 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-20T11:10:18.037 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-20T11:10:18.037 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-20T11:10:18.037 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T11:10:18.038 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-20T11:10:18.038 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-04-20T11:10:18.038 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T11:10:18.038 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2024-04-20T11:10:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2024-04-20T11:10:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-20T11:10:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-20T11:10:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T11:10:18.852 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T11:10:18.852 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T11:10:18.853 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Getting image source signatures 2024-04-20T11:10:18.853 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-20T11:10:18.853 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-20T11:10:18.854 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T11:10:18.854 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-20T11:10:20.165 INFO:teuthology.orchestra.run.smithi044.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T11:10:20.166 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T11:10:20.166 INFO:teuthology.orchestra.run.smithi044.stdout:Extracting ceph user uid/gid from container image... 2024-04-20T11:10:21.379 INFO:teuthology.orchestra.run.smithi044.stdout:stat: stdout 167 167 2024-04-20T11:10:21.380 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial keys... 2024-04-20T11:10:32.752 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQCnoiNmHqXJGhAA2smXdtNagAixZGPzdtQRpA== 2024-04-20T11:10:33.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQCpoiNmlDc8DRAAIoKdAZlOxdR2nYh3dqTG4A== 2024-04-20T11:10:35.095 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQCqoiNmaSnPHhAAhAktwOyGjznt7ALnX+/AOA== 2024-04-20T11:10:35.098 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial monmap... 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool for a [v2:172.21.15.44:3300,v1:172.21.15.44:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = quincy 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: set fsid to 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T11:10:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:10:36.220 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mon... 2024-04-20T11:10:37.614 INFO:teuthology.orchestra.run.smithi044.stdout:create mon.a on 2024-04-20T11:10:37.969 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-20T11:10:38.217 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-20T11:10:38.520 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-759aac0c-ff06-11ee-bc92-c7b262605968.target → /etc/systemd/system/ceph-759aac0c-ff06-11ee-bc92-c7b262605968.target. 2024-04-20T11:10:38.520 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-759aac0c-ff06-11ee-bc92-c7b262605968.target → /etc/systemd/system/ceph-759aac0c-ff06-11ee-bc92-c7b262605968.target. 2024-04-20T11:10:38.922 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a 2024-04-20T11:10:38.922 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service: Unit ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service not loaded. 2024-04-20T11:10:39.223 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-759aac0c-ff06-11ee-bc92-c7b262605968.target.wants/ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service → /etc/systemd/system/ceph-759aac0c-ff06-11ee-bc92-c7b262605968@.service. 2024-04-20T11:10:40.082 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:39 smithi044 podman[16471]: 2024-04-20 11:10:39.827675606 +0000 UTC m=+0.170459758 container create ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-20T11:10:40.082 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:40 smithi044 podman[16471]: 2024-04-20 11:10:40.080915412 +0000 UTC m=+0.423699538 container init ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T11:10:40.190 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T11:10:40.190 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-04-20T11:10:40.205 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T11:10:40.205 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-04-20T11:10:40.205 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-04-20T11:10:40.205 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T11:10:40.206 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon to start... 2024-04-20T11:10:40.206 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon... 2024-04-20T11:10:40.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:40 smithi044 podman[16471]: 2024-04-20 11:10:40.092687612 +0000 UTC m=+0.435471701 container start ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-20T11:10:40.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:40 smithi044 bash[16471]: ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 2024-04-20T11:10:40.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:40 smithi044 systemd[1]: Started Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:10:40.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:40 smithi044 ceph-mon[16490]: mkfs 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:40.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:40 smithi044 ceph-mon[16490]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T11:10:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: monmap epoch 1 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: last_changed 2024-04-20T11:10:35.587741+0000 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: min_mon_release 19 (squid) 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: election_strategy: 1 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: fsmap 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: mgrmap e1: no daemons active 2024-04-20T11:10:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:41 smithi044 ceph-mon[16490]: from='client.? 172.21.15.44:0/3239540191' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-20T11:10:41.596 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout cluster: 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout id: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout services: 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.765714s) 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout data: 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-20T11:10:41.597 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pgs: 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.846+0000 7f7c07e00700 1 Processor -- start 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.846+0000 7f7c07e00700 1 -- start start 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.847+0000 7f7c07e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0810aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.847+0000 7f7c07e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810b0c0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.847+0000 7f7c06a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0810aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.848+0000 7f7c06a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0810aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49952/0 (socket says 172.21.15.44:49952) 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.848+0000 7f7c06a00700 1 -- 172.21.15.44:0/222926623 learned_addr learned my addr 172.21.15.44:0/222926623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.848+0000 7f7c06a00700 1 -- 172.21.15.44:0/222926623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0810b880 con 0x7f7c0810a6d0 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.848+0000 7f7c06a00700 1 --2- 172.21.15.44:0/222926623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0810aaf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098a0 tx=0x7f7bf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea042190c2626772 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c05600700 1 -- 172.21.15.44:0/222926623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8004160 con 0x7f7c0810a6d0 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c05600700 1 -- 172.21.15.44:0/222926623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf8004300 con 0x7f7c0810a6d0 2024-04-20T11:10:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c05600700 1 -- 172.21.15.44:0/222926623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf80036a0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c07e00700 1 -- 172.21.15.44:0/222926623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 msgr2=0x7f7c0810aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c07e00700 1 --2- 172.21.15.44:0/222926623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0810aaf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098a0 tx=0x7f7bf8009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c07e00700 1 -- 172.21.15.44:0/222926623 shutdown_connections 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.849+0000 7f7c07e00700 1 --2- 172.21.15.44:0/222926623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0810aaf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.850+0000 7f7c07e00700 1 -- 172.21.15.44:0/222926623 >> 172.21.15.44:0/222926623 conn(0x7f7c0807db50 msgr2=0x7f7c081084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.850+0000 7f7c07e00700 1 -- 172.21.15.44:0/222926623 shutdown_connections 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.850+0000 7f7c07e00700 1 -- 172.21.15.44:0/222926623 wait complete. 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.851+0000 7f7c07e00700 1 Processor -- start 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.851+0000 7f7c07e00700 1 -- start start 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.851+0000 7f7c07e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.851+0000 7f7c07e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810bdb0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.852+0000 7f7c06a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:41.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.852+0000 7f7c06a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0819dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49956/0 (socket says 172.21.15.44:49956) 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.852+0000 7f7c06a00700 1 -- 172.21.15.44:0/3239540191 learned_addr learned my addr 172.21.15.44:0/3239540191 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.852+0000 7f7c06a00700 1 -- 172.21.15.44:0/3239540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819e250 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.852+0000 7f7c06a00700 1 --2- 172.21.15.44:0/3239540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0819dd10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c0810b360 tx=0x7f7bf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8003db0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf8011630 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819e4e0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8011930 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a11c0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7bf8011ad0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.853+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bf801b450 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.854+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c080640b0 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.858+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7bf8004260 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.938+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7c0802e160 con 0x7f7c0810a6d0 2024-04-20T11:10:41.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.939+0000 7f7bff400700 1 -- 172.21.15.44:0/3239540191 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7bf8025480 con 0x7f7c0810a6d0 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 msgr2=0x7f7c0819dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 --2- 172.21.15.44:0/3239540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0819dd10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c0810b360 tx=0x7f7bf8004750 comp rx=0 tx=0).stop 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 shutdown_connections 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 --2- 172.21.15.44:0/3239540191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c0810a6d0 0x7f7c0819dd10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 >> 172.21.15.44:0/3239540191 conn(0x7f7c0807db50 msgr2=0x7f7c08194930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 shutdown_connections 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:40.940+0000 7f7c07e00700 1 -- 172.21.15.44:0/3239540191 wait complete. 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:mon is available 2024-04-20T11:10:41.601 INFO:teuthology.orchestra.run.smithi044.stdout:Assimilating anything we can from ceph.conf... 2024-04-20T11:10:42.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:42 smithi044 ceph-mon[16490]: from='client.? 172.21.15.44:0/3177783870' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T11:10:42.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:42 smithi044 ceph-mon[16490]: from='client.? 172.21.15.44:0/3177783870' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-20T11:10:43.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:43.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T11:10:43.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T11:10:43.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.509+0000 7f480a200700 1 Processor -- start 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.509+0000 7f480a200700 1 -- start start 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.510+0000 7f480a200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f480410f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.510+0000 7f480a200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f480410faa0 con 0x7f480410f0b0 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.510+0000 7f4803400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f480410f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.510+0000 7f4803400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f480410f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49972/0 (socket says 172.21.15.44:49972) 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.510+0000 7f4803400700 1 -- 172.21.15.44:0/272385772 learned_addr learned my addr 172.21.15.44:0/272385772 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.511+0000 7f4803400700 1 -- 172.21.15.44:0/272385772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480410fe30 con 0x7f480410f0b0 2024-04-20T11:10:43.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.511+0000 7f4803400700 1 --2- 172.21.15.44:0/272385772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f480410f4d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47f40098a0 tx=0x7f47f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ab256acda051d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.512+0000 7f4802000700 1 -- 172.21.15.44:0/272385772 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4004160 con 0x7f480410f0b0 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.512+0000 7f4802000700 1 -- 172.21.15.44:0/272385772 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f4004300 con 0x7f480410f0b0 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.512+0000 7f4802000700 1 -- 172.21.15.44:0/272385772 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f40036a0 con 0x7f480410f0b0 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.512+0000 7f480a200700 1 -- 172.21.15.44:0/272385772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 msgr2=0x7f480410f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.512+0000 7f480a200700 1 --2- 172.21.15.44:0/272385772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f480410f4d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47f40098a0 tx=0x7f47f4009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.513+0000 7f480a200700 1 -- 172.21.15.44:0/272385772 shutdown_connections 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.513+0000 7f480a200700 1 --2- 172.21.15.44:0/272385772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f480410f4d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:43.282 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.513+0000 7f480a200700 1 -- 172.21.15.44:0/272385772 >> 172.21.15.44:0/272385772 conn(0x7f480407de10 msgr2=0x7f4804080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.513+0000 7f480a200700 1 -- 172.21.15.44:0/272385772 shutdown_connections 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.513+0000 7f480a200700 1 -- 172.21.15.44:0/272385772 wait complete. 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.514+0000 7f480a200700 1 Processor -- start 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.514+0000 7f480a200700 1 -- start start 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.514+0000 7f480a200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f48041a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.514+0000 7f480a200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4804110360 con 0x7f480410f0b0 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.515+0000 7f4803400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f48041a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.515+0000 7f4803400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f48041a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49986/0 (socket says 172.21.15.44:49986) 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.515+0000 7f4803400700 1 -- 172.21.15.44:0/3177783870 learned_addr learned my addr 172.21.15.44:0/3177783870 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.515+0000 7f4803400700 1 -- 172.21.15.44:0/3177783870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48041a27a0 con 0x7f480410f0b0 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.515+0000 7f4803400700 1 --2- 172.21.15.44:0/3177783870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f48041a2260 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000c00 tx=0x7f47f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4003db0 con 0x7f480410f0b0 2024-04-20T11:10:43.283 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f4011630 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48041a2a30 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4011950 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48041a5720 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f47f4011af0 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.516+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f47f401c450 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.517+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48040640b0 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.520+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f47f4032020 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.598+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f480402e110 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.609+0000 7f47fbe00700 1 -- 172.21.15.44:0/3177783870 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f47f4004260 con 0x7f480410f0b0 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.610+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 msgr2=0x7f48041a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.610+0000 7f480a200700 1 --2- 172.21.15.44:0/3177783870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f48041a2260 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000c00 tx=0x7f47f4004750 comp rx=0 tx=0).stop 2024-04-20T11:10:43.284 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.610+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 shutdown_connections 2024-04-20T11:10:43.285 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.610+0000 7f480a200700 1 --2- 172.21.15.44:0/3177783870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f480410f0b0 0x7f48041a2260 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:43.285 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.610+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 >> 172.21.15.44:0/3177783870 conn(0x7f480407de10 msgr2=0x7f4804198f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:43.285 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.610+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 shutdown_connections 2024-04-20T11:10:43.285 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:42.611+0000 7f480a200700 1 -- 172.21.15.44:0/3177783870 wait complete. 2024-04-20T11:10:43.285 INFO:teuthology.orchestra.run.smithi044.stdout:Generating new minimal ceph.conf... 2024-04-20T11:10:44.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:44 smithi044 ceph-mon[16490]: from='client.? 172.21.15.44:0/2883478842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:10:44.872 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.859+0000 7f32c6c00700 1 Processor -- start 2024-04-20T11:10:44.873 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.859+0000 7f32c6c00700 1 -- start start 2024-04-20T11:10:44.873 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.860+0000 7f32c6c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c010ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:44.873 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.860+0000 7f32c6c00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010b370 con 0x7f32c010a980 2024-04-20T11:10:44.873 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.860+0000 7f32bfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c010ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:44.873 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.860+0000 7f32bfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c010ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49998/0 (socket says 172.21.15.44:49998) 2024-04-20T11:10:44.873 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.860+0000 7f32bfe00700 1 -- 172.21.15.44:0/2637736398 learned_addr learned my addr 172.21.15.44:0/2637736398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.861+0000 7f32bfe00700 1 -- 172.21.15.44:0/2637736398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c010bb30 con 0x7f32c010a980 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.861+0000 7f32bfe00700 1 --2- 172.21.15.44:0/2637736398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c010ada0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f32b00098a0 tx=0x7f32b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72edcd1b68b41a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.861+0000 7f32bea00700 1 -- 172.21.15.44:0/2637736398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b0004160 con 0x7f32c010a980 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.861+0000 7f32bea00700 1 -- 172.21.15.44:0/2637736398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f32b0004300 con 0x7f32c010a980 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.861+0000 7f32bea00700 1 -- 172.21.15.44:0/2637736398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b0011c00 con 0x7f32c010a980 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 -- 172.21.15.44:0/2637736398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 msgr2=0x7f32c010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:44.878 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 --2- 172.21.15.44:0/2637736398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c010ada0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f32b00098a0 tx=0x7f32b0009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 -- 172.21.15.44:0/2637736398 shutdown_connections 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 --2- 172.21.15.44:0/2637736398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c010ada0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 -- 172.21.15.44:0/2637736398 >> 172.21.15.44:0/2637736398 conn(0x7f32c007dde0 msgr2=0x7f32c01084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 -- 172.21.15.44:0/2637736398 shutdown_connections 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.862+0000 7f32c6c00700 1 -- 172.21.15.44:0/2637736398 wait complete. 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.863+0000 7f32c6c00700 1 Processor -- start 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.864+0000 7f32c6c00700 1 -- start start 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.864+0000 7f32c6c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.864+0000 7f32c6c00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010eee0 con 0x7f32c010a980 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.864+0000 7f32bfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.864+0000 7f32bfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50012/0 (socket says 172.21.15.44:50012) 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.864+0000 7f32bfe00700 1 -- 172.21.15.44:0/2883478842 learned_addr learned my addr 172.21.15.44:0/2883478842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.865+0000 7f32bfe00700 1 -- 172.21.15.44:0/2883478842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c019ed90 con 0x7f32c010a980 2024-04-20T11:10:44.879 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.865+0000 7f32bfe00700 1 --2- 172.21.15.44:0/2883478842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c019e850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b0011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.865+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b0003b20 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.865+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f32b0003cc0 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.865+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b00249a0 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.865+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c019f020 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.866+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a1d10 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.866+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32b0022020 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.866+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32b002ddf0 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.867+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c010f060 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.870+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f32b0042e50 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.948+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f32c00640b0 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.949+0000 7f32bcc00700 1 -- 172.21.15.44:0/2883478842 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f32b0031030 con 0x7f32c010a980 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 msgr2=0x7f32c019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 --2- 172.21.15.44:0/2883478842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c019e850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b0011c80 comp rx=0 tx=0).stop 2024-04-20T11:10:44.880 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 shutdown_connections 2024-04-20T11:10:44.881 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 --2- 172.21.15.44:0/2883478842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c010a980 0x7f32c019e850 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:44.881 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 >> 172.21.15.44:0/2883478842 conn(0x7f32c007dde0 msgr2=0x7f32c0107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:44.881 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 shutdown_connections 2024-04-20T11:10:44.881 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:43.950+0000 7f32c6c00700 1 -- 172.21.15.44:0/2883478842 wait complete. 2024-04-20T11:10:44.882 INFO:teuthology.orchestra.run.smithi044.stdout:Restarting the monitor... 2024-04-20T11:10:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:44 smithi044 systemd[1]: Stopping Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:10:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a[16486]: 2024-04-20T11:10:45.376+0000 7fb86c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:10:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a[16486]: 2024-04-20T11:10:45.376+0000 7fb86c200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-20T11:10:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:45 smithi044 podman[16732]: 2024-04-20 11:10:45.40884035 +0000 UTC m=+0.162162913 container died ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-20T11:10:46.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:45 smithi044 podman[16732]: 2024-04-20 11:10:45.837700283 +0000 UTC m=+0.591022842 container cleanup ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-20T11:10:46.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:45 smithi044 bash[16732]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a 2024-04-20T11:10:46.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:46 smithi044 podman[16745]: 2024-04-20 11:10:46.08164732 +0000 UTC m=+0.649877560 container remove ccf227d3c3e95d352d4b6ac6a7286fb0fcfa6e5251647de56aafd100a68dcb54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:10:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:46 smithi044 systemd[1]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-20T11:10:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:46 smithi044 systemd[1]: Stopped Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:10:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:46 smithi044 systemd[1]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service: Consumed 2.022s CPU time. 2024-04-20T11:10:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:46 smithi044 systemd[1]: Starting Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:10:47.335 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:46 smithi044 podman[16835]: 2024-04-20 11:10:46.876377917 +0000 UTC m=+0.034599608 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:10:47.402 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 podman[16835]: 2024-04-20 11:10:47.02423256 +0000 UTC m=+0.182454276 container create fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:10:47.588 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 podman[16835]: 2024-04-20 11:10:47.529836737 +0000 UTC m=+0.688058493 container init fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:10:47.588 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 podman[16835]: 2024-04-20 11:10:47.540816002 +0000 UTC m=+0.699037703 container start fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T11:10:47.604 INFO:teuthology.orchestra.run.smithi044.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-20T11:10:47.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T11:10:47.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T11:10:47.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: pidfile_write: ignore empty --pid-file 2024-04-20T11:10:47.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: load: jerasure load: lrc 2024-04-20T11:10:47.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 bash[16835]: fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 2024-04-20T11:10:47.936 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: RocksDB version: 7.9.2 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Git sha 0 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: DB SUMMARY 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: DB Session ID: IFVNOQ8AS0V8BEDQUUSU 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: CURRENT file: CURRENT 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-20T11:10:47.937 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106190 ; 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.error_if_exists: 0 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.create_if_missing: 0 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.paranoid_checks: 1 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.env: 0x55c9dc853cc0 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.info_log: 0x55c9fc30fc60 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T11:10:47.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.statistics: (nil) 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.use_fsync: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_log_file_size: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.allow_fallocate: 1 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.use_direct_reads: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.db_log_dir: 2024-04-20T11:10:47.939 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.wal_dir: 2024-04-20T11:10:47.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T11:10:47.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T11:10:47.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T11:10:47.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T11:10:47.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.write_buffer_manager: 0x55c9fc2de1e0 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T11:10:47.941 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.unordered_write: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.row_cache: None 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.wal_filter: None 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.two_write_queues: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.wal_compression: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.atomic_flush: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T11:10:47.942 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.log_readahead_size: 0 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_background_jobs: 2 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_background_compactions: -1 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_subcompactions: 1 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T11:10:47.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_open_files: -1 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_background_flushes: -1 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Compression algorithms supported: 2024-04-20T11:10:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kZSTD supported: 0 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kXpressCompression supported: 0 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kZlibCompression supported: 1 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kSnappyCompression supported: 1 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kLZ4Compression supported: 1 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: kBZip2Compression supported: 0 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T11:10:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T11:10:47.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.merge_operator: 2024-04-20T11:10:47.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_filter: None 2024-04-20T11:10:47.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T11:10:47.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c9fc30fe20) 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_type: 0 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_index_type: 0 2024-04-20T11:10:47.947 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_shortening: 1 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: checksum: 4 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: no_block_cache: 0 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache: 0x55c9fc311090 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_options: 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: capacity : 536870912 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_shard_bits : 4 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: strict_capacity_limit : 0 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_compressed: (nil) 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: persistent_cache: (nil) 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size: 4096 2024-04-20T11:10:47.948 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size_deviation: 10 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_restart_interval: 16 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_block_restart_interval: 1 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: metadata_block_size: 4096 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: partition_filters: 0 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: use_delta_encoding: 1 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: filter_policy: bloomfilter 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: whole_key_filtering: 1 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: verify_compression: 0 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: format_version: 5 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_index_compression: 1 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_align: 0 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: max_auto_readahead_size: 262144 2024-04-20T11:10:47.949 INFO:journalctl@ceph.mon.a.smithi044.stdout: prepopulate_block_cache: 0 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout: initial_auto_readahead_size: 8192 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression: NoCompression 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.num_levels: 7 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T11:10:47.950 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T11:10:47.951 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T11:10:47.951 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T11:10:47.952 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T11:10:47.953 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T11:10:47.954 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T11:10:47.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.inplace_update_support: 0 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.bloom_locality: 0 2024-04-20T11:10:47.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.max_successive_merges: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.ttl: 2592000 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.enable_blob_files: false 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.min_blob_size: 0 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T11:10:47.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b0f1f3c5-24fe-4d39-9a28-82600bb92c79 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611447600537, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 systemd[1]: Started Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611447602689, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101188, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 99077, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14433, "raw_average_key_size": 49, "raw_value_size": 91282, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713611447, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b0f1f3c5-24fe-4d39-9a28-82600bb92c79", "db_session_id": "IFVNOQ8AS0V8BEDQUUSU", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713611447602865, "job": 1, "event": "recovery_finished"} 2024-04-20T11:10:47.958 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c9fc3fe000 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: DB pointer 0x55c9fc3e8000 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** DB Stats ** 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T11:10:47.959 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: L0 2/0 100.68 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: Sum 2/0 100.68 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 64.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T11:10:47.960 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative compaction: 0.00 GB write, 11.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval compaction: 0.00 GB write, 11.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache BinnedLRUCache@0x55c9fc311090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: starting mon.a rank 0 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:47.961 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???) e1 preinit fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).mds e1 new map 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).mds e1 print_map 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout: e1 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout: legacy client fscid: -1 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout: No filesystems configured 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-20T11:10:47.962 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: monmap epoch 1 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: last_changed 2024-04-20T11:10:35.587741+0000 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: min_mon_release 19 (squid) 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: election_strategy: 1 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: fsmap 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T11:10:47.963 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:47 smithi044 ceph-mon[16855]: mgrmap e1: no daemons active 2024-04-20T11:10:49.147 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.281+0000 7ff33e600700 1 Processor -- start 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.281+0000 7ff33e600700 1 -- start start 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.282+0000 7ff33e600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff338107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.282+0000 7ff33e600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff338107f50 con 0x7ff338107560 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.282+0000 7ff337400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff338107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.282+0000 7ff337400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff338107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50020/0 (socket says 172.21.15.44:50020) 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.282+0000 7ff337400700 1 -- 172.21.15.44:0/717444361 learned_addr learned my addr 172.21.15.44:0/717444361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.283+0000 7ff337400700 1 -- 172.21.15.44:0/717444361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338108760 con 0x7ff338107560 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.283+0000 7ff337400700 1 --2- 172.21.15.44:0/717444361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff338107980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff3280098a0 tx=0x7ff328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c6b16c2c3d4d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:49.155 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.284+0000 7ff336000700 1 -- 172.21.15.44:0/717444361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff328004160 con 0x7ff338107560 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.284+0000 7ff336000700 1 -- 172.21.15.44:0/717444361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff328004300 con 0x7ff338107560 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.284+0000 7ff336000700 1 -- 172.21.15.44:0/717444361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff328011c00 con 0x7ff338107560 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.284+0000 7ff33e600700 1 -- 172.21.15.44:0/717444361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 msgr2=0x7ff338107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.284+0000 7ff33e600700 1 --2- 172.21.15.44:0/717444361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff338107980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff3280098a0 tx=0x7ff328009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.285+0000 7ff33e600700 1 -- 172.21.15.44:0/717444361 shutdown_connections 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.285+0000 7ff33e600700 1 --2- 172.21.15.44:0/717444361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff338107980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.285+0000 7ff33e600700 1 -- 172.21.15.44:0/717444361 >> 172.21.15.44:0/717444361 conn(0x7ff338102b40 msgr2=0x7ff338104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.285+0000 7ff33e600700 1 -- 172.21.15.44:0/717444361 shutdown_connections 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.285+0000 7ff33e600700 1 -- 172.21.15.44:0/717444361 wait complete. 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.286+0000 7ff33e600700 1 Processor -- start 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.286+0000 7ff33e600700 1 -- start start 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.286+0000 7ff33e600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff33819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.286+0000 7ff33e600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff33807b150 con 0x7ff338107560 2024-04-20T11:10:49.156 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.287+0000 7ff337400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff33819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.287+0000 7ff337400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff33819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50026/0 (socket says 172.21.15.44:50026) 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.287+0000 7ff337400700 1 -- 172.21.15.44:0/1706840210 learned_addr learned my addr 172.21.15.44:0/1706840210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff337400700 1 -- 172.21.15.44:0/1706840210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33819a9c0 con 0x7ff338107560 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff337400700 1 --2- 172.21.15.44:0/1706840210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff33819a480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff328011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff328003b20 con 0x7ff338107560 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33819ac50 con 0x7ff338107560 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff328003cc0 con 0x7ff338107560 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33819d940 con 0x7ff338107560 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.288+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff328024940 con 0x7ff338107560 2024-04-20T11:10:49.157 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.289+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff328022020 con 0x7ff338107560 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.289+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff32802ddf0 con 0x7ff338107560 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.290+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3380640b0 con 0x7ff338107560 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.293+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ff3280042a0 con 0x7ff338107560 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.382+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff3381939e0 con 0x7ff338107560 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.385+0000 7ff32fe00700 1 -- 172.21.15.44:0/1706840210 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff32801ad10 con 0x7ff338107560 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 msgr2=0x7ff33819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 --2- 172.21.15.44:0/1706840210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff33819a480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff328011c80 comp rx=0 tx=0).stop 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 shutdown_connections 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 --2- 172.21.15.44:0/1706840210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff338107560 0x7ff33819a480 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 >> 172.21.15.44:0/1706840210 conn(0x7ff338102b40 msgr2=0x7ff338190aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 shutdown_connections 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:48.387+0000 7ff33e600700 1 -- 172.21.15.44:0/1706840210 wait complete. 2024-04-20T11:10:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-20T11:10:49.186 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:10:49.186 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mgr... 2024-04-20T11:10:49.188 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-20T11:10:49.188 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-20T11:10:49.628 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x 2024-04-20T11:10:49.628 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x.service: Unit ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x.service not loaded. 2024-04-20T11:10:49.649 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:49 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1706840210' entity='client.admin' 2024-04-20T11:10:49.893 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-759aac0c-ff06-11ee-bc92-c7b262605968.target.wants/ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-759aac0c-ff06-11ee-bc92-c7b262605968@.service. 2024-04-20T11:10:50.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:49 smithi044 systemd[1]: Starting Ceph mgr.x for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:10:50.618 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 podman[17078]: 2024-04-20 11:10:50.207060525 +0000 UTC m=+0.032951763 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:10:50.619 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 podman[17078]: 2024-04-20 11:10:50.308258486 +0000 UTC m=+0.134149729 container create afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:10:50.619 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 podman[17078]: 2024-04-20 11:10:50.536672025 +0000 UTC m=+0.362563303 container init afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T11:10:50.619 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 podman[17078]: 2024-04-20 11:10:50.546876795 +0000 UTC m=+0.372768071 container start afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T11:10:50.619 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 bash[17078]: afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf 2024-04-20T11:10:50.653 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T11:10:50.653 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-04-20T11:10:50.673 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T11:10:50.673 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-04-20T11:10:50.674 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-04-20T11:10:50.674 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T11:10:50.700 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T11:10:50.700 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-04-20T11:10:50.722 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T11:10:50.722 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-04-20T11:10:50.722 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-04-20T11:10:50.722 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-20T11:10:50.722 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr to start... 2024-04-20T11:10:50.722 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr... 2024-04-20T11:10:50.888 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 systemd[1]: Started Ceph mgr.x for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:10:50.889 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:50.809+0000 7fb8fc0fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:10:51.190 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:51.064+0000 7fb8fc0fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:10:51.439 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:51.200+0000 7fb8fc0fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:10:51.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:51 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2293293606' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T11:10:51.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:51.624+0000 7fb8fc0fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:10:52.083 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:51.739+0000 7fb8fc0fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:10:52.094 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:52.094 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:52.100 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T11:10:52.101 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T11:10:52.102 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:10:52.103 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.104 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T11:10:52.104 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:52.104 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T11:10:40.177000+0000", 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.340+0000 7f1863400700 1 Processor -- start 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.340+0000 7f1863400700 1 -- start start 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.341+0000 7f1863400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c1073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.341+0000 7f1863400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f185c07be30 con 0x7f185c106fa0 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.341+0000 7f1860a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c1073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.342+0000 7f1860a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c1073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49340/0 (socket says 172.21.15.44:49340) 2024-04-20T11:10:52.109 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.342+0000 7f1860a00700 1 -- 172.21.15.44:0/1796573904 learned_addr learned my addr 172.21.15.44:0/1796573904 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.342+0000 7f1860a00700 1 -- 172.21.15.44:0/1796573904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c107900 con 0x7f185c106fa0 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.342+0000 7f1860a00700 1 --2- 172.21.15.44:0/1796573904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c1073c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f18500098a0 tx=0x7f1850009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd55f38db696855 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.343+0000 7f185b400700 1 -- 172.21.15.44:0/1796573904 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850004160 con 0x7f185c106fa0 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.343+0000 7f185b400700 1 -- 172.21.15.44:0/1796573904 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1850004300 con 0x7f185c106fa0 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.343+0000 7f185b400700 1 -- 172.21.15.44:0/1796573904 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850011c00 con 0x7f185c106fa0 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.343+0000 7f1863400700 1 -- 172.21.15.44:0/1796573904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 msgr2=0x7f185c1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.343+0000 7f1863400700 1 --2- 172.21.15.44:0/1796573904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c1073c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f18500098a0 tx=0x7f1850009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.344+0000 7f1863400700 1 -- 172.21.15.44:0/1796573904 shutdown_connections 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.344+0000 7f1863400700 1 --2- 172.21.15.44:0/1796573904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c1073c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.344+0000 7f1863400700 1 -- 172.21.15.44:0/1796573904 >> 172.21.15.44:0/1796573904 conn(0x7f185c102be0 msgr2=0x7f185c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.344+0000 7f1863400700 1 -- 172.21.15.44:0/1796573904 shutdown_connections 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.344+0000 7f1863400700 1 -- 172.21.15.44:0/1796573904 wait complete. 2024-04-20T11:10:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.345+0000 7f1863400700 1 Processor -- start 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.345+0000 7f1863400700 1 -- start start 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.345+0000 7f1863400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.345+0000 7f1863400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f185c1089f0 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.346+0000 7f1860a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.346+0000 7f1860a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49354/0 (socket says 172.21.15.44:49354) 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.346+0000 7f1860a00700 1 -- 172.21.15.44:0/2293293606 learned_addr learned my addr 172.21.15.44:0/2293293606 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.346+0000 7f1860a00700 1 -- 172.21.15.44:0/2293293606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19ece0 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.346+0000 7f1860a00700 1 --2- 172.21.15.44:0/2293293606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c19e7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1850011bd0 tx=0x7f1850011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.346+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850003b20 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.347+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1850003cc0 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.347+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19ef70 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.347+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850024930 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.347+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c1a1c60 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.347+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1850022020 con 0x7f185c106fa0 2024-04-20T11:10:52.111 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.347+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1850019620 con 0x7f185c106fa0 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.348+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c0640b0 con 0x7f185c106fa0 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.352+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f185001aca0 con 0x7f185c106fa0 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.432+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f185c02e190 con 0x7f185c106fa0 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.433+0000 7f1859600700 1 -- 172.21.15.44:0/2293293606 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f18500198c0 con 0x7f185c106fa0 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.434+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 msgr2=0x7f185c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.434+0000 7f1863400700 1 --2- 172.21.15.44:0/2293293606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c19e7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1850011bd0 tx=0x7f1850011cb0 comp rx=0 tx=0).stop 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.435+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 shutdown_connections 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.435+0000 7f1863400700 1 --2- 172.21.15.44:0/2293293606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f185c106fa0 0x7f185c19e7a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.435+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 >> 172.21.15.44:0/2293293606 conn(0x7f185c102be0 msgr2=0x7f185c195400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.435+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 shutdown_connections 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:51.435+0000 7f1863400700 1 -- 172.21.15.44:0/2293293606 wait complete. 2024-04-20T11:10:52.112 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (1/15)... 2024-04-20T11:10:53.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:53.375+0000 7fb8fc0fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:10:53.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:53.606+0000 7fb8fc0fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:10:54.194 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:53.824+0000 7fb8fc0fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:10:54.456 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:54.230+0000 7fb8fc0fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:10:54.456 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:54.365+0000 7fb8fc0fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:10:54.728 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:54.642+0000 7fb8fc0fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:10:55.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:54 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3040510174' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T11:10:55.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:54.913+0000 7fb8fc0fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-04-20T11:10:55.606 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T11:10:55.607 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T11:10:55.616 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T11:10:55.617 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T11:10:40.177000+0000", 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.727+0000 7f9301600700 1 Processor -- start 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.727+0000 7f9301600700 1 -- start start 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.727+0000 7f9301600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc108af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.727+0000 7f9301600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1090c0 con 0x7f92fc1086d0 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.728+0000 7f92faa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc108af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.728+0000 7f92faa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc108af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49360/0 (socket says 172.21.15.44:49360) 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.728+0000 7f92faa00700 1 -- 172.21.15.44:0/3935989974 learned_addr learned my addr 172.21.15.44:0/3935989974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.729+0000 7f92faa00700 1 -- 172.21.15.44:0/3935989974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc109880 con 0x7f92fc1086d0 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.729+0000 7f92faa00700 1 --2- 172.21.15.44:0/3935989974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc108af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0098a0 tx=0x7f92ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498464c77fd88a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.729+0000 7f92f9600700 1 -- 172.21.15.44:0/3935989974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec004160 con 0x7f92fc1086d0 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.729+0000 7f92f9600700 1 -- 172.21.15.44:0/3935989974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f92ec004300 con 0x7f92fc1086d0 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.729+0000 7f92f9600700 1 -- 172.21.15.44:0/3935989974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec011c00 con 0x7f92fc1086d0 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 -- 172.21.15.44:0/3935989974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 msgr2=0x7f92fc108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 --2- 172.21.15.44:0/3935989974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc108af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0098a0 tx=0x7f92ec009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 -- 172.21.15.44:0/3935989974 shutdown_connections 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 --2- 172.21.15.44:0/3935989974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc108af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 -- 172.21.15.44:0/3935989974 >> 172.21.15.44:0/3935989974 conn(0x7f92fc103cf0 msgr2=0x7f92fc106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 -- 172.21.15.44:0/3935989974 shutdown_connections 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.730+0000 7f9301600700 1 -- 172.21.15.44:0/3935989974 wait complete. 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.731+0000 7f9301600700 1 Processor -- start 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.731+0000 7f9301600700 1 -- start start 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.732+0000 7f9301600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.732+0000 7f9301600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10cc30 con 0x7f92fc1086d0 2024-04-20T11:10:55.620 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.732+0000 7f92faa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.732+0000 7f92faa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49370/0 (socket says 172.21.15.44:49370) 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.732+0000 7f92faa00700 1 -- 172.21.15.44:0/3040510174 learned_addr learned my addr 172.21.15.44:0/3040510174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.732+0000 7f92faa00700 1 -- 172.21.15.44:0/3040510174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc19cbc0 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.733+0000 7f92faa00700 1 --2- 172.21.15.44:0/3040510174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc19c680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.733+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec003b20 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.733+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f92ec003cc0 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.733+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc19ce50 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.733+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92ec0249a0 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.733+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc19fb40 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.734+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f92ec022020 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.734+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f92ec0193c0 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.734+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc10cdb0 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.737+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f92ec042e50 con 0x7f92fc1086d0 2024-04-20T11:10:55.621 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.814+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f92fc1a0070 con 0x7f92fc1086d0 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.815+0000 7f92f3400700 1 -- 172.21.15.44:0/3040510174 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f92ec031030 con 0x7f92fc1086d0 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.816+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 msgr2=0x7f92fc19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.816+0000 7f9301600700 1 --2- 172.21.15.44:0/3040510174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc19c680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92ec000c00 tx=0x7f92ec011c80 comp rx=0 tx=0).stop 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.816+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 shutdown_connections 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.816+0000 7f9301600700 1 --2- 172.21.15.44:0/3040510174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92fc1086d0 0x7f92fc19c680 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.816+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 >> 172.21.15.44:0/3040510174 conn(0x7f92fc103cf0 msgr2=0x7f92fc1047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.817+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 shutdown_connections 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:54.817+0000 7f9301600700 1 -- 172.21.15.44:0/3040510174 wait complete. 2024-04-20T11:10:55.622 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (2/15)... 2024-04-20T11:10:55.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:55 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:55.735+0000 7fb8fc0fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:10:56.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:56.299+0000 7fb8fc0fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:10:56.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:56.426+0000 7fb8fc0fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:10:57.194 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:56.695+0000 7fb8fc0fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:10:57.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:57.360+0000 7fb8fc0fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:10:57.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:57.485+0000 7fb8fc0fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:10:58.042 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:57.817+0000 7fb8fc0fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:10:58.042 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:58.008+0000 7fb8fc0fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:10:58.292 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:58.139+0000 7fb8fc0fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:10:58.645 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3677816209' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T11:10:58.645 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:58.296+0000 7fb8fc0fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:10:58.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:58.685+0000 7fb8fc0fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:10:58.984 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-04-20T11:10:58.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T11:10:58.986 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T11:10:58.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T11:10:58.997 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:10:58.998 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T11:10:40.177000+0000", 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.197+0000 7ffba8e00700 1 Processor -- start 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.198+0000 7ffba8e00700 1 -- start start 2024-04-20T11:10:58.999 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.198+0000 7ffba8e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.198+0000 7ffba8e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffba407c800 con 0x7ffba407ddc0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.198+0000 7ffba3400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.199+0000 7ffba3400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49384/0 (socket says 172.21.15.44:49384) 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.199+0000 7ffba3400700 1 -- 172.21.15.44:0/3632565986 learned_addr learned my addr 172.21.15.44:0/3632565986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.199+0000 7ffba3400700 1 -- 172.21.15.44:0/3632565986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba407c980 con 0x7ffba407ddc0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.199+0000 7ffba3400700 1 --2- 172.21.15.44:0/3632565986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba407c2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffb940098a0 tx=0x7ffb94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9886538ad3cdd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.199+0000 7ffba2000700 1 -- 172.21.15.44:0/3632565986 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb94004160 con 0x7ffba407ddc0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba2000700 1 -- 172.21.15.44:0/3632565986 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ffb94004300 con 0x7ffba407ddc0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba2000700 1 -- 172.21.15.44:0/3632565986 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb94011c00 con 0x7ffba407ddc0 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 -- 172.21.15.44:0/3632565986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 msgr2=0x7ffba407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 --2- 172.21.15.44:0/3632565986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba407c2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffb940098a0 tx=0x7ffb94009bb0 comp rx=0 tx=0).stop 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 -- 172.21.15.44:0/3632565986 shutdown_connections 2024-04-20T11:10:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 --2- 172.21.15.44:0/3632565986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba407c2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 -- 172.21.15.44:0/3632565986 >> 172.21.15.44:0/3632565986 conn(0x7ffba4103af0 msgr2=0x7ffba4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 -- 172.21.15.44:0/3632565986 shutdown_connections 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.200+0000 7ffba8e00700 1 -- 172.21.15.44:0/3632565986 wait complete. 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.201+0000 7ffba8e00700 1 Processor -- start 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.202+0000 7ffba8e00700 1 -- start start 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.202+0000 7ffba8e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.202+0000 7ffba8e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffba410eb00 con 0x7ffba407ddc0 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.202+0000 7ffba3400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.202+0000 7ffba3400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49394/0 (socket says 172.21.15.44:49394) 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.202+0000 7ffba3400700 1 -- 172.21.15.44:0/3677816209 learned_addr learned my addr 172.21.15.44:0/3677816209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.203+0000 7ffba3400700 1 -- 172.21.15.44:0/3677816209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba419ec50 con 0x7ffba407ddc0 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.203+0000 7ffba3400700 1 --2- 172.21.15.44:0/3677816209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba419e710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb94011bd0 tx=0x7ffb94011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:10:59.001 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.203+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb94003970 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.203+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ffb94003b10 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.203+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb940245a0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.203+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba419eee0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.204+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba41a1bd0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.204+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb94024be0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.204+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb94019700 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.205+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba40640b0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.207+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ffb94003cc0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.288+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffba41a1da0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.289+0000 7ffb9be00700 1 -- 172.21.15.44:0/3677816209 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7ffb9401a9a0 con 0x7ffba407ddc0 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 msgr2=0x7ffba419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:10:59.002 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 --2- 172.21.15.44:0/3677816209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba419e710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb94011bd0 tx=0x7ffb94011cb0 comp rx=0 tx=0).stop 2024-04-20T11:10:59.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 shutdown_connections 2024-04-20T11:10:59.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 --2- 172.21.15.44:0/3677816209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba407ddc0 0x7ffba419e710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:10:59.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 >> 172.21.15.44:0/3677816209 conn(0x7ffba4103af0 msgr2=0x7ffba4104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:10:59.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 shutdown_connections 2024-04-20T11:10:59.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:10:58.290+0000 7ffba8e00700 1 -- 172.21.15.44:0/3677816209 wait complete. 2024-04-20T11:10:59.003 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (3/15)... 2024-04-20T11:10:59.773 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:59.479+0000 7fb8fc0fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:11:00.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:10:59.772+0000 7fb8fc0fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:11:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: Activating manager daemon x 2024-04-20T11:11:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: mgrmap e2: x(active, starting, since 0.0118126s) 2024-04-20T11:11:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: Manager daemon x is now available 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' 2024-04-20T11:11:00.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:10:59 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T11:11:01.153 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:00 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' 2024-04-20T11:11:01.153 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:00 smithi044 ceph-mon[16855]: from='mgr.14100 172.21.15.44:0/209620529' entity='mgr.x' 2024-04-20T11:11:01.153 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:00 smithi044 ceph-mon[16855]: mgrmap e3: x(active, since 1.02217s) 2024-04-20T11:11:03.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:11:03.003 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T11:11:03.004 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T11:11:03.006 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T11:11:03.007 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T11:11:03.009 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T11:11:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T11:11:03.014 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T11:11:03.014 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T11:10:40.177000+0000", 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.612+0000 7feb74200700 1 Processor -- start 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.612+0000 7feb74200700 1 -- start start 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.612+0000 7feb74200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c10f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.612+0000 7feb74200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10faa0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.613+0000 7feb71800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c10f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.613+0000 7feb71800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c10f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58000/0 (socket says 172.21.15.44:58000) 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.613+0000 7feb71800700 1 -- 172.21.15.44:0/2659768921 learned_addr learned my addr 172.21.15.44:0/2659768921 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.614+0000 7feb71800700 1 -- 172.21.15.44:0/2659768921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c10fe30 con 0x7feb6c10f0b0 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.614+0000 7feb71800700 1 --2- 172.21.15.44:0/2659768921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c10f4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb600098a0 tx=0x7feb60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbe0c2e192c1393 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:03.015 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.615+0000 7feb6be00700 1 -- 172.21.15.44:0/2659768921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60004160 con 0x7feb6c10f0b0 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.615+0000 7feb6be00700 1 -- 172.21.15.44:0/2659768921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7feb60004300 con 0x7feb6c10f0b0 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.615+0000 7feb6be00700 1 -- 172.21.15.44:0/2659768921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60011c00 con 0x7feb6c10f0b0 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 -- 172.21.15.44:0/2659768921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 msgr2=0x7feb6c10f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 --2- 172.21.15.44:0/2659768921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c10f4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb600098a0 tx=0x7feb60009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 -- 172.21.15.44:0/2659768921 shutdown_connections 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 --2- 172.21.15.44:0/2659768921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c10f4d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 -- 172.21.15.44:0/2659768921 >> 172.21.15.44:0/2659768921 conn(0x7feb6c07de10 msgr2=0x7feb6c080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 -- 172.21.15.44:0/2659768921 shutdown_connections 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.616+0000 7feb74200700 1 -- 172.21.15.44:0/2659768921 wait complete. 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.617+0000 7feb74200700 1 Processor -- start 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.617+0000 7feb74200700 1 -- start start 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.617+0000 7feb74200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:03.016 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.617+0000 7feb74200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c1131e0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.618+0000 7feb71800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.618+0000 7feb71800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c1a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58002/0 (socket says 172.21.15.44:58002) 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.618+0000 7feb71800700 1 -- 172.21.15.44:0/2062631550 learned_addr learned my addr 172.21.15.44:0/2062631550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.618+0000 7feb71800700 1 -- 172.21.15.44:0/2062631550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1a30b0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.618+0000 7feb71800700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c1a2b70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb60003fb0 tx=0x7feb60011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.619+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb600039c0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.619+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7feb60003b60 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.619+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb6001a6d0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.619+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a3340 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.619+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a6030 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.620+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7feb60024430 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.620+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c113360 con 0x7feb6c10f0b0 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.620+0000 7feb6a000700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7feb54039560 0x7feb5403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:03.017 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.620+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb6004d7e0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.621+0000 7feb70e00700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7feb54039560 0x7feb5403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.621+0000 7feb70e00700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7feb54039560 0x7feb5403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb58006fd0 tx=0x7feb58006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.627+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7feb60003d10 con 0x7feb6c10f0b0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.839+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7feb6002d3f0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.932+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feb6c1a65e0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.933+0000 7feb6a000700 1 -- 172.21.15.44:0/2062631550 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7feb60044fd0 con 0x7feb6c10f0b0 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7feb54039560 msgr2=0x7feb5403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7feb54039560 0x7feb5403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb58006fd0 tx=0x7feb58006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 msgr2=0x7feb6c1a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c1a2b70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb60003fb0 tx=0x7feb60011bd0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 shutdown_connections 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7feb54039560 0x7feb5403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.018 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 --2- 172.21.15.44:0/2062631550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb6c10f0b0 0x7feb6c1a2b70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:03.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 >> 172.21.15.44:0/2062631550 conn(0x7feb6c07de10 msgr2=0x7feb6c07f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:03.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 shutdown_connections 2024-04-20T11:11:03.019 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:01.937+0000 7feb74200700 1 -- 172.21.15.44:0/2062631550 wait complete. 2024-04-20T11:11:03.019 INFO:teuthology.orchestra.run.smithi044.stdout:mgr is available 2024-04-20T11:11:03.162 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:02 smithi044 ceph-mon[16855]: mgrmap e4: x(active, since 2s) 2024-04-20T11:11:03.162 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:02 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2062631550' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T11:11:04.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:03 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3340367359' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T11:11:04.544 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-04-20T11:11:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T11:11:04.547 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T11:11:04.547 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.617+0000 7f0111a00700 1 Processor -- start 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.617+0000 7f0111a00700 1 -- start start 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.617+0000 7f0111a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.617+0000 7f0111a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f010c107f50 con 0x7f010c107560 2024-04-20T11:11:04.553 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.618+0000 7f010aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.618+0000 7f010aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58010/0 (socket says 172.21.15.44:58010) 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.618+0000 7f010aa00700 1 -- 172.21.15.44:0/1374633491 learned_addr learned my addr 172.21.15.44:0/1374633491 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.618+0000 7f010aa00700 1 -- 172.21.15.44:0/1374633491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c108760 con 0x7f010c107560 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.618+0000 7f010aa00700 1 --2- 172.21.15.44:0/1374633491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c107980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0098a0 tx=0x7f00fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef48d0e56e202c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.619+0000 7f010a000700 1 -- 172.21.15.44:0/1374633491 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00fc004160 con 0x7f010c107560 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.619+0000 7f010a000700 1 -- 172.21.15.44:0/1374633491 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f00fc004300 con 0x7f010c107560 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.619+0000 7f010a000700 1 -- 172.21.15.44:0/1374633491 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00fc011c00 con 0x7f010c107560 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.619+0000 7f0111a00700 1 -- 172.21.15.44:0/1374633491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 msgr2=0x7f010c107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.619+0000 7f0111a00700 1 --2- 172.21.15.44:0/1374633491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c107980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0098a0 tx=0x7f00fc009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.620+0000 7f0111a00700 1 -- 172.21.15.44:0/1374633491 shutdown_connections 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.620+0000 7f0111a00700 1 --2- 172.21.15.44:0/1374633491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c107980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.620+0000 7f0111a00700 1 -- 172.21.15.44:0/1374633491 >> 172.21.15.44:0/1374633491 conn(0x7f010c102b40 msgr2=0x7f010c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:04.554 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.620+0000 7f0111a00700 1 -- 172.21.15.44:0/1374633491 shutdown_connections 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.620+0000 7f0111a00700 1 -- 172.21.15.44:0/1374633491 wait complete. 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.621+0000 7f0111a00700 1 Processor -- start 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.621+0000 7f0111a00700 1 -- start start 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.621+0000 7f0111a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c192780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.621+0000 7f0111a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f010c10bb10 con 0x7f010c107560 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.622+0000 7f010aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c192780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.622+0000 7f010aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c192780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58020/0 (socket says 172.21.15.44:58020) 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.622+0000 7f010aa00700 1 -- 172.21.15.44:0/3340367359 learned_addr learned my addr 172.21.15.44:0/3340367359 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.622+0000 7f010aa00700 1 -- 172.21.15.44:0/3340367359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c192cc0 con 0x7f010c107560 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.622+0000 7f010aa00700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c192780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000c00 tx=0x7f00fc011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.623+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00fc003b20 con 0x7f010c107560 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.623+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f00fc003cc0 con 0x7f010c107560 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.623+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c190eb0 con 0x7f010c107560 2024-04-20T11:11:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.623+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00fc0249a0 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.623+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1913f0 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.624+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c10bc90 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.624+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f00fc022020 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.625+0000 7f0103e00700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f00f00395b0 0x7f00f003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.625+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f00fc04d300 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.625+0000 7f0102000700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f00f00395b0 0x7f00f003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.626+0000 7f0102000700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f00f00395b0 0x7f00f003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006fd0 tx=0x7f00f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.631+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f00fc019dd0 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.859+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f010c0640b0 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.860+0000 7f0103e00700 1 -- 172.21.15.44:0/3340367359 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f00fc02c560 con 0x7f010c107560 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f00f00395b0 msgr2=0x7f00f003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f00f00395b0 0x7f00f003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006fd0 tx=0x7f00f4006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:04.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 msgr2=0x7f010c192780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c192780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000c00 tx=0x7f00fc011c80 comp rx=0 tx=0).stop 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 shutdown_connections 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f00f00395b0 0x7f00f003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 --2- 172.21.15.44:0/3340367359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f010c107560 0x7f010c192780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.863+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 >> 172.21.15.44:0/3340367359 conn(0x7f010c102b40 msgr2=0x7f010c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.864+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 shutdown_connections 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:03.864+0000 7f0111a00700 1 -- 172.21.15.44:0/3340367359 wait complete. 2024-04-20T11:11:04.557 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling cephadm module... 2024-04-20T11:11:05.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: ignoring --setuser ceph since I am not root 2024-04-20T11:11:05.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: ignoring --setgroup ceph since I am not root 2024-04-20T11:11:05.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:05 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1974437215' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-20T11:11:06.025 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:05.774+0000 7f1be52dc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:11:06.026 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:06.024+0000 7f1be52dc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:11:06.210 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.136+0000 7fabe2000700 1 Processor -- start 2024-04-20T11:11:06.211 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.136+0000 7fabe2000700 1 -- start start 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.137+0000 7fabe2000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc10f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.137+0000 7fabe2000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc10faa0 con 0x7fabdc10f0b0 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.137+0000 7fabdb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc10f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.138+0000 7fabdb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc10f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58028/0 (socket says 172.21.15.44:58028) 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.138+0000 7fabdb400700 1 -- 172.21.15.44:0/2777913283 learned_addr learned my addr 172.21.15.44:0/2777913283 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.138+0000 7fabdb400700 1 -- 172.21.15.44:0/2777913283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc10fe30 con 0x7fabdc10f0b0 2024-04-20T11:11:06.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.138+0000 7fabdb400700 1 --2- 172.21.15.44:0/2777913283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc10f4d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0098a0 tx=0x7fabcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e500c1e195722cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:06.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.139+0000 7fabda000700 1 -- 172.21.15.44:0/2777913283 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabcc004160 con 0x7fabdc10f0b0 2024-04-20T11:11:06.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.139+0000 7fabda000700 1 -- 172.21.15.44:0/2777913283 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fabcc004300 con 0x7fabdc10f0b0 2024-04-20T11:11:06.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.139+0000 7fabda000700 1 -- 172.21.15.44:0/2777913283 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabcc011c00 con 0x7fabdc10f0b0 2024-04-20T11:11:06.214 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 -- 172.21.15.44:0/2777913283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 msgr2=0x7fabdc10f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:06.214 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 --2- 172.21.15.44:0/2777913283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc10f4d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0098a0 tx=0x7fabcc009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:06.214 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 -- 172.21.15.44:0/2777913283 shutdown_connections 2024-04-20T11:11:06.214 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 --2- 172.21.15.44:0/2777913283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc10f4d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:06.214 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 -- 172.21.15.44:0/2777913283 >> 172.21.15.44:0/2777913283 conn(0x7fabdc07de10 msgr2=0x7fabdc080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:06.214 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 -- 172.21.15.44:0/2777913283 shutdown_connections 2024-04-20T11:11:06.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.140+0000 7fabe2000700 1 -- 172.21.15.44:0/2777913283 wait complete. 2024-04-20T11:11:06.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.141+0000 7fabe2000700 1 Processor -- start 2024-04-20T11:11:06.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.141+0000 7fabe2000700 1 -- start start 2024-04-20T11:11:06.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.142+0000 7fabe2000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc1a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:06.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.142+0000 7fabe2000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1131e0 con 0x7fabdc10f0b0 2024-04-20T11:11:06.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.142+0000 7fabdb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc1a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:06.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.142+0000 7fabdb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc1a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58032/0 (socket says 172.21.15.44:58032) 2024-04-20T11:11:06.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.142+0000 7fabdb400700 1 -- 172.21.15.44:0/1974437215 learned_addr learned my addr 172.21.15.44:0/1974437215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:06.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.142+0000 7fabdb400700 1 -- 172.21.15.44:0/1974437215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc1a3090 con 0x7fabdc10f0b0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.143+0000 7fabdb400700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc1a2b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabcc000c00 tx=0x7fabcc011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.143+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabcc003b20 con 0x7fabdc10f0b0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.143+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fabcc003cc0 con 0x7fabdc10f0b0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.143+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc1a3320 con 0x7fabdc10f0b0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.143+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabcc0249a0 con 0x7fabdc10f0b0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.143+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc1a6010 con 0x7fabdc10f0b0 2024-04-20T11:11:06.217 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.144+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fabcc022020 con 0x7fabdc10f0b0 2024-04-20T11:11:06.218 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.144+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc113360 con 0x7fabdc10f0b0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.145+0000 7fabd3e00700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7fabc0039670 0x7fabc003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.145+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fabcc04d330 con 0x7fabdc10f0b0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.145+0000 7fabdaa00700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7fabc0039670 0x7fabc003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.146+0000 7fabdaa00700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7fabc0039670 0x7fabc003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.150+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fabcc02c560 con 0x7fabdc10f0b0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.438+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fabdc1a6660 con 0x7fabdc10f0b0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.491+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fabcc04d090 con 0x7fabdc10f0b0 2024-04-20T11:11:06.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.494+0000 7fabd3e00700 1 -- 172.21.15.44:0/1974437215 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fabcc01a430 con 0x7fabdc10f0b0 2024-04-20T11:11:06.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7fabc0039670 msgr2=0x7fabc003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7fabc0039670 0x7fabc003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 msgr2=0x7fabdc1a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc1a2b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabcc000c00 tx=0x7fabcc011c80 comp rx=0 tx=0).stop 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 shutdown_connections 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7fabc0039670 0x7fabc003bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 --2- 172.21.15.44:0/1974437215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc10f0b0 0x7fabdc1a2b50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 >> 172.21.15.44:0/1974437215 conn(0x7fabdc07de10 msgr2=0x7fabdc07f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 shutdown_connections 2024-04-20T11:11:06.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:05.496+0000 7fabe2000700 1 -- 172.21.15.44:0/1974437215 wait complete. 2024-04-20T11:11:06.384 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:06.164+0000 7f1be52dc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:11:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:06 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1974437215' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-20T11:11:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:06 smithi044 ceph-mon[16855]: mgrmap e5: x(active, since 5s) 2024-04-20T11:11:06.696 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:06.590+0000 7f1be52dc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:11:07.061 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:06.709+0000 7f1be52dc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:11:07.866 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:07 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/239814902' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T11:11:07.876 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:11:07.877 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-20T11:11:07.877 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T11:11:07.877 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T11:11:07.877 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T11:11:07.877 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.851+0000 7f5df1800700 1 Processor -- start 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.851+0000 7f5df1800700 1 -- start start 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.852+0000 7f5df1800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.852+0000 7f5df1800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec108f30 con 0x7f5dec108540 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.852+0000 7f5debe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.853+0000 7f5debe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58046/0 (socket says 172.21.15.44:58046) 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.853+0000 7f5debe00700 1 -- 172.21.15.44:0/3430025819 learned_addr learned my addr 172.21.15.44:0/3430025819 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:07.884 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.853+0000 7f5debe00700 1 -- 172.21.15.44:0/3430025819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec1096f0 con 0x7f5dec108540 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.853+0000 7f5debe00700 1 --2- 172.21.15.44:0/3430025819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec108960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0098a0 tx=0x7f5ddc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa951d443d185679 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.854+0000 7f5deaa00700 1 -- 172.21.15.44:0/3430025819 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ddc004160 con 0x7f5dec108540 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.854+0000 7f5deaa00700 1 -- 172.21.15.44:0/3430025819 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5ddc004300 con 0x7f5dec108540 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.854+0000 7f5deaa00700 1 -- 172.21.15.44:0/3430025819 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ddc011c00 con 0x7f5dec108540 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.854+0000 7f5df1800700 1 -- 172.21.15.44:0/3430025819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 msgr2=0x7f5dec108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.854+0000 7f5df1800700 1 --2- 172.21.15.44:0/3430025819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec108960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0098a0 tx=0x7f5ddc009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.855+0000 7f5df1800700 1 -- 172.21.15.44:0/3430025819 shutdown_connections 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.855+0000 7f5df1800700 1 --2- 172.21.15.44:0/3430025819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec108960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.855+0000 7f5df1800700 1 -- 172.21.15.44:0/3430025819 >> 172.21.15.44:0/3430025819 conn(0x7f5dec103b60 msgr2=0x7f5dec105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.855+0000 7f5df1800700 1 -- 172.21.15.44:0/3430025819 shutdown_connections 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.855+0000 7f5df1800700 1 -- 172.21.15.44:0/3430025819 wait complete. 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.856+0000 7f5df1800700 1 Processor -- start 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.856+0000 7f5df1800700 1 -- start start 2024-04-20T11:11:07.885 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.856+0000 7f5df1800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.856+0000 7f5df1800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec10caa0 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.857+0000 7f5debe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.857+0000 7f5debe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58048/0 (socket says 172.21.15.44:58048) 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.857+0000 7f5debe00700 1 -- 172.21.15.44:0/239814902 learned_addr learned my addr 172.21.15.44:0/239814902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.857+0000 7f5debe00700 1 -- 172.21.15.44:0/239814902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19ec90 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.858+0000 7f5debe00700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec19e750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009fb0 tx=0x7f5ddc011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.858+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ddc0038b0 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.858+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5ddc01a430 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.858+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ddc0195e0 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.858+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19ef20 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.858+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a1c10 con 0x7f5dec108540 2024-04-20T11:11:07.886 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.859+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec0640b0 con 0x7f5dec108540 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.859+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5ddc01aa50 con 0x7f5dec108540 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.860+0000 7f5de8c00700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 0x7f5dd003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.860+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ddc04d9f0 con 0x7f5dec108540 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.860+0000 7f5deb400700 1 -- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 msgr2=0x7f5dd003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.860+0000 7f5deb400700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 0x7f5dd003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:06.865+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ddc020240 con 0x7f5dec108540 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.061+0000 7f5deb400700 1 -- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 msgr2=0x7f5dd003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.061+0000 7f5deb400700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 0x7f5dd003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.156+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5dec1a1de0 con 0x7f5dec108540 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.156+0000 7f5de8c00700 1 -- 172.21.15.44:0/239814902 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f5dec1a1de0 con 0x7f5dec108540 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 msgr2=0x7f5dd003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 0x7f5dd003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:07.887 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 msgr2=0x7f5dec19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec19e750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009fb0 tx=0x7f5ddc011c60 comp rx=0 tx=0).stop 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 shutdown_connections 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f5dd00395b0 0x7f5dd003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 --2- 172.21.15.44:0/239814902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dec108540 0x7f5dec19e750 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 >> 172.21.15.44:0/239814902 conn(0x7f5dec103b60 msgr2=0x7f5dec104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 shutdown_connections 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:07.159+0000 7f5df1800700 1 -- 172.21.15.44:0/239814902 wait complete. 2024-04-20T11:11:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2024-04-20T11:11:07.889 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 5... 2024-04-20T11:11:08.503 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:08.312+0000 7f1be52dc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:11:08.766 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:08.549+0000 7f1be52dc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:11:08.766 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:08.765+0000 7f1be52dc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:11:09.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:09.153+0000 7f1be52dc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:11:09.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:09.278+0000 7f1be52dc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:11:09.794 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:09.534+0000 7f1be52dc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:11:09.794 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:09.793+0000 7f1be52dc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:11:10.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:10.600+0000 7f1be52dc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:11:11.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:11.162+0000 7f1be52dc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:11:11.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:11.289+0000 7f1be52dc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:11:11.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:11.560+0000 7f1be52dc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:11:12.673 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:12.221+0000 7f1be52dc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:11:12.673 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:12.347+0000 7f1be52dc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:11:12.673 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:12.672+0000 7f1be52dc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:11:12.944 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:12.854+0000 7f1be52dc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:11:13.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:12.981+0000 7f1be52dc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:11:13.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:13.134+0000 7f1be52dc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:11:13.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:13.509+0000 7f1be52dc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:11:14.576 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:14.284+0000 7f1be52dc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:11:14.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:14.575+0000 7f1be52dc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:11:14.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: Active manager daemon x restarted 2024-04-20T11:11:14.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: Activating manager daemon x 2024-04-20T11:11:14.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: osdmap e2: 0 total, 0 up, 0 in 2024-04-20T11:11:14.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: mgrmap e6: x(active, starting, since 0.00926556s) 2024-04-20T11:11:14.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:11:14.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T11:11:14.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:11:14.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:11:14.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:11:14.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:14 smithi044 ceph-mon[16855]: Manager daemon x is now available 2024-04-20T11:11:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: Found migration_current of "None". Setting to last migration. 2024-04-20T11:11:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:15] ENGINE Bus STARTING 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:15] ENGINE Serving on http://172.21.15.44:8765 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:15] ENGINE Serving on https://172.21.15.44:7150 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:15] ENGINE Bus STARTED 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T11:11:15.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:15 smithi044 ceph-mon[16855]: mgrmap e7: x(active, since 1.01823s) 2024-04-20T11:11:16.341 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:11:16.341 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-20T11:11:16.341 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.503+0000 7f6243a00700 1 Processor -- start 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.503+0000 7f6243a00700 1 -- start start 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.504+0000 7f6243a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.504+0000 7f6243a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f623c107f50 con 0x7f623c107560 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.504+0000 7f6241000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.504+0000 7f6241000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51948/0 (socket says 172.21.15.44:51948) 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.504+0000 7f6241000700 1 -- 172.21.15.44:0/1689691649 learned_addr learned my addr 172.21.15.44:0/1689691649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.505+0000 7f6241000700 1 -- 172.21.15.44:0/1689691649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c108760 con 0x7f623c107560 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.505+0000 7f6241000700 1 --2- 172.21.15.44:0/1689691649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c107980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098a0 tx=0x7f622c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1c0225c935e7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.505+0000 7f623be00700 1 -- 172.21.15.44:0/1689691649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f622c004160 con 0x7f623c107560 2024-04-20T11:11:16.342 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.505+0000 7f623be00700 1 -- 172.21.15.44:0/1689691649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f622c004300 con 0x7f623c107560 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.505+0000 7f623be00700 1 -- 172.21.15.44:0/1689691649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f622c011c00 con 0x7f623c107560 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 -- 172.21.15.44:0/1689691649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 msgr2=0x7f623c107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 --2- 172.21.15.44:0/1689691649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c107980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098a0 tx=0x7f622c009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 -- 172.21.15.44:0/1689691649 shutdown_connections 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 --2- 172.21.15.44:0/1689691649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c107980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 -- 172.21.15.44:0/1689691649 >> 172.21.15.44:0/1689691649 conn(0x7f623c102b40 msgr2=0x7f623c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 -- 172.21.15.44:0/1689691649 shutdown_connections 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.506+0000 7f6243a00700 1 -- 172.21.15.44:0/1689691649 wait complete. 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6243a00700 1 Processor -- start 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6243a00700 1 -- start start 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6243a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6243a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10bb10 con 0x7f623c107560 2024-04-20T11:11:16.343 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6241000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6241000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51960/0 (socket says 172.21.15.44:51960) 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.508+0000 7f6241000700 1 -- 172.21.15.44:0/1114162120 learned_addr learned my addr 172.21.15.44:0/1114162120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.509+0000 7f6241000700 1 -- 172.21.15.44:0/1114162120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c19ee10 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.509+0000 7f6241000700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c19e8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f622c000c00 tx=0x7f622c011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.509+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f622c003b20 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.509+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f622c003cc0 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.509+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c19f0a0 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.509+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f622c0249a0 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.510+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1a1d90 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.511+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f622c022020 con 0x7f623c107560 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.511+0000 7f623a000700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.511+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 --> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f623c0640b0 con 0x7f6220039600 2024-04-20T11:11:16.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.511+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f622c04d320 con 0x7f623c107560 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.511+0000 7f6233e00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 msgr2=0x7f622003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.511+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.712+0000 7f6233e00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 msgr2=0x7f622003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:08.712+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:09.113+0000 7f6233e00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 msgr2=0x7f622003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:09.113+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:09.914+0000 7f6233e00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 msgr2=0x7f622003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:09.914+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:11.516+0000 7f6233e00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 msgr2=0x7f622003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1801045838 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:11.516+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:14.586+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f622c02d3f0 con 0x7f623c107560 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:14.586+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 msgr2=0x7f622003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:14.586+0000 7f623a000700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.591+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f622c024b40 con 0x7f623c107560 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.591+0000 7f623a000700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f622003ce20 0x7f622003f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.591+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f623c0640b0 con 0x7f622003ce20 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.593+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f622003ce20 0x7f622003f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.595+0000 7f6233e00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f622003ce20 0x7f622003f210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6224003a10 tx=0x7f62240092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.597+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f623c0640b0 con 0x7f622003ce20 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.606+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f623c10bc90 con 0x7f622003ce20 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.607+0000 7f623a000700 1 -- 172.21.15.44:0/1114162120 <== mgr.14120 v2:172.21.15.44:6800/1299451251 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f623c10bc90 con 0x7f622003ce20 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f622003ce20 msgr2=0x7f622003f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f622003ce20 0x7f622003f210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6224003a10 tx=0x7f62240092b0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 msgr2=0x7f623c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c19e8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f622c000c00 tx=0x7f622c011c80 comp rx=0 tx=0).stop 2024-04-20T11:11:16.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 shutdown_connections 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f623c107560 0x7f623c19e8d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f622003ce20 0x7f622003f210 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 --2- 172.21.15.44:0/1114162120 >> [v2:172.21.15.44:6800/1801045838,v1:172.21.15.44:6801/1801045838] conn(0x7f6220039600 0x7f622003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 >> 172.21.15.44:0/1114162120 conn(0x7f623c102b40 msgr2=0x7f623c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 shutdown_connections 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:15.608+0000 7f6243a00700 1 -- 172.21.15.44:0/1114162120 wait complete. 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 5 is available 2024-04-20T11:11:16.347 INFO:teuthology.orchestra.run.smithi044.stdout:Setting orchestrator backend to cephadm... 2024-04-20T11:11:16.900 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:16 smithi044 ceph-mon[16855]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T11:11:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:17 smithi044 ceph-mon[16855]: mgrmap e8: x(active, since 2s) 2024-04-20T11:11:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:17 smithi044 ceph-mon[16855]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:17 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:17 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:18.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.984+0000 7fc933e00700 1 Processor -- start 2024-04-20T11:11:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.985+0000 7fc933e00700 1 -- start start 2024-04-20T11:11:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.985+0000 7fc933e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.985+0000 7fc933e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c07c820 con 0x7fc92c07de30 2024-04-20T11:11:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.985+0000 7fc931400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:18.039 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.985+0000 7fc931400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52030/0 (socket says 172.21.15.44:52030) 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.986+0000 7fc931400700 1 -- 172.21.15.44:0/1527795683 learned_addr learned my addr 172.21.15.44:0/1527795683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.986+0000 7fc931400700 1 -- 172.21.15.44:0/1527795683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c07c9a0 con 0x7fc92c07de30 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.986+0000 7fc931400700 1 --2- 172.21.15.44:0/1527795683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c07c2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098a0 tx=0x7fc920009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981e91cf1d757a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.987+0000 7fc92be00700 1 -- 172.21.15.44:0/1527795683 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc920004160 con 0x7fc92c07de30 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.987+0000 7fc92be00700 1 -- 172.21.15.44:0/1527795683 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc920004300 con 0x7fc92c07de30 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.987+0000 7fc92be00700 1 -- 172.21.15.44:0/1527795683 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc920011c00 con 0x7fc92c07de30 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.987+0000 7fc933e00700 1 -- 172.21.15.44:0/1527795683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 msgr2=0x7fc92c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.987+0000 7fc933e00700 1 --2- 172.21.15.44:0/1527795683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c07c2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098a0 tx=0x7fc920009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.988+0000 7fc933e00700 1 -- 172.21.15.44:0/1527795683 shutdown_connections 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.988+0000 7fc933e00700 1 --2- 172.21.15.44:0/1527795683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c07c2e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.988+0000 7fc933e00700 1 -- 172.21.15.44:0/1527795683 >> 172.21.15.44:0/1527795683 conn(0x7fc92c103c50 msgr2=0x7fc92c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.988+0000 7fc933e00700 1 -- 172.21.15.44:0/1527795683 shutdown_connections 2024-04-20T11:11:18.040 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.988+0000 7fc933e00700 1 -- 172.21.15.44:0/1527795683 wait complete. 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc933e00700 1 Processor -- start 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc933e00700 1 -- start start 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc933e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc933e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c10eca0 con 0x7fc92c07de30 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc931400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc931400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52034/0 (socket says 172.21.15.44:52034) 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.989+0000 7fc931400700 1 -- 172.21.15.44:0/26110393 learned_addr learned my addr 172.21.15.44:0/26110393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.990+0000 7fc931400700 1 -- 172.21.15.44:0/26110393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c19ed90 con 0x7fc92c07de30 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.990+0000 7fc931400700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c19e850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc920011bd0 tx=0x7fc920011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.990+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc920003b20 con 0x7fc92c07de30 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.990+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc920003cc0 con 0x7fc92c07de30 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.990+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c19f020 con 0x7fc92c07de30 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.991+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc920024930 con 0x7fc92c07de30 2024-04-20T11:11:18.041 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.991+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c1a1d10 con 0x7fc92c07de30 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.992+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92c0640b0 con 0x7fc92c07de30 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.992+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc920022020 con 0x7fc92c07de30 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.992+0000 7fc92a000700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc9140395b0 0x7fc91403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.992+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc92004d770 con 0x7fc92c07de30 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.993+0000 7fc930a00700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc9140395b0 0x7fc91403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.993+0000 7fc930a00700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc9140395b0 0x7fc91403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc918006fd0 tx=0x7fc918006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:16.998+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc92001aca0 con 0x7fc92c07de30 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.252+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc92c104ed0 con 0x7fc9140395b0 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.260+0000 7fc92a000700 1 -- 172.21.15.44:0/26110393 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc92c104ed0 con 0x7fc9140395b0 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.263+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc9140395b0 msgr2=0x7fc91403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.263+0000 7fc933e00700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc9140395b0 0x7fc91403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc918006fd0 tx=0x7fc918006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:18.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.263+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 msgr2=0x7fc92c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.263+0000 7fc933e00700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c19e850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc920011bd0 tx=0x7fc920011cb0 comp rx=0 tx=0).stop 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.264+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 shutdown_connections 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.264+0000 7fc933e00700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc9140395b0 0x7fc91403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.264+0000 7fc933e00700 1 --2- 172.21.15.44:0/26110393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc92c07de30 0x7fc92c19e850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.264+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 >> 172.21.15.44:0/26110393 conn(0x7fc92c103c50 msgr2=0x7fc92c1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.264+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 shutdown_connections 2024-04-20T11:11:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:17.264+0000 7fc933e00700 1 -- 172.21.15.44:0/26110393 wait complete. 2024-04-20T11:11:19.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-20T11:11:19.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.629+0000 7fe5c2200700 1 Processor -- start 2024-04-20T11:11:19.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.629+0000 7fe5c2200700 1 -- start start 2024-04-20T11:11:19.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.629+0000 7fe5c2200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:19.599 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.629+0000 7fe5c2200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1090b0 con 0x7fe5bc1086c0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.630+0000 7fe5bb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.630+0000 7fe5bb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59700/0 (socket says 172.21.15.44:59700) 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.630+0000 7fe5bb400700 1 -- 172.21.15.44:0/3499394574 learned_addr learned my addr 172.21.15.44:0/3499394574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.630+0000 7fe5bb400700 1 -- 172.21.15.44:0/3499394574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc109870 con 0x7fe5bc1086c0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.631+0000 7fe5bb400700 1 --2- 172.21.15.44:0/3499394574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc108ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0098a0 tx=0x7fe5ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3bc46d32cbe55a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.631+0000 7fe5ba000700 1 -- 172.21.15.44:0/3499394574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5ac004160 con 0x7fe5bc1086c0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.631+0000 7fe5ba000700 1 -- 172.21.15.44:0/3499394574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe5ac004300 con 0x7fe5bc1086c0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.631+0000 7fe5ba000700 1 -- 172.21.15.44:0/3499394574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5ac011c00 con 0x7fe5bc1086c0 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 -- 172.21.15.44:0/3499394574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 msgr2=0x7fe5bc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 --2- 172.21.15.44:0/3499394574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc108ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0098a0 tx=0x7fe5ac009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 -- 172.21.15.44:0/3499394574 shutdown_connections 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 --2- 172.21.15.44:0/3499394574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc108ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:19.600 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 -- 172.21.15.44:0/3499394574 >> 172.21.15.44:0/3499394574 conn(0x7fe5bc103cc0 msgr2=0x7fe5bc1060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 -- 172.21.15.44:0/3499394574 shutdown_connections 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.632+0000 7fe5c2200700 1 -- 172.21.15.44:0/3499394574 wait complete. 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.633+0000 7fe5c2200700 1 Processor -- start 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.634+0000 7fe5c2200700 1 -- start start 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.634+0000 7fe5c2200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.634+0000 7fe5c2200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc10cc20 con 0x7fe5bc1086c0 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.634+0000 7fe5bb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.634+0000 7fe5bb400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc19c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59710/0 (socket says 172.21.15.44:59710) 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.634+0000 7fe5bb400700 1 -- 172.21.15.44:0/1025299751 learned_addr learned my addr 172.21.15.44:0/1025299751 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5bb400700 1 -- 172.21.15.44:0/1025299751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc19cc70 con 0x7fe5bc1086c0 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5bb400700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc19c730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0118c0 tx=0x7fe5ac011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5ac003af0 con 0x7fe5bc1086c0 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe5ac020070 con 0x7fe5bc1086c0 2024-04-20T11:11:19.601 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc19cf00 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5ac0195e0 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.635+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc19fbf0 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.637+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe5ac019870 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.637+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc10cda0 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.637+0000 7fe5b3e00700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fe5a0039600 0x7fe5a003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.637+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe5ac04db20 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.638+0000 7fe5baa00700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fe5a0039600 0x7fe5a003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.638+0000 7fe5baa00700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fe5a0039600 0x7fe5a003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006fd0 tx=0x7fe5a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.643+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe5ac023530 con 0x7fe5bc1086c0 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.873+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe5bc104f50 con 0x7fe5a0039600 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.874+0000 7fe5b3e00700 1 -- 172.21.15.44:0/1025299751 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe5bc104f50 con 0x7fe5a0039600 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fe5a0039600 msgr2=0x7fe5a003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fe5a0039600 0x7fe5a003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006fd0 tx=0x7fe5a8006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:19.602 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 msgr2=0x7fe5bc19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc19c730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0118c0 tx=0x7fe5ac011c80 comp rx=0 tx=0).stop 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 shutdown_connections 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fe5a0039600 0x7fe5a003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 --2- 172.21.15.44:0/1025299751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1086c0 0x7fe5bc19c730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 >> 172.21.15.44:0/1025299751 conn(0x7fe5bc103cc0 msgr2=0x7fe5bc104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 shutdown_connections 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:18.877+0000 7fe5c2200700 1 -- 172.21.15.44:0/1025299751 wait complete. 2024-04-20T11:11:19.603 INFO:teuthology.orchestra.run.smithi044.stdout:Generating ssh key... 2024-04-20T11:11:19.883 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:19 smithi044 ceph-mon[16855]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:20.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-mon[16855]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:20.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-mon[16855]: Generating ssh key... 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: Generating public/private rsa key pair. 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: Your identification has been saved in /tmp/tmp1v2gxbw6/key. 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: Your public key has been saved in /tmp/tmp1v2gxbw6/key.pub. 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: The key fingerprint is: 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: SHA256:ladB4g5kd3Ya8rM4NYh6BU+sIb6rfY2ignWkTCJyMYc ceph-759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: The key's randomart image is: 2024-04-20T11:11:21.337 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: +---[RSA 3072]----+ 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: | . +.= = . | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: | E oo.B.O = | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: | = .oo= X . | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: |+ o o..+ + B | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: |o= o... S o | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: | + o. . | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: |.. . . o | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: |o .o o . | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: | .oo.o | 2024-04-20T11:11:21.338 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:20 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: +----[SHA256]-----+ 2024-04-20T11:11:21.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.200+0000 7f9192200700 1 Processor -- start 2024-04-20T11:11:21.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.200+0000 7f9192200700 1 -- start start 2024-04-20T11:11:21.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.201+0000 7f9192200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c10ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:21.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.201+0000 7f9192200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10b2d0 con 0x7f918c10a8e0 2024-04-20T11:11:21.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.201+0000 7f918b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c10ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:21.721 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.201+0000 7f918b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c10ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59718/0 (socket says 172.21.15.44:59718) 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.201+0000 7f918b400700 1 -- 172.21.15.44:0/3450908486 learned_addr learned my addr 172.21.15.44:0/3450908486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.202+0000 7f918b400700 1 -- 172.21.15.44:0/3450908486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c10ba90 con 0x7f918c10a8e0 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.202+0000 7f918b400700 1 --2- 172.21.15.44:0/3450908486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c10ad00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e80 tx=0x7f917c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abc5228387bed4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.203+0000 7f918aa00700 1 -- 172.21.15.44:0/3450908486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c017070 con 0x7f918c10a8e0 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.203+0000 7f918aa00700 1 -- 172.21.15.44:0/3450908486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f917c00b8c0 con 0x7f918c10a8e0 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.203+0000 7f918aa00700 1 -- 172.21.15.44:0/3450908486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c004620 con 0x7f918c10a8e0 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.203+0000 7f9192200700 1 -- 172.21.15.44:0/3450908486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 msgr2=0x7f918c10ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.203+0000 7f9192200700 1 --2- 172.21.15.44:0/3450908486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c10ad00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e80 tx=0x7f917c00b330 comp rx=0 tx=0).stop 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.204+0000 7f9192200700 1 -- 172.21.15.44:0/3450908486 shutdown_connections 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.204+0000 7f9192200700 1 --2- 172.21.15.44:0/3450908486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c10ad00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:21.722 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.204+0000 7f9192200700 1 -- 172.21.15.44:0/3450908486 >> 172.21.15.44:0/3450908486 conn(0x7f918c07dde0 msgr2=0x7f918c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.204+0000 7f9192200700 1 -- 172.21.15.44:0/3450908486 shutdown_connections 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.204+0000 7f9192200700 1 -- 172.21.15.44:0/3450908486 wait complete. 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.205+0000 7f9192200700 1 Processor -- start 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.205+0000 7f9192200700 1 -- start start 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.205+0000 7f9192200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.205+0000 7f9192200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10ee40 con 0x7f918c10a8e0 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.206+0000 7f918b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.206+0000 7f918b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59734/0 (socket says 172.21.15.44:59734) 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.206+0000 7f918b400700 1 -- 172.21.15.44:0/729970842 learned_addr learned my addr 172.21.15.44:0/729970842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.206+0000 7f918b400700 1 -- 172.21.15.44:0/729970842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c19ec80 con 0x7f918c10a8e0 2024-04-20T11:11:21.723 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.206+0000 7f918b400700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c19e740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f917c009530 tx=0x7f917c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.207+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c021020 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.207+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f917c00be00 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.207+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c017040 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.207+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c19ef10 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.207+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c1a1c00 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.208+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c10efc0 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.208+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f917c024020 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.208+0000 7f9188c00700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f91700396c0 0x7f917003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.209+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f917c00f050 con 0x7f918c10a8e0 2024-04-20T11:11:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.209+0000 7f9182a00700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f91700396c0 0x7f917003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.210+0000 7f9182a00700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f91700396c0 0x7f917003bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9174006fd0 tx=0x7f9174006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.214+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f917c02b020 con 0x7f918c10a8e0 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.446+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f918c1081e0 con 0x7f91700396c0 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.966+0000 7f9188c00700 1 -- 172.21.15.44:0/729970842 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f918c1081e0 con 0x7f91700396c0 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f91700396c0 msgr2=0x7f917003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f91700396c0 0x7f917003bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9174006fd0 tx=0x7f9174006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 msgr2=0x7f918c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c19e740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f917c009530 tx=0x7f917c003d00 comp rx=0 tx=0).stop 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 shutdown_connections 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f91700396c0 0x7f917003bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 --2- 172.21.15.44:0/729970842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f918c10a8e0 0x7f918c19e740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 >> 172.21.15.44:0/729970842 conn(0x7f918c07dde0 msgr2=0x7f918c107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:21.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 shutdown_connections 2024-04-20T11:11:21.726 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:20.969+0000 7f9192200700 1 -- 172.21.15.44:0/729970842 wait complete. 2024-04-20T11:11:21.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:21 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:21.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:21 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.360+0000 7fd25b200700 1 Processor -- start 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.361+0000 7fd25b200700 1 -- start start 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.361+0000 7fd25b200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd254107380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.361+0000 7fd25b200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd25407be10 con 0x7fd254106f60 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.362+0000 7fd253e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd254107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.362+0000 7fd253e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd254107380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59750/0 (socket says 172.21.15.44:59750) 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.362+0000 7fd253e00700 1 -- 172.21.15.44:0/3234370603 learned_addr learned my addr 172.21.15.44:0/3234370603 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.362+0000 7fd253e00700 1 -- 172.21.15.44:0/3234370603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2541078c0 con 0x7fd254106f60 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.362+0000 7fd253e00700 1 --2- 172.21.15.44:0/3234370603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd254107380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098a0 tx=0x7fd244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff014528dcc5d08f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:23.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.363+0000 7fd253400700 1 -- 172.21.15.44:0/3234370603 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd244004160 con 0x7fd254106f60 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.363+0000 7fd253400700 1 -- 172.21.15.44:0/3234370603 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd244004300 con 0x7fd254106f60 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.363+0000 7fd253400700 1 -- 172.21.15.44:0/3234370603 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd244011c00 con 0x7fd254106f60 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 -- 172.21.15.44:0/3234370603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 msgr2=0x7fd254107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 --2- 172.21.15.44:0/3234370603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd254107380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd2440098a0 tx=0x7fd244009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 -- 172.21.15.44:0/3234370603 shutdown_connections 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 --2- 172.21.15.44:0/3234370603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd254107380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 -- 172.21.15.44:0/3234370603 >> 172.21.15.44:0/3234370603 conn(0x7fd254102b40 msgr2=0x7fd254104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:23.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 -- 172.21.15.44:0/3234370603 shutdown_connections 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.364+0000 7fd25b200700 1 -- 172.21.15.44:0/3234370603 wait complete. 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.365+0000 7fd25b200700 1 Processor -- start 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.366+0000 7fd25b200700 1 -- start start 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.366+0000 7fd25b200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd25419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.366+0000 7fd25b200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541089b0 con 0x7fd254106f60 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.366+0000 7fd253e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd25419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.366+0000 7fd253e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd25419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59752/0 (socket says 172.21.15.44:59752) 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.366+0000 7fd253e00700 1 -- 172.21.15.44:0/668174369 learned_addr learned my addr 172.21.15.44:0/668174369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.367+0000 7fd253e00700 1 -- 172.21.15.44:0/668174369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25419edb0 con 0x7fd254106f60 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.367+0000 7fd253e00700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd25419e870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd244003fb0 tx=0x7fd244011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.367+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2440039c0 con 0x7fd254106f60 2024-04-20T11:11:23.350 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.367+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd244003b60 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.367+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24401a6b0 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.368+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25419f040 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.368+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2541a1d30 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.369+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd244024430 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.369+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2540640b0 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.369+0000 7fd251600700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd2380395b0 0x7fd23803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.370+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd24404dab0 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.370+0000 7fd24b400700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd2380395b0 0x7fd23803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.371+0000 7fd24b400700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd2380395b0 0x7fd23803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ad30 tx=0x7fd23c0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.376+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd244003d10 con 0x7fd254106f60 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.608+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd2540008d0 con 0x7fd2380395b0 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.609+0000 7fd251600700 1 -- 172.21.15.44:0/668174369 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd2540008d0 con 0x7fd2380395b0 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.612+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd2380395b0 msgr2=0x7fd23803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:23.351 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.612+0000 7fd25b200700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd2380395b0 0x7fd23803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ad30 tx=0x7fd23c0093f0 comp rx=0 tx=0).stop 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.612+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 msgr2=0x7fd25419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.612+0000 7fd25b200700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd25419e870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd244003fb0 tx=0x7fd244011bd0 comp rx=0 tx=0).stop 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.613+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 shutdown_connections 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.613+0000 7fd25b200700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd2380395b0 0x7fd23803ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.613+0000 7fd25b200700 1 --2- 172.21.15.44:0/668174369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd254106f60 0x7fd25419e870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.613+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 >> 172.21.15.44:0/668174369 conn(0x7fd254102b40 msgr2=0x7fd254195510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.613+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 shutdown_connections 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:22.613+0000 7fd25b200700 1 -- 172.21.15.44:0/668174369 wait complete. 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-20T11:11:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:Adding key to root@localhost authorized_keys... 2024-04-20T11:11:23.362 INFO:teuthology.orchestra.run.smithi044.stdout:Adding host smithi044... 2024-04-20T11:11:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:23 smithi044 ceph-mon[16855]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:24.887 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:24 smithi044 ceph-mon[16855]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "addr": "172.21.15.44", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:25.862 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:25 smithi044 ceph-mon[16855]: Deploying cephadm binary to smithi044 2024-04-20T11:11:31.344 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Added host 'smithi044' with addr '172.21.15.44' 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.949+0000 7fd9cf200700 1 Processor -- start 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.950+0000 7fd9cf200700 1 -- start start 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.950+0000 7fd9cf200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c8110520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.950+0000 7fd9cf200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8110af0 con 0x7fd9c8110100 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.950+0000 7fd9cde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c8110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.951+0000 7fd9cde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c8110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59758/0 (socket says 172.21.15.44:59758) 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.951+0000 7fd9cde00700 1 -- 172.21.15.44:0/2886064822 learned_addr learned my addr 172.21.15.44:0/2886064822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.951+0000 7fd9cde00700 1 -- 172.21.15.44:0/2886064822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c810ecd0 con 0x7fd9c8110100 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.952+0000 7fd9cde00700 1 --2- 172.21.15.44:0/2886064822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c8110520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0098a0 tx=0x7fd9bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32129f8d0107fc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.952+0000 7fd9cca00700 1 -- 172.21.15.44:0/2886064822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9bc004160 con 0x7fd9c8110100 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.952+0000 7fd9cca00700 1 -- 172.21.15.44:0/2886064822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd9bc004300 con 0x7fd9c8110100 2024-04-20T11:11:31.345 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.952+0000 7fd9cca00700 1 -- 172.21.15.44:0/2886064822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9bc011c00 con 0x7fd9c8110100 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 -- 172.21.15.44:0/2886064822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 msgr2=0x7fd9c8110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 --2- 172.21.15.44:0/2886064822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c8110520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0098a0 tx=0x7fd9bc009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 -- 172.21.15.44:0/2886064822 shutdown_connections 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 --2- 172.21.15.44:0/2886064822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c8110520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 -- 172.21.15.44:0/2886064822 >> 172.21.15.44:0/2886064822 conn(0x7fd9c807db50 msgr2=0x7fd9c807ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 -- 172.21.15.44:0/2886064822 shutdown_connections 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.953+0000 7fd9cf200700 1 -- 172.21.15.44:0/2886064822 wait complete. 2024-04-20T11:11:31.346 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.954+0000 7fd9cf200700 1 Processor -- start 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.954+0000 7fd9cf200700 1 -- start start 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.955+0000 7fd9cf200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c81a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.955+0000 7fd9cf200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8113090 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.955+0000 7fd9cde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c81a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.955+0000 7fd9cde00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c81a29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59760/0 (socket says 172.21.15.44:59760) 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.955+0000 7fd9cde00700 1 -- 172.21.15.44:0/3560997433 learned_addr learned my addr 172.21.15.44:0/3560997433 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.955+0000 7fd9cde00700 1 -- 172.21.15.44:0/3560997433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c81a2ee0 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.956+0000 7fd9cde00700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c81a29a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0118c0 tx=0x7fd9bc011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.956+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9bc003b20 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.956+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd9bc003cc0 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.956+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9bc0247c0 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.956+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81a3170 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.956+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81a5e60 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.958+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c8113210 con 0x7fd9c8110100 2024-04-20T11:11:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.958+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd9bc022020 con 0x7fd9c8110100 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.958+0000 7fd9c2a00700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd9b0039670 0x7fd9b003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.958+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd9bc04da10 con 0x7fd9c8110100 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.958+0000 7fd9cd400700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd9b0039670 0x7fd9b003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.959+0000 7fd9cd400700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd9b0039670 0x7fd9b003bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8006fd0 tx=0x7fd9b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:23.964+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd9bc04d2e0 con 0x7fd9c8110100 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:24.198+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "addr": "172.21.15.44", "target": ["mon-mgr", ""]}) -- 0x7fd9c807ec90 con 0x7fd9b0039670 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.506+0000 7fd9c2a00700 1 -- 172.21.15.44:0/3560997433 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd9c807ec90 con 0x7fd9b0039670 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd9b0039670 msgr2=0x7fd9b003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd9b0039670 0x7fd9b003bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8006fd0 tx=0x7fd9b8006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 msgr2=0x7fd9c81a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c81a29a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0118c0 tx=0x7fd9bc011c80 comp rx=0 tx=0).stop 2024-04-20T11:11:31.348 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 shutdown_connections 2024-04-20T11:11:31.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fd9b0039670 0x7fd9b003bb30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:31.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 --2- 172.21.15.44:0/3560997433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9c8110100 0x7fd9c81a29a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:31.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 >> 172.21.15.44:0/3560997433 conn(0x7fd9c807db50 msgr2=0x7fd9c807e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:31.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.510+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 shutdown_connections 2024-04-20T11:11:31.349 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:30.511+0000 7fd9cf200700 1 -- 172.21.15.44:0/3560997433 wait complete. 2024-04-20T11:11:31.349 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mon service... 2024-04-20T11:11:31.638 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:31 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:31.638 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:31 smithi044 ceph-mon[16855]: Added host smithi044 2024-04-20T11:11:31.638 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:31 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:33.484 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-20T11:11:33.484 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.975+0000 7fc11b600700 1 Processor -- start 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.976+0000 7fc11b600700 1 -- start start 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.976+0000 7fc11b600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc114108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.976+0000 7fc11b600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc114109060 con 0x7fc114108670 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.977+0000 7fc118c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc114108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.977+0000 7fc118c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc114108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39158/0 (socket says 172.21.15.44:39158) 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.977+0000 7fc118c00700 1 -- 172.21.15.44:0/2056140755 learned_addr learned my addr 172.21.15.44:0/2056140755 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.977+0000 7fc118c00700 1 -- 172.21.15.44:0/2056140755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc114109820 con 0x7fc114108670 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.978+0000 7fc118c00700 1 --2- 172.21.15.44:0/2056140755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc114108a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1080098a0 tx=0x7fc108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efb02e7e70dfdae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.978+0000 7fc113400700 1 -- 172.21.15.44:0/2056140755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc108004160 con 0x7fc114108670 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.978+0000 7fc113400700 1 -- 172.21.15.44:0/2056140755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc108004300 con 0x7fc114108670 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.978+0000 7fc113400700 1 -- 172.21.15.44:0/2056140755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc108011c00 con 0x7fc114108670 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 -- 172.21.15.44:0/2056140755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 msgr2=0x7fc114108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:33.485 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 --2- 172.21.15.44:0/2056140755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc114108a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc1080098a0 tx=0x7fc108009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 -- 172.21.15.44:0/2056140755 shutdown_connections 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 --2- 172.21.15.44:0/2056140755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc114108a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 -- 172.21.15.44:0/2056140755 >> 172.21.15.44:0/2056140755 conn(0x7fc114103c50 msgr2=0x7fc114106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 -- 172.21.15.44:0/2056140755 shutdown_connections 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.979+0000 7fc11b600700 1 -- 172.21.15.44:0/2056140755 wait complete. 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.980+0000 7fc11b600700 1 Processor -- start 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.980+0000 7fc11b600700 1 -- start start 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.980+0000 7fc11b600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc11419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.981+0000 7fc11b600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410cbd0 con 0x7fc114108670 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.981+0000 7fc118c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc11419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:33.486 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.981+0000 7fc118c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc11419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39160/0 (socket says 172.21.15.44:39160) 2024-04-20T11:11:33.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.981+0000 7fc118c00700 1 -- 172.21.15.44:0/2370952216 learned_addr learned my addr 172.21.15.44:0/2370952216 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:33.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.981+0000 7fc118c00700 1 -- 172.21.15.44:0/2370952216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419cba0 con 0x7fc114108670 2024-04-20T11:11:33.489 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.981+0000 7fc118c00700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc11419c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc108003fb0 tx=0x7fc108011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.982+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1080039c0 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.982+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc108003b60 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.982+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419ce30 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.982+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc10801a6d0 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.982+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11419fb20 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.983+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc1080249a0 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.983+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11410cd50 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.984+0000 7fc111600700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc0fc0395b0 0x7fc0fc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.984+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc10804da50 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.984+0000 7fc113e00700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc0fc0395b0 0x7fc0fc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.985+0000 7fc113e00700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc0fc0395b0 0x7fc0fc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc100006fd0 tx=0x7fc100006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:31.989+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc10802d3f0 con 0x7fc114108670 2024-04-20T11:11:33.490 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.250+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc114104ed0 con 0x7fc0fc0395b0 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.258+0000 7fc111600700 1 -- 172.21.15.44:0/2370952216 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc114104ed0 con 0x7fc0fc0395b0 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc0fc0395b0 msgr2=0x7fc0fc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc0fc0395b0 0x7fc0fc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc100006fd0 tx=0x7fc100006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 msgr2=0x7fc11419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc11419c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc108003fb0 tx=0x7fc108011bd0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 shutdown_connections 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7fc0fc0395b0 0x7fc0fc03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 --2- 172.21.15.44:0/2370952216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc114108670 0x7fc11419c660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 >> 172.21.15.44:0/2370952216 conn(0x7fc114103c50 msgr2=0x7fc1141047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.261+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 shutdown_connections 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:32.262+0000 7fc11b600700 1 -- 172.21.15.44:0/2370952216 wait complete. 2024-04-20T11:11:33.491 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mgr service... 2024-04-20T11:11:33.590 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:33 smithi044 ceph-mon[16855]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:33.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:33 smithi044 ceph-mon[16855]: Saving service mon spec with placement count:5 2024-04-20T11:11:33.591 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:33 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:35.168 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:35 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:35.168 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:35 smithi044 ceph-mon[16855]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:11:35.168 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:35 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:35.168 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:35 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.528+0000 7f918ce00700 1 Processor -- start 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.528+0000 7f918ce00700 1 -- start start 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.528+0000 7f918ce00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f9188108b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.529+0000 7f918ce00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91881090d0 con 0x7f91881086e0 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.529+0000 7f9185e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f9188108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.529+0000 7f9185e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f9188108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39168/0 (socket says 172.21.15.44:39168) 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.529+0000 7f9185e00700 1 -- 172.21.15.44:0/3099504267 learned_addr learned my addr 172.21.15.44:0/3099504267 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.530+0000 7f9185e00700 1 -- 172.21.15.44:0/3099504267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9188109890 con 0x7f91881086e0 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.530+0000 7f9185e00700 1 --2- 172.21.15.44:0/3099504267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f9188108b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91780098a0 tx=0x7f9178009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99227314e4db6bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.516 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.530+0000 7f9184a00700 1 -- 172.21.15.44:0/3099504267 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9178004160 con 0x7f91881086e0 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.530+0000 7f9184a00700 1 -- 172.21.15.44:0/3099504267 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9178004300 con 0x7f91881086e0 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.530+0000 7f9184a00700 1 -- 172.21.15.44:0/3099504267 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9178011bb0 con 0x7f91881086e0 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 -- 172.21.15.44:0/3099504267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 msgr2=0x7f9188108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 --2- 172.21.15.44:0/3099504267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f9188108b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91780098a0 tx=0x7f9178009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 -- 172.21.15.44:0/3099504267 shutdown_connections 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 --2- 172.21.15.44:0/3099504267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f9188108b00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 -- 172.21.15.44:0/3099504267 >> 172.21.15.44:0/3099504267 conn(0x7f9188103d20 msgr2=0x7f9188106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 -- 172.21.15.44:0/3099504267 shutdown_connections 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.532+0000 7f918ce00700 1 -- 172.21.15.44:0/3099504267 wait complete. 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.534+0000 7f918ce00700 1 Processor -- start 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.534+0000 7f918ce00700 1 -- start start 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.534+0000 7f918ce00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f918819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.517 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.534+0000 7f918ce00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f918810cc40 con 0x7f91881086e0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.535+0000 7f9185e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f918819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.535+0000 7f9185e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f918819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39172/0 (socket says 172.21.15.44:39172) 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.535+0000 7f9185e00700 1 -- 172.21.15.44:0/3900648349 learned_addr learned my addr 172.21.15.44:0/3900648349 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.535+0000 7f9185e00700 1 -- 172.21.15.44:0/3900648349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918819cad0 con 0x7f91881086e0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.536+0000 7f9185e00700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f918819c590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9178000c00 tx=0x7f9178004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.536+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9178011e60 con 0x7f91881086e0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.536+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9178003af0 con 0x7f91881086e0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.536+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91780246a0 con 0x7f91881086e0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.536+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918819cd60 con 0x7f91881086e0 2024-04-20T11:11:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.536+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918819d1c0 con 0x7f91881086e0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.537+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91880640b0 con 0x7f91881086e0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.537+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9178022020 con 0x7f91881086e0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.538+0000 7f917ea00700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f916c039670 0x7f916c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.538+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f917804d830 con 0x7f91881086e0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.539+0000 7f9185400700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f916c039670 0x7f916c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.540+0000 7f9185400700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f916c039670 0x7f916c03bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9174006fd0 tx=0x7f9174006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.544+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f917802a2a0 con 0x7f91881086e0 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.796+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9188104e50 con 0x7f916c039670 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.803+0000 7f917ea00700 1 -- 172.21.15.44:0/3900648349 <== mgr.14120 v2:172.21.15.44:6800/1299451251 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9188104e50 con 0x7f916c039670 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f916c039670 msgr2=0x7f916c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f916c039670 0x7f916c03bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9174006fd0 tx=0x7f9174006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 msgr2=0x7f918819c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f918819c590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9178000c00 tx=0x7f9178004620 comp rx=0 tx=0).stop 2024-04-20T11:11:35.519 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 shutdown_connections 2024-04-20T11:11:35.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f916c039670 0x7f916c03bb30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 --2- 172.21.15.44:0/3900648349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91881086e0 0x7f918819c590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:35.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 >> 172.21.15.44:0/3900648349 conn(0x7f9188103d20 msgr2=0x7f9188104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:35.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 shutdown_connections 2024-04-20T11:11:35.520 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:34.806+0000 7f918ce00700 1 -- 172.21.15.44:0/3900648349 wait complete. 2024-04-20T11:11:36.142 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:36 smithi044 ceph-mon[16855]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:36.143 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:36 smithi044 ceph-mon[16855]: Saving service mgr spec with placement count:2 2024-04-20T11:11:37.272 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.142+0000 7f1aa8a00700 1 Processor -- start 2024-04-20T11:11:37.272 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.142+0000 7f1aa8a00700 1 -- start start 2024-04-20T11:11:37.272 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.142+0000 7f1aa8a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa410ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:37.272 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.142+0000 7f1aa8a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410b2d0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.272 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.143+0000 7f1aa1e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa410ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.143+0000 7f1aa1e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa410ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39188/0 (socket says 172.21.15.44:39188) 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.143+0000 7f1aa1e00700 1 -- 172.21.15.44:0/3933238450 learned_addr learned my addr 172.21.15.44:0/3933238450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.144+0000 7f1aa1e00700 1 -- 172.21.15.44:0/3933238450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa410ba90 con 0x7f1aa410a8e0 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.144+0000 7f1aa1e00700 1 --2- 172.21.15.44:0/3933238450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa410ad00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a940098a0 tx=0x7f1a94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c91e7b76bfd073 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.144+0000 7f1aa0a00700 1 -- 172.21.15.44:0/3933238450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a94004160 con 0x7f1aa410a8e0 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.144+0000 7f1aa0a00700 1 -- 172.21.15.44:0/3933238450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a94004300 con 0x7f1aa410a8e0 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.144+0000 7f1aa0a00700 1 -- 172.21.15.44:0/3933238450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a94011bb0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.145+0000 7f1aa8a00700 1 -- 172.21.15.44:0/3933238450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 msgr2=0x7f1aa410ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.145+0000 7f1aa8a00700 1 --2- 172.21.15.44:0/3933238450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa410ad00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a940098a0 tx=0x7f1a94009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.145+0000 7f1aa8a00700 1 -- 172.21.15.44:0/3933238450 shutdown_connections 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.145+0000 7f1aa8a00700 1 --2- 172.21.15.44:0/3933238450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa410ad00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:37.274 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.145+0000 7f1aa8a00700 1 -- 172.21.15.44:0/3933238450 >> 172.21.15.44:0/3933238450 conn(0x7f1aa407de10 msgr2=0x7f1aa41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:37.275 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.146+0000 7f1aa8a00700 1 -- 172.21.15.44:0/3933238450 shutdown_connections 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.146+0000 7f1aa8a00700 1 -- 172.21.15.44:0/3933238450 wait complete. 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.147+0000 7f1aa8a00700 1 Processor -- start 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.147+0000 7f1aa8a00700 1 -- start start 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.147+0000 7f1aa8a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.148+0000 7f1aa8a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410ee40 con 0x7f1aa410a8e0 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.148+0000 7f1aa1e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.148+0000 7f1aa1e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39200/0 (socket says 172.21.15.44:39200) 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.148+0000 7f1aa1e00700 1 -- 172.21.15.44:0/2657082624 learned_addr learned my addr 172.21.15.44:0/2657082624 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.148+0000 7f1aa1e00700 1 -- 172.21.15.44:0/2657082624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa419ed70 con 0x7f1aa410a8e0 2024-04-20T11:11:37.277 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.148+0000 7f1aa1e00700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa419e830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000c00 tx=0x7f1a94004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.149+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a94011e60 con 0x7f1aa410a8e0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.149+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1a94003af0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.149+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a940246a0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.149+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa419f000 con 0x7f1aa410a8e0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.149+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa419f460 con 0x7f1aa410a8e0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.150+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa40640b0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.150+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1a94022020 con 0x7f1aa410a8e0 2024-04-20T11:11:37.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.151+0000 7f1a9aa00700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f1a88039600 0x7f1a8803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:37.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.151+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a9404d7f0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.151+0000 7f1aa1400700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f1a88039600 0x7f1a8803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:37.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.152+0000 7f1aa1400700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f1a88039600 0x7f1a8803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a90006fd0 tx=0x7f1a90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.156+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1a940212c0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.386+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1aa41a1cf0 con 0x7f1aa410a8e0 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.393+0000 7f1a9aa00700 1 -- 172.21.15.44:0/2657082624 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f1a9404d140 con 0x7f1aa410a8e0 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f1a88039600 msgr2=0x7f1a8803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f1a88039600 0x7f1a8803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1a90006fd0 tx=0x7f1a90006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 msgr2=0x7f1aa419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa419e830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000c00 tx=0x7f1a94004620 comp rx=0 tx=0).stop 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 shutdown_connections 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f1a88039600 0x7f1a8803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 --2- 172.21.15.44:0/2657082624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa410a8e0 0x7f1aa419e830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 >> 172.21.15.44:0/2657082624 conn(0x7f1aa407de10 msgr2=0x7f1aa4107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 shutdown_connections 2024-04-20T11:11:37.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:36.396+0000 7f1aa8a00700 1 -- 172.21.15.44:0/2657082624 wait complete. 2024-04-20T11:11:37.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:37 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2657082624' entity='client.admin' 2024-04-20T11:11:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:37 smithi044 ceph-mon[16855]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:11:39.124 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:39 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:39.124 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:39 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1038263649' entity='client.admin' 2024-04-20T11:11:39.124 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:39 smithi044 ceph-mon[16855]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:11:39.137 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.266+0000 7f2167600700 1 Processor -- start 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.266+0000 7f2167600700 1 -- start start 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.267+0000 7f2167600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216010b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.267+0000 7f2167600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f216007beb0 con 0x7f21601090d0 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.267+0000 7f2164c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216010b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.267+0000 7f2164c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216010b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39214/0 (socket says 172.21.15.44:39214) 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.267+0000 7f2164c00700 1 -- 172.21.15.44:0/389295834 learned_addr learned my addr 172.21.15.44:0/389295834 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.268+0000 7f2164c00700 1 -- 172.21.15.44:0/389295834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216010ba00 con 0x7f21601090d0 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.268+0000 7f2164c00700 1 --2- 172.21.15.44:0/389295834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216010b4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21540098a0 tx=0x7f2154009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562e7849b0fe0264 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:39.143 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.268+0000 7f215f400700 1 -- 172.21.15.44:0/389295834 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2154004160 con 0x7f21601090d0 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.268+0000 7f215f400700 1 -- 172.21.15.44:0/389295834 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2154004300 con 0x7f21601090d0 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.268+0000 7f215f400700 1 -- 172.21.15.44:0/389295834 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2154011bb0 con 0x7f21601090d0 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.269+0000 7f2167600700 1 -- 172.21.15.44:0/389295834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 msgr2=0x7f216010b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.269+0000 7f2167600700 1 --2- 172.21.15.44:0/389295834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216010b4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21540098a0 tx=0x7f2154009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.269+0000 7f2167600700 1 -- 172.21.15.44:0/389295834 shutdown_connections 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.269+0000 7f2167600700 1 --2- 172.21.15.44:0/389295834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216010b4c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.269+0000 7f2167600700 1 -- 172.21.15.44:0/389295834 >> 172.21.15.44:0/389295834 conn(0x7f2160102b40 msgr2=0x7f2160104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.269+0000 7f2167600700 1 -- 172.21.15.44:0/389295834 shutdown_connections 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.270+0000 7f2167600700 1 -- 172.21.15.44:0/389295834 wait complete. 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.271+0000 7f2167600700 1 Processor -- start 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.271+0000 7f2167600700 1 -- start start 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.271+0000 7f2167600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:39.144 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.271+0000 7f2167600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f216010cad0 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.271+0000 7f2164c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.272+0000 7f2164c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39220/0 (socket says 172.21.15.44:39220) 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.272+0000 7f2164c00700 1 -- 172.21.15.44:0/1038263649 learned_addr learned my addr 172.21.15.44:0/1038263649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.272+0000 7f2164c00700 1 -- 172.21.15.44:0/1038263649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216019ee50 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.272+0000 7f2164c00700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216019e910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.273+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2154011e60 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.273+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2154003b20 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.273+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216019f0e0 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.273+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21540247a0 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.273+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216019f540 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.274+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2154022020 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.274+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21600640b0 con 0x7f21601090d0 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.275+0000 7f215d600700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f2148039600 0x7f214803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:39.145 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.275+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f215404d630 con 0x7f21601090d0 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.275+0000 7f215fe00700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f2148039600 0x7f214803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.276+0000 7f215fe00700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f2148039600 0x7f214803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f214c006fd0 tx=0x7f214c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.281+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f215402ce10 con 0x7f21601090d0 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.508+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f21601a1ec0 con 0x7f21601090d0 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.515+0000 7f215d600700 1 -- 172.21.15.44:0/1038263649 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f21540212c0 con 0x7f21601090d0 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f2148039600 msgr2=0x7f214803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f2148039600 0x7f214803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f214c006fd0 tx=0x7f214c006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 msgr2=0x7f216019e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216019e910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154011ca0 comp rx=0 tx=0).stop 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 shutdown_connections 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f2148039600 0x7f214803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 --2- 172.21.15.44:0/1038263649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21601090d0 0x7f216019e910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.519+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 >> 172.21.15.44:0/1038263649 conn(0x7f2160102b40 msgr2=0x7f2160103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.520+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 shutdown_connections 2024-04-20T11:11:39.147 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:38.520+0000 7f2167600700 1 -- 172.21.15.44:0/1038263649 wait complete. 2024-04-20T11:11:39.147 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling the dashboard module... 2024-04-20T11:11:40.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3274896498' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-20T11:11:40.859 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:40 smithi044 ceph-mon[16855]: from='mgr.14120 172.21.15.44:0/705195468' entity='mgr.x' 2024-04-20T11:11:40.859 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: ignoring --setuser ceph since I am not root 2024-04-20T11:11:40.859 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: ignoring --setgroup ceph since I am not root 2024-04-20T11:11:40.859 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:40.857+0000 7fa5dcf5d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.772+0000 7f5240800700 1 Processor -- start 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.773+0000 7f5240800700 1 -- start start 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.773+0000 7f5240800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f5238107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.773+0000 7f5240800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5238107f50 con 0x7f5238107560 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.774+0000 7f523de00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f5238107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.774+0000 7f523de00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f5238107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45424/0 (socket says 172.21.15.44:45424) 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.774+0000 7f523de00700 1 -- 172.21.15.44:0/584306481 learned_addr learned my addr 172.21.15.44:0/584306481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:41.278 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.774+0000 7f523de00700 1 -- 172.21.15.44:0/584306481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5238108760 con 0x7f5238107560 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.775+0000 7f523de00700 1 --2- 172.21.15.44:0/584306481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f5238107980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f522c00bb40 tx=0x7f522c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7c3900cd1834f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.775+0000 7f523ca00700 1 -- 172.21.15.44:0/584306481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c00dbe0 con 0x7f5238107560 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.775+0000 7f523ca00700 1 -- 172.21.15.44:0/584306481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f522c004530 con 0x7f5238107560 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.775+0000 7f523ca00700 1 -- 172.21.15.44:0/584306481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c010490 con 0x7f5238107560 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.776+0000 7f5240800700 1 -- 172.21.15.44:0/584306481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 msgr2=0x7f5238107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.776+0000 7f5240800700 1 --2- 172.21.15.44:0/584306481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f5238107980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f522c00bb40 tx=0x7f522c00be50 comp rx=0 tx=0).stop 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.776+0000 7f5240800700 1 -- 172.21.15.44:0/584306481 shutdown_connections 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.776+0000 7f5240800700 1 --2- 172.21.15.44:0/584306481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f5238107980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.776+0000 7f5240800700 1 -- 172.21.15.44:0/584306481 >> 172.21.15.44:0/584306481 conn(0x7f5238102b40 msgr2=0x7f5238104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.777+0000 7f5240800700 1 -- 172.21.15.44:0/584306481 shutdown_connections 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.777+0000 7f5240800700 1 -- 172.21.15.44:0/584306481 wait complete. 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.778+0000 7f5240800700 1 Processor -- start 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.778+0000 7f5240800700 1 -- start start 2024-04-20T11:11:41.279 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.778+0000 7f5240800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f523819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.778+0000 7f5240800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f523807b150 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.778+0000 7f523de00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f523819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.779+0000 7f523de00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f523819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45438/0 (socket says 172.21.15.44:45438) 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.779+0000 7f523de00700 1 -- 172.21.15.44:0/3274896498 learned_addr learned my addr 172.21.15.44:0/3274896498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.779+0000 7f523de00700 1 -- 172.21.15.44:0/3274896498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523819ed40 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.779+0000 7f523de00700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f523819e800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f522c00d650 tx=0x7f522c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.779+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c010790 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.780+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f522c01d070 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.780+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c019d60 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.780+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523819efd0 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.780+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a1cc0 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.781+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52380640b0 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.781+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f522c028060 con 0x7f5238107560 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.781+0000 7f5232a00700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f52200395b0 0x7f522003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:41.280 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.781+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f522c04d8e0 con 0x7f5238107560 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.782+0000 7f523d400700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f52200395b0 0x7f522003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.783+0000 7f523d400700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f52200395b0 0x7f522003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5228006fd0 tx=0x7f5228006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:39.787+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f522c015070 con 0x7f5238107560 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.088+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f52381a1e90 con 0x7f5238107560 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.526+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f52381a1e90 con 0x7f5238107560 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.530+0000 7f5232a00700 1 -- 172.21.15.44:0/3274896498 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f522c01f030 con 0x7f5238107560 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f52200395b0 msgr2=0x7f522003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f52200395b0 0x7f522003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5228006fd0 tx=0x7f5228006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 msgr2=0x7f523819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f523819e800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f522c00d650 tx=0x7f522c00ded0 comp rx=0 tx=0).stop 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 shutdown_connections 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f52200395b0 0x7f522003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 --2- 172.21.15.44:0/3274896498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5238107560 0x7f523819e800 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.531+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 >> 172.21.15.44:0/3274896498 conn(0x7f5238102b40 msgr2=0x7f5238190a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.532+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 shutdown_connections 2024-04-20T11:11:41.281 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:40.532+0000 7f5240800700 1 -- 172.21.15.44:0/3274896498 wait complete. 2024-04-20T11:11:41.431 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:41.129+0000 7fa5dcf5d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:11:41.431 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:41.267+0000 7fa5dcf5d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:11:41.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:41 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3274896498' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-20T11:11:41.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:41 smithi044 ceph-mon[16855]: mgrmap e9: x(active, since 25s) 2024-04-20T11:11:42.010 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:41.698+0000 7fa5dcf5d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:11:42.010 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:41.817+0000 7fa5dcf5d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:11:42.810 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:42 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3323846129' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T11:11:43.792 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:43.445+0000 7fa5dcf5d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:11:43.792 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:43.683+0000 7fa5dcf5d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.880+0000 7f0d9e600700 1 Processor -- start 2024-04-20T11:11:43.811 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.880+0000 7f0d9e600700 1 -- start start 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.880+0000 7f0d9e600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d98108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.880+0000 7f0d9e600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98109060 con 0x7f0d98108670 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.880+0000 7f0d97400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d98108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.881+0000 7f0d97400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d98108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45476/0 (socket says 172.21.15.44:45476) 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.881+0000 7f0d97400700 1 -- 172.21.15.44:0/257696806 learned_addr learned my addr 172.21.15.44:0/257696806 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.881+0000 7f0d97400700 1 -- 172.21.15.44:0/257696806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d98109820 con 0x7f0d98108670 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.881+0000 7f0d97400700 1 --2- 172.21.15.44:0/257696806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d98108a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d880098a0 tx=0x7f0d88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1071a297670cd8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.882+0000 7f0d96000700 1 -- 172.21.15.44:0/257696806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d88004160 con 0x7f0d98108670 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.882+0000 7f0d96000700 1 -- 172.21.15.44:0/257696806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d88004300 con 0x7f0d98108670 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.882+0000 7f0d96000700 1 -- 172.21.15.44:0/257696806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d88011bb0 con 0x7f0d98108670 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 -- 172.21.15.44:0/257696806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 msgr2=0x7f0d98108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 --2- 172.21.15.44:0/257696806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d98108a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d880098a0 tx=0x7f0d88009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:43.812 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 -- 172.21.15.44:0/257696806 shutdown_connections 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 --2- 172.21.15.44:0/257696806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d98108a90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 -- 172.21.15.44:0/257696806 >> 172.21.15.44:0/257696806 conn(0x7f0d98103c50 msgr2=0x7f0d98106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 -- 172.21.15.44:0/257696806 shutdown_connections 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.883+0000 7f0d9e600700 1 -- 172.21.15.44:0/257696806 wait complete. 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.884+0000 7f0d9e600700 1 Processor -- start 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.884+0000 7f0d9e600700 1 -- start start 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.885+0000 7f0d9e600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d9819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.885+0000 7f0d9e600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9810cbd0 con 0x7f0d98108670 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.885+0000 7f0d97400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d9819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.885+0000 7f0d97400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d9819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45490/0 (socket says 172.21.15.44:45490) 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.885+0000 7f0d97400700 1 -- 172.21.15.44:0/3323846129 learned_addr learned my addr 172.21.15.44:0/3323846129 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.885+0000 7f0d97400700 1 -- 172.21.15.44:0/3323846129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9819cba0 con 0x7f0d98108670 2024-04-20T11:11:43.813 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.886+0000 7f0d97400700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d9819c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000c00 tx=0x7f0d88004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.886+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d88011e60 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.886+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9819ce30 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.886+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d88003af0 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.886+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d880246a0 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.886+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9819d290 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.888+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d980640b0 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.888+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f0d88022020 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.889+0000 7f0d8fe00700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 0x7f0d7c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.889+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0d8804d830 con 0x7f0d98108670 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.889+0000 7f0d96a00700 1 -- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 msgr2=0x7f0d7c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.889+0000 7f0d96a00700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 0x7f0d7c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:11:43.814 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:41.895+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0d8802a2a0 con 0x7f0d98108670 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.090+0000 7f0d96a00700 1 -- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 msgr2=0x7f0d7c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.090+0000 7f0d96a00700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 0x7f0d7c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.202+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0d9819fc40 con 0x7f0d98108670 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.202+0000 7f0d8fe00700 1 -- 172.21.15.44:0/3323846129 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f0d8804d180 con 0x7f0d98108670 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 msgr2=0x7f0d7c03bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 0x7f0d7c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:43.815 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 msgr2=0x7f0d9819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d9819c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000c00 tx=0x7f0d88004620 comp rx=0 tx=0).stop 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 shutdown_connections 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f0d7c0396b0 0x7f0d7c03bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 --2- 172.21.15.44:0/3323846129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d98108670 0x7f0d9819c660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 >> 172.21.15.44:0/3323846129 conn(0x7f0d98103c50 msgr2=0x7f0d98104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 shutdown_connections 2024-04-20T11:11:43.816 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:42.205+0000 7f0d9e600700 1 -- 172.21.15.44:0/3323846129 wait complete. 2024-04-20T11:11:43.817 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2024-04-20T11:11:43.817 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 9... 2024-04-20T11:11:44.193 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:43.930+0000 7fa5dcf5d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:11:44.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:44.336+0000 7fa5dcf5d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:11:44.724 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:44.463+0000 7fa5dcf5d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:11:44.725 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:44.723+0000 7fa5dcf5d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:11:45.284 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:44.987+0000 7fa5dcf5d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:11:46.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:45.805+0000 7fa5dcf5d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:11:46.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:46.395+0000 7fa5dcf5d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:11:46.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:46.527+0000 7fa5dcf5d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:11:47.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:46.803+0000 7fa5dcf5d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:11:47.939 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:47.476+0000 7fa5dcf5d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:11:47.939 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:47.605+0000 7fa5dcf5d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:11:47.939 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:47.937+0000 7fa5dcf5d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:11:48.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:48.121+0000 7fa5dcf5d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:11:48.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:48.252+0000 7fa5dcf5d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:11:48.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:48.408+0000 7fa5dcf5d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:11:49.194 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:48.796+0000 7fa5dcf5d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:11:49.888 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:49.588+0000 7fa5dcf5d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:11:50.194 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:11:49.887+0000 7fa5dcf5d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: Active manager daemon x restarted 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: Activating manager daemon x 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: osdmap e3: 0 total, 0 up, 0 in 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: mgrmap e10: x(active, starting, since 0.0103189s) 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:11:50.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:49 smithi044 ceph-mon[16855]: Manager daemon x is now available 2024-04-20T11:11:51.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:11:51.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:11:51.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T11:11:51.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:50 smithi044 ceph-mon[16855]: mgrmap e11: x(active, since 1.01393s) 2024-04-20T11:11:51.534 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-04-20T11:11:51.534 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-20T11:11:51.534 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.481+0000 7f04cea00700 1 Processor -- start 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.481+0000 7f04cea00700 1 -- start start 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.481+0000 7f04cea00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c8108ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.481+0000 7f04cea00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8109090 con 0x7f04c81086a0 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.482+0000 7f04c7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c8108ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.482+0000 7f04c7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c8108ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45506/0 (socket says 172.21.15.44:45506) 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.482+0000 7f04c7e00700 1 -- 172.21.15.44:0/4247827459 learned_addr learned my addr 172.21.15.44:0/4247827459 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.482+0000 7f04c7e00700 1 -- 172.21.15.44:0/4247827459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c8109850 con 0x7f04c81086a0 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.482+0000 7f04c7e00700 1 --2- 172.21.15.44:0/4247827459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c8108ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098a0 tx=0x7f04b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a621930d36a8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.483+0000 7f04c6a00700 1 -- 172.21.15.44:0/4247827459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b8004160 con 0x7f04c81086a0 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.483+0000 7f04c6a00700 1 -- 172.21.15.44:0/4247827459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04b8004300 con 0x7f04c81086a0 2024-04-20T11:11:51.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.483+0000 7f04c6a00700 1 -- 172.21.15.44:0/4247827459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b8011bb0 con 0x7f04c81086a0 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.483+0000 7f04cea00700 1 -- 172.21.15.44:0/4247827459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 msgr2=0x7f04c8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.483+0000 7f04cea00700 1 --2- 172.21.15.44:0/4247827459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c8108ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098a0 tx=0x7f04b8009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.484+0000 7f04cea00700 1 -- 172.21.15.44:0/4247827459 shutdown_connections 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.484+0000 7f04cea00700 1 --2- 172.21.15.44:0/4247827459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c8108ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.484+0000 7f04cea00700 1 -- 172.21.15.44:0/4247827459 >> 172.21.15.44:0/4247827459 conn(0x7f04c8103cc0 msgr2=0x7f04c81060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.484+0000 7f04cea00700 1 -- 172.21.15.44:0/4247827459 shutdown_connections 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.484+0000 7f04cea00700 1 -- 172.21.15.44:0/4247827459 wait complete. 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.485+0000 7f04cea00700 1 Processor -- start 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.485+0000 7f04cea00700 1 -- start start 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.485+0000 7f04cea00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.485+0000 7f04cea00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810cc00 con 0x7f04c81086a0 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.485+0000 7f04c7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.485+0000 7f04c7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45520/0 (socket says 172.21.15.44:45520) 2024-04-20T11:11:51.536 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.486+0000 7f04c7e00700 1 -- 172.21.15.44:0/3644106121 learned_addr learned my addr 172.21.15.44:0/3644106121 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.486+0000 7f04c7e00700 1 -- 172.21.15.44:0/3644106121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c807e0b0 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.486+0000 7f04c7e00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c807db70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009520 tx=0x7f04b8011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.486+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b8011e60 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.486+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c807c2a0 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.486+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04b80039c0 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.487+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b801a5a0 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.487+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c807c760 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.488+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f04b801a860 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.488+0000 7f04c4c00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.488+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 --> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04ac03c1d0 con 0x7f04ac039600 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.488+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f04b804da20 con 0x7f04c81086a0 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.488+0000 7f04c7400700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 msgr2=0x7f04ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.488+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.689+0000 7f04c7400700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 msgr2=0x7f04ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:51.537 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:44.689+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:45.090+0000 7f04c7400700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 msgr2=0x7f04ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:45.090+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:45.891+0000 7f04c7400700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 msgr2=0x7f04ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:45.891+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:47.493+0000 7f04c7400700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 msgr2=0x7f04ac03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/1299451251 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:47.493+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:49.898+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f04b801aba0 con 0x7f04c81086a0 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:49.898+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 msgr2=0x7f04ac03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:49.898+0000 7f04c4c00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.900+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f04b804f790 con 0x7f04c81086a0 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.900+0000 7f04c4c00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04ac03cfe0 0x7f04ac03f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.901+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04b800b860 con 0x7f04ac03cfe0 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.901+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04ac03cfe0 0x7f04ac03f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:51.538 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.902+0000 7f04c7400700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04ac03cfe0 0x7f04ac03f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04b4003a10 tx=0x7f04b40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.903+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f04b800b860 con 0x7f04ac03cfe0 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.911+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f04c810cd80 con 0x7f04ac03cfe0 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04c4c00700 1 -- 172.21.15.44:0/3644106121 <== mgr.14152 v2:172.21.15.44:6800/2425002048 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f04c810cd80 con 0x7f04ac03cfe0 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04ac03cfe0 msgr2=0x7f04ac03f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04ac03cfe0 0x7f04ac03f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04b4003a10 tx=0x7f04b40092b0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 msgr2=0x7f04c807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c807db70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009520 tx=0x7f04b8011c30 comp rx=0 tx=0).stop 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 shutdown_connections 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04ac03cfe0 0x7f04ac03f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04c81086a0 0x7f04c807db70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 --2- 172.21.15.44:0/3644106121 >> [v2:172.21.15.44:6800/1299451251,v1:172.21.15.44:6801/1299451251] conn(0x7f04ac039600 0x7f04ac03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 >> 172.21.15.44:0/3644106121 conn(0x7f04c8103cc0 msgr2=0x7f04c8104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 shutdown_connections 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:50.912+0000 7f04cea00700 1 -- 172.21.15.44:0/3644106121 wait complete. 2024-04-20T11:11:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 9 is available 2024-04-20T11:11:51.540 INFO:teuthology.orchestra.run.smithi044.stdout:Generating a dashboard self-signed certificate... 2024-04-20T11:11:52.160 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:51 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:50] ENGINE Bus STARTING 2024-04-20T11:11:52.161 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:51 smithi044 ceph-mon[16855]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T11:11:52.161 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:51 smithi044 ceph-mon[16855]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T11:11:52.161 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:51 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:50] ENGINE Serving on http://172.21.15.44:8765 2024-04-20T11:11:52.161 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:51 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:51] ENGINE Serving on https://172.21.15.44:7150 2024-04-20T11:11:52.161 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:51 smithi044 ceph-mon[16855]: [20/Apr/2024:11:11:51] ENGINE Bus STARTED 2024-04-20T11:11:53.130 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:52 smithi044 ceph-mon[16855]: mgrmap e12: x(active, since 2s) 2024-04-20T11:11:53.130 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.875+0000 7f7771a00700 1 Processor -- start 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.876+0000 7f7771a00700 1 -- start start 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.876+0000 7f7771a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.876+0000 7f7771a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f776c109060 con 0x7f776c108670 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.876+0000 7f776aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.876+0000 7f776aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54442/0 (socket says 172.21.15.44:54442) 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.876+0000 7f776aa00700 1 -- 172.21.15.44:0/654198546 learned_addr learned my addr 172.21.15.44:0/654198546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.877+0000 7f776aa00700 1 -- 172.21.15.44:0/654198546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c109820 con 0x7f776c108670 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.877+0000 7f776aa00700 1 --2- 172.21.15.44:0/654198546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c108a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098a0 tx=0x7f775c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37b32140c400ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.878+0000 7f7769600700 1 -- 172.21.15.44:0/654198546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c004160 con 0x7f776c108670 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.878+0000 7f7769600700 1 -- 172.21.15.44:0/654198546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f775c004300 con 0x7f776c108670 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.878+0000 7f7769600700 1 -- 172.21.15.44:0/654198546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c011bb0 con 0x7f776c108670 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.878+0000 7f7771a00700 1 -- 172.21.15.44:0/654198546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 msgr2=0x7f776c108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.878+0000 7f7771a00700 1 --2- 172.21.15.44:0/654198546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c108a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098a0 tx=0x7f775c009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.879+0000 7f7771a00700 1 -- 172.21.15.44:0/654198546 shutdown_connections 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.879+0000 7f7771a00700 1 --2- 172.21.15.44:0/654198546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c108a90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.879+0000 7f7771a00700 1 -- 172.21.15.44:0/654198546 >> 172.21.15.44:0/654198546 conn(0x7f776c103c50 msgr2=0x7f776c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.879+0000 7f7771a00700 1 -- 172.21.15.44:0/654198546 shutdown_connections 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.879+0000 7f7771a00700 1 -- 172.21.15.44:0/654198546 wait complete. 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.880+0000 7f7771a00700 1 Processor -- start 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.880+0000 7f7771a00700 1 -- start start 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.881+0000 7f7771a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.881+0000 7f7771a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10cbd0 con 0x7f776c108670 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.881+0000 7f776aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:54.043 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.881+0000 7f776aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54444/0 (socket says 172.21.15.44:54444) 2024-04-20T11:11:54.044 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.881+0000 7f776aa00700 1 -- 172.21.15.44:0/1708908297 learned_addr learned my addr 172.21.15.44:0/1708908297 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:54.044 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.882+0000 7f776aa00700 1 -- 172.21.15.44:0/1708908297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c19edb0 con 0x7f776c108670 2024-04-20T11:11:54.044 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.882+0000 7f776aa00700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c19e870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f775c000c00 tx=0x7f775c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:54.044 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.882+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c011e60 con 0x7f776c108670 2024-04-20T11:11:54.044 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.882+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f775c003af0 con 0x7f776c108670 2024-04-20T11:11:54.044 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.882+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c0246a0 con 0x7f776c108670 2024-04-20T11:11:54.046 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.882+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c19f040 con 0x7f776c108670 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.883+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c19f4a0 con 0x7f776c108670 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.884+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c0640b0 con 0x7f776c108670 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.884+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f775c022020 con 0x7f776c108670 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.884+0000 7f7763400700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7750039660 0x7f775003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.885+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f775c04d8e0 con 0x7f776c108670 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.885+0000 7f776a000700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7750039660 0x7f775003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.886+0000 7f776a000700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7750039660 0x7f775003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7758006fd0 tx=0x7f7758006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:52.891+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f775c0212c0 con 0x7f776c108670 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.129+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f776c104e80 con 0x7f7750039660 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.286+0000 7f7763400700 1 -- 172.21.15.44:0/1708908297 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f776c104e80 con 0x7f7750039660 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7750039660 msgr2=0x7f775003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7750039660 0x7f775003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7758006fd0 tx=0x7f7758006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 msgr2=0x7f776c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:54.047 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c19e870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f775c000c00 tx=0x7f775c004620 comp rx=0 tx=0).stop 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 shutdown_connections 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7750039660 0x7f775003bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 --2- 172.21.15.44:0/1708908297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f776c108670 0x7f776c19e870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.289+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 >> 172.21.15.44:0/1708908297 conn(0x7f776c103c50 msgr2=0x7f776c104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.290+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 shutdown_connections 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:53.290+0000 7f7771a00700 1 -- 172.21.15.44:0/1708908297 wait complete. 2024-04-20T11:11:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial admin user... 2024-04-20T11:11:54.379 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:54 smithi044 ceph-mon[16855]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:54.379 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:54 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:11:54.379 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:54 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:11:56.013 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:11:56.013 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:55 smithi044 ceph-mon[16855]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:11:56.013 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:11:56.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pKrDfBNAVm8oh/ozKCQY4OFqQ/7j9yPrGJGAajfxUcH77US4UYDQ.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713611515, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-20T11:11:56.026 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.705+0000 7f4087e00700 1 Processor -- start 2024-04-20T11:11:56.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.705+0000 7f4087e00700 1 -- start start 2024-04-20T11:11:56.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.705+0000 7f4087e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f4080108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:56.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.705+0000 7f4087e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4080109060 con 0x7f4080108670 2024-04-20T11:11:56.027 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.706+0000 7f4085400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f4080108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.706+0000 7f4085400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f4080108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54454/0 (socket says 172.21.15.44:54454) 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.706+0000 7f4085400700 1 -- 172.21.15.44:0/3797536145 learned_addr learned my addr 172.21.15.44:0/3797536145 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.707+0000 7f4085400700 1 -- 172.21.15.44:0/3797536145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4080109820 con 0x7f4080108670 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.707+0000 7f4085400700 1 --2- 172.21.15.44:0/3797536145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f4080108a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40740098a0 tx=0x7f4074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e026cb70a56cafc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.707+0000 7f407fe00700 1 -- 172.21.15.44:0/3797536145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074004160 con 0x7f4080108670 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.707+0000 7f407fe00700 1 -- 172.21.15.44:0/3797536145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4074004300 con 0x7f4080108670 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.707+0000 7f407fe00700 1 -- 172.21.15.44:0/3797536145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074011bb0 con 0x7f4080108670 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 -- 172.21.15.44:0/3797536145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 msgr2=0x7f4080108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:56.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 --2- 172.21.15.44:0/3797536145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f4080108a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40740098a0 tx=0x7f4074009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 -- 172.21.15.44:0/3797536145 shutdown_connections 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 --2- 172.21.15.44:0/3797536145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f4080108a90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 -- 172.21.15.44:0/3797536145 >> 172.21.15.44:0/3797536145 conn(0x7f4080103c50 msgr2=0x7f4080106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 -- 172.21.15.44:0/3797536145 shutdown_connections 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.708+0000 7f4087e00700 1 -- 172.21.15.44:0/3797536145 wait complete. 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.709+0000 7f4087e00700 1 Processor -- start 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.709+0000 7f4087e00700 1 -- start start 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.710+0000 7f4087e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f408007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.710+0000 7f4087e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f408010cbd0 con 0x7f4080108670 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.710+0000 7f4085400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f408007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.710+0000 7f4085400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f408007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54470/0 (socket says 172.21.15.44:54470) 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.710+0000 7f4085400700 1 -- 172.21.15.44:0/2328021033 learned_addr learned my addr 172.21.15.44:0/2328021033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:56.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.710+0000 7f4085400700 1 -- 172.21.15.44:0/2328021033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408007e0b0 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.711+0000 7f4085400700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f408007db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4074011bf0 tx=0x7f4074011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.711+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074003930 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.711+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408007c2a0 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.711+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4074003ad0 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.711+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074019760 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.711+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408007c760 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.712+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4074019a60 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.713+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40800640b0 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.713+0000 7f407e000700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f40680395b0 0x7f406803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.713+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f407404e2f0 con 0x7f4080108670 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.713+0000 7f4084a00700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f40680395b0 0x7f406803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.714+0000 7f4084a00700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f40680395b0 0x7f406803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:56.034 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.721+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f407401a7b0 con 0x7f4080108670 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:54.980+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f408007c9e0 con 0x7f40680395b0 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.250+0000 7f407e000700 1 -- 172.21.15.44:0/2328021033 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f408007c9e0 con 0x7f40680395b0 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.252+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f40680395b0 msgr2=0x7f406803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.252+0000 7f4087e00700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f40680395b0 0x7f406803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c006e40 comp rx=0 tx=0).stop 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 msgr2=0x7f408007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f408007db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4074011bf0 tx=0x7f4074011cd0 comp rx=0 tx=0).stop 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 shutdown_connections 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f40680395b0 0x7f406803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 --2- 172.21.15.44:0/2328021033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4080108670 0x7f408007db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 >> 172.21.15.44:0/2328021033 conn(0x7f4080103c50 msgr2=0x7f40801047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 shutdown_connections 2024-04-20T11:11:56.035 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:55.253+0000 7f4087e00700 1 -- 172.21.15.44:0/2328021033 wait complete. 2024-04-20T11:11:56.036 INFO:teuthology.orchestra.run.smithi044.stdout:Fetching dashboard port number... 2024-04-20T11:11:58.089 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:57 smithi044 ceph-mon[16855]: mgrmap e13: x(active, since 6s) 2024-04-20T11:11:58.090 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:11:57 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/873033063' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-20T11:11:58.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 8443 2024-04-20T11:11:58.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.664+0000 7f9ed9a00700 1 Processor -- start 2024-04-20T11:11:58.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.664+0000 7f9ed9a00700 1 -- start start 2024-04-20T11:11:58.556 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.664+0000 7f9ed9a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed4107380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:58.557 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.664+0000 7f9ed9a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed407be10 con 0x7f9ed4106f60 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.665+0000 7f9ed2a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed4107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.665+0000 7f9ed2a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed4107380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54482/0 (socket says 172.21.15.44:54482) 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.665+0000 7f9ed2a00700 1 -- 172.21.15.44:0/3347708743 learned_addr learned my addr 172.21.15.44:0/3347708743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.665+0000 7f9ed2a00700 1 -- 172.21.15.44:0/3347708743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed41078c0 con 0x7f9ed4106f60 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.666+0000 7f9ed2a00700 1 --2- 172.21.15.44:0/3347708743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed4107380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40098a0 tx=0x7f9ec4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bad223ad892e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.666+0000 7f9ed2000700 1 -- 172.21.15.44:0/3347708743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ec4004160 con 0x7f9ed4106f60 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.666+0000 7f9ed2000700 1 -- 172.21.15.44:0/3347708743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ec4004300 con 0x7f9ed4106f60 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.666+0000 7f9ed2000700 1 -- 172.21.15.44:0/3347708743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ec4011bb0 con 0x7f9ed4106f60 2024-04-20T11:11:58.558 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 -- 172.21.15.44:0/3347708743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 msgr2=0x7f9ed4107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 --2- 172.21.15.44:0/3347708743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed4107380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40098a0 tx=0x7f9ec4009bb0 comp rx=0 tx=0).stop 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 -- 172.21.15.44:0/3347708743 shutdown_connections 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 --2- 172.21.15.44:0/3347708743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed4107380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 -- 172.21.15.44:0/3347708743 >> 172.21.15.44:0/3347708743 conn(0x7f9ed4102b40 msgr2=0x7f9ed4104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 -- 172.21.15.44:0/3347708743 shutdown_connections 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.667+0000 7f9ed9a00700 1 -- 172.21.15.44:0/3347708743 wait complete. 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.668+0000 7f9ed9a00700 1 Processor -- start 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.668+0000 7f9ed9a00700 1 -- start start 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.669+0000 7f9ed9a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.669+0000 7f9ed9a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41089f0 con 0x7f9ed4106f60 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.669+0000 7f9ed2a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.669+0000 7f9ed2a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54498/0 (socket says 172.21.15.44:54498) 2024-04-20T11:11:58.561 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.669+0000 7f9ed2a00700 1 -- 172.21.15.44:0/873033063 learned_addr learned my addr 172.21.15.44:0/873033063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:11:58.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.669+0000 7f9ed2a00700 1 -- 172.21.15.44:0/873033063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed419ed40 con 0x7f9ed4106f60 2024-04-20T11:11:58.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.670+0000 7f9ed2a00700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed419e800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4000c00 tx=0x7f9ec4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:58.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.670+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ec4011e60 con 0x7f9ed4106f60 2024-04-20T11:11:58.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.670+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ec4003b20 con 0x7f9ed4106f60 2024-04-20T11:11:58.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.670+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed419efd0 con 0x7f9ed4106f60 2024-04-20T11:11:58.562 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.670+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ec40247a0 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.670+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed419f430 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.671+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed40640b0 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.672+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9ec4022020 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.672+0000 7f9ecbe00700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9eb8039660 0x7f9eb803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.673+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ec404d870 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.673+0000 7f9eca000700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9eb8039660 0x7f9eb803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.674+0000 7f9eca000700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9eb8039660 0x7f9eb803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00ad30 tx=0x7f9ebc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.678+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9ec4016260 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.837+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9ec4024940 con 0x7f9ed4106f60 2024-04-20T11:11:58.565 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.920+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f9ed4063660 con 0x7f9ed4106f60 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.921+0000 7f9ecbe00700 1 -- 172.21.15.44:0/873033063 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9ec4044e20 con 0x7f9ed4106f60 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.924+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9eb8039660 msgr2=0x7f9eb803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.924+0000 7f9ed9a00700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9eb8039660 0x7f9eb803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00ad30 tx=0x7f9ebc0093f0 comp rx=0 tx=0).stop 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.924+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 msgr2=0x7f9ed419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.924+0000 7f9ed9a00700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed419e800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4000c00 tx=0x7f9ec4011ca0 comp rx=0 tx=0).stop 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.925+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 shutdown_connections 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.925+0000 7f9ed9a00700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9eb8039660 0x7f9eb803bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.925+0000 7f9ed9a00700 1 --2- 172.21.15.44:0/873033063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ed4106f60 0x7f9ed419e800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.925+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 >> 172.21.15.44:0/873033063 conn(0x7f9ed4102b40 msgr2=0x7f9ed4195470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.925+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 shutdown_connections 2024-04-20T11:11:58.566 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:56.925+0000 7f9ed9a00700 1 -- 172.21.15.44:0/873033063 wait complete. 2024-04-20T11:11:58.600 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T11:11:58.600 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-04-20T11:11:58.625 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T11:11:58.625 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-04-20T11:11:58.626 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-04-20T11:11:58.626 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-20T11:11:58.633 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph Dashboard is now available at: 2024-04-20T11:11:58.633 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:11:58.633 INFO:teuthology.orchestra.run.smithi044.stdout: URL: https://smithi044.front.sepia.ceph.com:8443/ 2024-04-20T11:11:58.634 INFO:teuthology.orchestra.run.smithi044.stdout: User: admin 2024-04-20T11:11:58.634 INFO:teuthology.orchestra.run.smithi044.stdout: Password: hghq8x453b 2024-04-20T11:11:58.634 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:11:58.634 INFO:teuthology.orchestra.run.smithi044.stdout:Saving cluster configuration to /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config directory 2024-04-20T11:11:58.637 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling autotune for osd_memory_target 2024-04-20T11:12:00.210 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.264+0000 7f1ccf000700 1 Processor -- start 2024-04-20T11:12:00.211 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.264+0000 7f1ccf000700 1 -- start start 2024-04-20T11:12:00.211 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.265+0000 7f1ccf000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:00.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.265+0000 7f1ccf000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc807c820 con 0x7f1cc807de30 2024-04-20T11:12:00.212 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.265+0000 7f1cc7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.265+0000 7f1cc7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53466/0 (socket says 172.21.15.44:53466) 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.265+0000 7f1cc7e00700 1 -- 172.21.15.44:0/691063471 learned_addr learned my addr 172.21.15.44:0/691063471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.266+0000 7f1cc7e00700 1 -- 172.21.15.44:0/691063471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc807c9a0 con 0x7f1cc807de30 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.266+0000 7f1cc7e00700 1 --2- 172.21.15.44:0/691063471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc807c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009e80 tx=0x7f1cb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07149181929bed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.266+0000 7f1cc7400700 1 -- 172.21.15.44:0/691063471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cb8017070 con 0x7f1cc807de30 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1cc7400700 1 -- 172.21.15.44:0/691063471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1cb800b8c0 con 0x7f1cc807de30 2024-04-20T11:12:00.213 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1cc7400700 1 -- 172.21.15.44:0/691063471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cb8004580 con 0x7f1cc807de30 2024-04-20T11:12:00.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1ccf000700 1 -- 172.21.15.44:0/691063471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 msgr2=0x7f1cc807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:00.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1ccf000700 1 --2- 172.21.15.44:0/691063471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc807c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009e80 tx=0x7f1cb800b330 comp rx=0 tx=0).stop 2024-04-20T11:12:00.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1ccf000700 1 -- 172.21.15.44:0/691063471 shutdown_connections 2024-04-20T11:12:00.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1ccf000700 1 --2- 172.21.15.44:0/691063471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc807c2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:00.215 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.267+0000 7f1ccf000700 1 -- 172.21.15.44:0/691063471 >> 172.21.15.44:0/691063471 conn(0x7f1cc8103c50 msgr2=0x7f1cc8106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.268+0000 7f1ccf000700 1 -- 172.21.15.44:0/691063471 shutdown_connections 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.268+0000 7f1ccf000700 1 -- 172.21.15.44:0/691063471 wait complete. 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.269+0000 7f1ccf000700 1 Processor -- start 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.269+0000 7f1ccf000700 1 -- start start 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.269+0000 7f1ccf000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.269+0000 7f1ccf000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810ec90 con 0x7f1cc807de30 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.270+0000 7f1cc7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.270+0000 7f1cc7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53472/0 (socket says 172.21.15.44:53472) 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.270+0000 7f1cc7e00700 1 -- 172.21.15.44:0/4046173573 learned_addr learned my addr 172.21.15.44:0/4046173573 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:00.216 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.270+0000 7f1cc7e00700 1 -- 172.21.15.44:0/4046173573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc819ed70 con 0x7f1cc807de30 2024-04-20T11:12:00.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.270+0000 7f1cc7e00700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc819e830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009530 tx=0x7f1cb8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:00.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.271+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cb8021020 con 0x7f1cc807de30 2024-04-20T11:12:00.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.271+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1cb80048a0 con 0x7f1cc807de30 2024-04-20T11:12:00.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.271+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cb8017040 con 0x7f1cc807de30 2024-04-20T11:12:00.219 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.271+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc819f000 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.271+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc81a1cf0 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.272+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1cb8024020 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.272+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc80640b0 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.273+0000 7f1cc5600700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1cac0396f0 0x7f1cac03bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.273+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1cb800f050 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.273+0000 7f1cbf400700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1cac0396f0 0x7f1cac03bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.274+0000 7f1cbf400700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1cac0396f0 0x7f1cac03bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.279+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1cb802b020 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.510+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f1cc81a1ec0 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.511+0000 7f1cc5600700 1 -- 172.21.15.44:0/4046173573 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f1cc81a1ec0 con 0x7f1cc807de30 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1cac0396f0 msgr2=0x7f1cac03bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1cac0396f0 0x7f1cac03bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 msgr2=0x7f1cc819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc819e830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009530 tx=0x7f1cb8003d00 comp rx=0 tx=0).stop 2024-04-20T11:12:00.220 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 shutdown_connections 2024-04-20T11:12:00.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1cac0396f0 0x7f1cac03bbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:00.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 --2- 172.21.15.44:0/4046173573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc807de30 0x7f1cc819e830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:00.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.514+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 >> 172.21.15.44:0/4046173573 conn(0x7f1cc8103c50 msgr2=0x7f1cc8104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:00.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.515+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 shutdown_connections 2024-04-20T11:12:00.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:11:59.515+0000 7f1ccf000700 1 -- 172.21.15.44:0/4046173573 wait complete. 2024-04-20T11:12:01.788 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.819+0000 7f01d6c00700 1 Processor -- start 2024-04-20T11:12:01.788 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.819+0000 7f01d6c00700 1 -- start start 2024-04-20T11:12:01.788 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.820+0000 7f01d6c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d010ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:01.788 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.820+0000 7f01d6c00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01d010b2b0 con 0x7f01d010a8c0 2024-04-20T11:12:01.788 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.820+0000 7f01cfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d010ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.820+0000 7f01cfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d010ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53474/0 (socket says 172.21.15.44:53474) 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.820+0000 7f01cfe00700 1 -- 172.21.15.44:0/154720486 learned_addr learned my addr 172.21.15.44:0/154720486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.821+0000 7f01cfe00700 1 -- 172.21.15.44:0/154720486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d010ba70 con 0x7f01d010a8c0 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.821+0000 7f01cfe00700 1 --2- 172.21.15.44:0/154720486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d010ace0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01c00098a0 tx=0x7f01c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2af3231cd833b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01cea00700 1 -- 172.21.15.44:0/154720486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c0004160 con 0x7f01d010a8c0 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01cea00700 1 -- 172.21.15.44:0/154720486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f01c0004300 con 0x7f01d010a8c0 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01cea00700 1 -- 172.21.15.44:0/154720486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c0011bb0 con 0x7f01d010a8c0 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01d6c00700 1 -- 172.21.15.44:0/154720486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 msgr2=0x7f01d010ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01d6c00700 1 --2- 172.21.15.44:0/154720486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d010ace0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01c00098a0 tx=0x7f01c0009bb0 comp rx=0 tx=0).stop 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01d6c00700 1 -- 172.21.15.44:0/154720486 shutdown_connections 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01d6c00700 1 --2- 172.21.15.44:0/154720486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d010ace0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01d6c00700 1 -- 172.21.15.44:0/154720486 >> 172.21.15.44:0/154720486 conn(0x7f01d007de10 msgr2=0x7f01d01084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:01.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.822+0000 7f01d6c00700 1 -- 172.21.15.44:0/154720486 shutdown_connections 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.823+0000 7f01d6c00700 1 -- 172.21.15.44:0/154720486 wait complete. 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.823+0000 7f01d6c00700 1 Processor -- start 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.824+0000 7f01d6c00700 1 -- start start 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.824+0000 7f01d6c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.824+0000 7f01d6c00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01d010ee20 con 0x7f01d010a8c0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.824+0000 7f01cfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.824+0000 7f01cfe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53482/0 (socket says 172.21.15.44:53482) 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.824+0000 7f01cfe00700 1 -- 172.21.15.44:0/2353057153 learned_addr learned my addr 172.21.15.44:0/2353057153 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.825+0000 7f01cfe00700 1 -- 172.21.15.44:0/2353057153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d019ec80 con 0x7f01d010a8c0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.825+0000 7f01cfe00700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d019e740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01c0011bf0 tx=0x7f01c0011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.825+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c0003930 con 0x7f01d010a8c0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.825+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f01c0003ad0 con 0x7f01d010a8c0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.825+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c0019760 con 0x7f01d010a8c0 2024-04-20T11:12:01.793 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.825+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d019ef10 con 0x7f01d010a8c0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.826+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d019f370 con 0x7f01d010a8c0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.827+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d00640b0 con 0x7f01d010a8c0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.827+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f01c001ab50 con 0x7f01d010a8c0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.828+0000 7f01ccc00700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01b40397c0 0x7f01b403bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.828+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f01c004ddf0 con 0x7f01d010a8c0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.828+0000 7f01cf400700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01b40397c0 0x7f01b403bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.829+0000 7f01cf400700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01b40397c0 0x7f01b403bc80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:01.796 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:00.833+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f01c001a430 con 0x7f01d010a8c0 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.187+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f01d01a1d40 con 0x7f01d010a8c0 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.195+0000 7f01ccc00700 1 -- 172.21.15.44:0/2353057153 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f01c002dc90 con 0x7f01d010a8c0 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.197+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01b40397c0 msgr2=0x7f01b403bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.197+0000 7f01d6c00700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01b40397c0 0x7f01b403bc80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.197+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 msgr2=0x7f01d019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.197+0000 7f01d6c00700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d019e740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01c0011bf0 tx=0x7f01c0011cd0 comp rx=0 tx=0).stop 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.198+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 shutdown_connections 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.198+0000 7f01d6c00700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01b40397c0 0x7f01b403bc80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.198+0000 7f01d6c00700 1 --2- 172.21.15.44:0/2353057153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01d010a8c0 0x7f01d019e740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.198+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 >> 172.21.15.44:0/2353057153 conn(0x7f01d007de10 msgr2=0x7f01d0107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.198+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 shutdown_connections 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr 2024-04-20T11:12:01.198+0000 7f01d6c00700 1 -- 172.21.15.44:0/2353057153 wait complete. 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.797 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout:Or, if you are only running a single cluster on this host: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: ceph telemetry on 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout:For more information see: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:01.798 INFO:teuthology.orchestra.run.smithi044.stdout:Bootstrap complete. 2024-04-20T11:12:01.908 INFO:tasks.cephadm:Fetching config... 2024-04-20T11:12:01.909 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:12:01.909 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-20T11:12:01.964 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-20T11:12:01.965 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:12:01.965 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-20T11:12:02.015 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-20T11:12:02.015 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:12:02.015 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-20T11:12:02.082 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-20T11:12:02.082 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:12:02.082 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-20T11:12:02.175 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-20T11:12:02.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T11:12:02.357 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:12:02.380 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:02 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2353057153' entity='client.admin' 2024-04-20T11:12:02.392 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T11:12:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:12:02.441 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T11:12:02.482 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jokyoZtXsEsBSCceZZQ1BkjgPYcSlnQm0cfHBHiFHy7d89sU3IOvJhqD6uF1/L3geGYrO5F+21s89ITLrY0FvzLYJFXzrzzb7NkQ0vIt+3dLKONZVBM6CfhnrmM8VzshvPrkUuL3a94ETgCJhsxS6arznjqLzbHwOZ+eC5BB1SSg0UCOwMesgKCnnDcwzTeNvKtThiSjeBFZwOHSbT4+xRXP5BIkHmndzSuB6+G531MLVaKvtMYCGBbgh/LonSmYaBNlaSRpvkP6ZQb+7ymXZ8IsNpI77KAgMD29pGq2B+zWjidxE9dlPSvXjzb+KT7/H9R2+dt2npm+qnyMl55nIydEYfTjmRN6BniYq8jlqDUGK6O95HrhNkyJQmVaUzpgcLecCEOUYmOBBUQ3kiCabTSXIl/zpM8SuCcc/uwpSwTDGcuiRiON3VA1bNMlKGt7lSPP5lklqQroX1NPo0xEtBeRcPDXSOdTfpMhh7ANJ7eUfLiX/ZG6dz/Ou4Yi9Rc= ceph-759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:12:02.494 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-20T11:12:02.921 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.687+0000 7f6305400700 1 -- 172.21.15.44:0/2873223813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 msgr2=0x7f6300105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.687+0000 7f6305400700 1 --2- 172.21.15.44:0/2873223813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300105340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009910 tx=0x7f62f0009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.688+0000 7f6305400700 1 -- 172.21.15.44:0/2873223813 shutdown_connections 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.688+0000 7f6305400700 1 --2- 172.21.15.44:0/2873223813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300105340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.688+0000 7f6305400700 1 -- 172.21.15.44:0/2873223813 >> 172.21.15.44:0/2873223813 conn(0x7f63000fc960 msgr2=0x7f63000feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.688+0000 7f6305400700 1 -- 172.21.15.44:0/2873223813 shutdown_connections 2024-04-20T11:12:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.688+0000 7f6305400700 1 -- 172.21.15.44:0/2873223813 wait complete. 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.689+0000 7f6305400700 1 Processor -- start 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.689+0000 7f6305400700 1 -- start start 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.689+0000 7f6305400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.689+0000 7f6305400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6300108630 con 0x7f6300102f50 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.690+0000 7f62fe800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.690+0000 7f62fe800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53506/0 (socket says 172.21.15.44:53506) 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.690+0000 7f62fe800700 1 -- 172.21.15.44:0/916277521 learned_addr learned my addr 172.21.15.44:0/916277521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.690+0000 7f62fe800700 1 -- 172.21.15.44:0/916277521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019a2d0 con 0x7f6300102f50 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.691+0000 7f62fe800700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300199d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005950 tx=0x7f62f0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.691+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f0027070 con 0x7f6300102f50 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.691+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019a5c0 con 0x7f6300102f50 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.691+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f62f0022470 con 0x7f6300102f50 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.691+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f000f590 con 0x7f6300102f50 2024-04-20T11:12:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.691+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630019d210 con 0x7f6300102f50 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.692+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f62f002a070 con 0x7f6300102f50 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.692+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6300108db0 con 0x7f6300102f50 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.693+0000 7f62f7400700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f62e40396b0 0x7f62e403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.693+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f62f001e030 con 0x7f6300102f50 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.693+0000 7f62fde00700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f62e40396b0 0x7f62e403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.694+0000 7f62fde00700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f62e40396b0 0x7f62e403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f62ec006fd0 tx=0x7f62ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.699+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f62f0013070 con 0x7f6300102f50 2024-04-20T11:12:05.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.956+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f630002e200 con 0x7f6300102f50 2024-04-20T11:12:05.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.959+0000 7f62f7400700 1 -- 172.21.15.44:0/916277521 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f62f000fdb0 con 0x7f6300102f50 2024-04-20T11:12:05.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f62e40396b0 msgr2=0x7f62e403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:05.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f62e40396b0 0x7f62e403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f62ec006fd0 tx=0x7f62ec006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:05.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 msgr2=0x7f6300199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:05.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300199d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005950 tx=0x7f62f0004740 comp rx=0 tx=0).stop 2024-04-20T11:12:05.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 shutdown_connections 2024-04-20T11:12:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f62e40396b0 0x7f62e403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 --2- 172.21.15.44:0/916277521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6300102f50 0x7f6300199d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.963+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 >> 172.21.15.44:0/916277521 conn(0x7f63000fc960 msgr2=0x7f63000fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.964+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 shutdown_connections 2024-04-20T11:12:05.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:05.964+0000 7f6305400700 1 -- 172.21.15.44:0/916277521 wait complete. 2024-04-20T11:12:06.804 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-20T11:12:06.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-20T11:12:07.164 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:06 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/916277521' entity='client.admin' 2024-04-20T11:12:07.896 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:09.252 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:09.253 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:09.253 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:12:09.253 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:09.253 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:09.253 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 -- 172.21.15.44:0/2370743810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 msgr2=0x7f8190103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 --2- 172.21.15.44:0/2370743810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190103fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8180009910 tx=0x7f8180009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 -- 172.21.15.44:0/2370743810 shutdown_connections 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 --2- 172.21.15.44:0/2370743810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190103fe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 -- 172.21.15.44:0/2370743810 >> 172.21.15.44:0/2370743810 conn(0x7f81900fdaa0 msgr2=0x7f81900ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 -- 172.21.15.44:0/2370743810 shutdown_connections 2024-04-20T11:12:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.998+0000 7f8195400700 1 -- 172.21.15.44:0/2370743810 wait complete. 2024-04-20T11:12:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.999+0000 7f8195400700 1 Processor -- start 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.999+0000 7f8195400700 1 -- start start 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.999+0000 7f8195400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:09.999+0000 7f8195400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81901087c0 con 0x7f8190103bc0 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.000+0000 7f818e800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.000+0000 7f818e800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53984/0 (socket says 172.21.15.44:53984) 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.000+0000 7f818e800700 1 -- 172.21.15.44:0/4001403502 learned_addr learned my addr 172.21.15.44:0/4001403502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:10.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.000+0000 7f818e800700 1 -- 172.21.15.44:0/4001403502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8190198280 con 0x7f8190103bc0 2024-04-20T11:12:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.000+0000 7f818e800700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190197d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8180009ed0 tx=0x7f8180004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.001+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8180027070 con 0x7f8190103bc0 2024-04-20T11:12:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.001+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8190198510 con 0x7f8190103bc0 2024-04-20T11:12:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.001+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8180022470 con 0x7f8190103bc0 2024-04-20T11:12:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.001+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f818000f5d0 con 0x7f8190103bc0 2024-04-20T11:12:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.001+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819019b200 con 0x7f8190103bc0 2024-04-20T11:12:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.002+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8190108f40 con 0x7f8190103bc0 2024-04-20T11:12:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.002+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f818002a070 con 0x7f8190103bc0 2024-04-20T11:12:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.003+0000 7f8187400700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f81740396b0 0x7f817403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.003+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f818001e030 con 0x7f8190103bc0 2024-04-20T11:12:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.003+0000 7f818de00700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f81740396b0 0x7f817403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.004+0000 7f818de00700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f81740396b0 0x7f817403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f817c006fd0 tx=0x7f817c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.008+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8180013070 con 0x7f8190103bc0 2024-04-20T11:12:10.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.246+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f8190062e90 con 0x7f81740396b0 2024-04-20T11:12:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.254+0000 7f8187400700 1 -- 172.21.15.44:0/4001403502 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8190062e90 con 0x7f81740396b0 2024-04-20T11:12:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f81740396b0 msgr2=0x7f817403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f81740396b0 0x7f817403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f817c006fd0 tx=0x7f817c006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 msgr2=0x7f8190197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190197d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8180009ed0 tx=0x7f8180004690 comp rx=0 tx=0).stop 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 shutdown_connections 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f81740396b0 0x7f817403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 --2- 172.21.15.44:0/4001403502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8190103bc0 0x7f8190197d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.257+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 >> 172.21.15.44:0/4001403502 conn(0x7f81900fdaa0 msgr2=0x7f81900fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.258+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 shutdown_connections 2024-04-20T11:12:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:10.258+0000 7f8195400700 1 -- 172.21.15.44:0/4001403502 wait complete. 2024-04-20T11:12:11.042 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi078 2024-04-20T11:12:11.042 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:12:11.042 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.conf 2024-04-20T11:12:11.060 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:12:11.061 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:12:11.118 INFO:tasks.cephadm:Adding host smithi078 to orchestrator... 2024-04-20T11:12:11.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch host add smithi078 2024-04-20T11:12:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:11.678 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:11.678 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:11.679 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:11.679 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:11.679 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:11.679 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:11.679 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:11 smithi044 ceph-mon[16855]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:12:12.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:12 smithi044 ceph-mon[16855]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:12:12.572 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:12 smithi044 ceph-mon[16855]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:12:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:13 smithi044 ceph-mon[16855]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:13 smithi044 ceph-mon[16855]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:12:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.668+0000 7f9e00a00700 1 -- 172.21.15.44:0/1513242158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 msgr2=0x7f9dfc10a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.668+0000 7f9e00a00700 1 --2- 172.21.15.44:0/1513242158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc10a970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009910 tx=0x7f9dec009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.669+0000 7f9e00a00700 1 -- 172.21.15.44:0/1513242158 shutdown_connections 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.669+0000 7f9e00a00700 1 --2- 172.21.15.44:0/1513242158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc10a970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.669+0000 7f9e00a00700 1 -- 172.21.15.44:0/1513242158 >> 172.21.15.44:0/1513242158 conn(0x7f9dfc0fffd0 msgr2=0x7f9dfc1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.669+0000 7f9e00a00700 1 -- 172.21.15.44:0/1513242158 shutdown_connections 2024-04-20T11:12:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.669+0000 7f9e00a00700 1 -- 172.21.15.44:0/1513242158 wait complete. 2024-04-20T11:12:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.670+0000 7f9e00a00700 1 Processor -- start 2024-04-20T11:12:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.671+0000 7f9e00a00700 1 -- start start 2024-04-20T11:12:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.671+0000 7f9e00a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.671+0000 7f9e00a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc10ec90 con 0x7f9dfc10a590 2024-04-20T11:12:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.671+0000 7f9dfa800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.671+0000 7f9dfa800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54000/0 (socket says 172.21.15.44:54000) 2024-04-20T11:12:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.671+0000 7f9dfa800700 1 -- 172.21.15.44:0/2702359394 learned_addr learned my addr 172.21.15.44:0/2702359394 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9dfa800700 1 -- 172.21.15.44:0/2702359394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc19a530 con 0x7f9dfc10a590 2024-04-20T11:12:13.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9dfa800700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc199ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dec006010 tx=0x7f9dec004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dec026070 con 0x7f9dfc10a590 2024-04-20T11:12:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9dec0044a0 con 0x7f9dfc10a590 2024-04-20T11:12:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dec003b10 con 0x7f9dfc10a590 2024-04-20T11:12:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dfc19a7c0 con 0x7f9dfc10a590 2024-04-20T11:12:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.672+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dfc19e870 con 0x7f9dfc10a590 2024-04-20T11:12:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.674+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dfc105c00 con 0x7f9dfc10a590 2024-04-20T11:12:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.674+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9dec029070 con 0x7f9dfc10a590 2024-04-20T11:12:13.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.674+0000 7f9df3400700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9de00396b0 0x7f9de003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:13.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.674+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9dec01e070 con 0x7f9dfc10a590 2024-04-20T11:12:13.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.675+0000 7f9df9e00700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9de00396b0 0x7f9de003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:13.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.675+0000 7f9df9e00700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9de00396b0 0x7f9de003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9de8006fd0 tx=0x7f9de8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.680+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9dec013070 con 0x7f9dfc10a590 2024-04-20T11:12:13.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:13.916+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}) -- 0x7f9dfc19b300 con 0x7f9de00396b0 2024-04-20T11:12:15.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:15 smithi044 ceph-mon[16855]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:15.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:15 smithi044 ceph-mon[16855]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:16.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:16 smithi044 ceph-mon[16855]: Deploying cephadm binary to smithi078 2024-04-20T11:12:17.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:17 smithi044 ceph-mon[16855]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.953+0000 7f9df3400700 1 -- 172.21.15.44:0/2702359394 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9dfc19b300 con 0x7f9de00396b0 2024-04-20T11:12:18.953 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi078' with addr '172.21.15.78' 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.956+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9de00396b0 msgr2=0x7f9de003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.956+0000 7f9e00a00700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9de00396b0 0x7f9de003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9de8006fd0 tx=0x7f9de8006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.956+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 msgr2=0x7f9dfc199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.956+0000 7f9e00a00700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc199ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dec006010 tx=0x7f9dec004cc0 comp rx=0 tx=0).stop 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.957+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 shutdown_connections 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.957+0000 7f9e00a00700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9de00396b0 0x7f9de003bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.957+0000 7f9e00a00700 1 --2- 172.21.15.44:0/2702359394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dfc10a590 0x7f9dfc199ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.957+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 >> 172.21.15.44:0/2702359394 conn(0x7f9dfc0fffd0 msgr2=0x7f9dfc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.957+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 shutdown_connections 2024-04-20T11:12:18.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:18.957+0000 7f9e00a00700 1 -- 172.21.15.44:0/2702359394 wait complete. 2024-04-20T11:12:19.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:19 smithi044 ceph-mon[16855]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:19.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:19 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:19.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:19 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:19.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T11:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:20.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:20 smithi044 ceph-mon[16855]: Added host smithi078 2024-04-20T11:12:20.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:20.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:21 smithi044 ceph-mon[16855]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:22.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 -- 172.21.15.44:0/3901335680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 msgr2=0x7fd770108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:22.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 --2- 172.21.15.44:0/3901335680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770108730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd764009910 tx=0x7fd764009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:22.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 -- 172.21.15.44:0/3901335680 shutdown_connections 2024-04-20T11:12:22.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 --2- 172.21.15.44:0/3901335680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770108730 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:22.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 -- 172.21.15.44:0/3901335680 >> 172.21.15.44:0/3901335680 conn(0x7fd770076d00 msgr2=0x7fd770077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:22.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 -- 172.21.15.44:0/3901335680 shutdown_connections 2024-04-20T11:12:22.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.429+0000 7fd776e00700 1 -- 172.21.15.44:0/3901335680 wait complete. 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.431+0000 7fd776e00700 1 Processor -- start 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.431+0000 7fd776e00700 1 -- start start 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.431+0000 7fd776e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.431+0000 7fd776e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010ca10 con 0x7fd770108350 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.432+0000 7fd775a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.432+0000 7fd775a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51774/0 (socket says 172.21.15.44:51774) 2024-04-20T11:12:22.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.432+0000 7fd775a00700 1 -- 172.21.15.44:0/2503365725 learned_addr learned my addr 172.21.15.44:0/2503365725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:22.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.432+0000 7fd775a00700 1 -- 172.21.15.44:0/2503365725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77019a510 con 0x7fd770108350 2024-04-20T11:12:22.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.432+0000 7fd775a00700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770199fd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7640052a0 tx=0x7fd764004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:22.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.433+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd764026070 con 0x7fd770108350 2024-04-20T11:12:22.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.433+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd76400bbf0 con 0x7fd770108350 2024-04-20T11:12:22.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.433+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77019a800 con 0x7fd770108350 2024-04-20T11:12:22.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.433+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd764003a30 con 0x7fd770108350 2024-04-20T11:12:22.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.433+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77019e8a0 con 0x7fd770108350 2024-04-20T11:12:22.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.434+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd770103a00 con 0x7fd770108350 2024-04-20T11:12:22.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.434+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd764029070 con 0x7fd770108350 2024-04-20T11:12:22.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.435+0000 7fd76e000700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd7580396b0 0x7fd75803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:22.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.435+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd76401e070 con 0x7fd770108350 2024-04-20T11:12:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.435+0000 7fd775000700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd7580396b0 0x7fd75803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.436+0000 7fd775000700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd7580396b0 0x7fd75803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd75c006fd0 tx=0x7fd75c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:22.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.440+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd764013070 con 0x7fd770108350 2024-04-20T11:12:22.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.679+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd77019b330 con 0x7fd7580396b0 2024-04-20T11:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.681+0000 7fd76e000700 1 -- 172.21.15.44:0/2503365725 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd77019b330 con 0x7fd7580396b0 2024-04-20T11:12:22.681 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:22.681 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}] 2024-04-20T11:12:22.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd7580396b0 msgr2=0x7fd75803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd7580396b0 0x7fd75803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd75c006fd0 tx=0x7fd75c006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 msgr2=0x7fd770199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770199fd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7640052a0 tx=0x7fd764004cc0 comp rx=0 tx=0).stop 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 shutdown_connections 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd7580396b0 0x7fd75803bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 --2- 172.21.15.44:0/2503365725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd770108350 0x7fd770199fd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 >> 172.21.15.44:0/2503365725 conn(0x7fd770076d00 msgr2=0x7fd770104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:22.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 shutdown_connections 2024-04-20T11:12:22.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:22.684+0000 7fd776e00700 1 -- 172.21.15.44:0/2503365725 wait complete. 2024-04-20T11:12:22.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:22 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:22.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:22 smithi044 ceph-mon[16855]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:22.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:22 smithi044 ceph-mon[16855]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:12:23.592 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-04-20T11:12:23.592 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:12:23.593 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-04-20T11:12:23.611 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:12:23.611 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:12:23.668 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-04-20T11:12:23.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch host add smithi165 2024-04-20T11:12:24.099 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:25.300 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:26.235 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:25 smithi044 ceph-mon[16855]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:12:26.235 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:25 smithi044 ceph-mon[16855]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:12:26.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:25 smithi044 ceph-mon[16855]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.279+0000 7f4544600700 1 -- 172.21.15.44:0/1985840157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 msgr2=0x7f453c077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.279+0000 7f4544600700 1 --2- 172.21.15.44:0/1985840157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c077360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f453000b0a0 tx=0x7f453000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.280+0000 7f4544600700 1 -- 172.21.15.44:0/1985840157 shutdown_connections 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.280+0000 7f4544600700 1 --2- 172.21.15.44:0/1985840157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c077360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.280+0000 7f4544600700 1 -- 172.21.15.44:0/1985840157 >> 172.21.15.44:0/1985840157 conn(0x7f453c100160 msgr2=0x7f453c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.280+0000 7f4544600700 1 -- 172.21.15.44:0/1985840157 shutdown_connections 2024-04-20T11:12:26.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.280+0000 7f4544600700 1 -- 172.21.15.44:0/1985840157 wait complete. 2024-04-20T11:12:26.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.281+0000 7f4544600700 1 Processor -- start 2024-04-20T11:12:26.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.281+0000 7f4544600700 1 -- start start 2024-04-20T11:12:26.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.281+0000 7f4544600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:26.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.281+0000 7f4544600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f453c113240 con 0x7f453c076f80 2024-04-20T11:12:26.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.282+0000 7f4541c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:26.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.282+0000 7f4541c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51780/0 (socket says 172.21.15.44:51780) 2024-04-20T11:12:26.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.282+0000 7f4541c00700 1 -- 172.21.15.44:0/3693816961 learned_addr learned my addr 172.21.15.44:0/3693816961 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:26.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.282+0000 7f4541c00700 1 -- 172.21.15.44:0/3693816961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c19eb80 con 0x7f453c076f80 2024-04-20T11:12:26.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.282+0000 7f4541c00700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c19e640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4530000f80 tx=0x7f453000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:26.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.283+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4530017040 con 0x7f453c076f80 2024-04-20T11:12:26.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.283+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4530022070 con 0x7f453c076f80 2024-04-20T11:12:26.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.283+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c19ee10 con 0x7f453c076f80 2024-04-20T11:12:26.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.283+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4530004a00 con 0x7f453c076f80 2024-04-20T11:12:26.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.283+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c19f270 con 0x7f453c076f80 2024-04-20T11:12:26.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.284+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4530003a00 con 0x7f453c076f80 2024-04-20T11:12:26.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.284+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c0786c0 con 0x7f453c076f80 2024-04-20T11:12:26.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.285+0000 7f453a000700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4524039840 0x7f452403bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:26.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.285+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f453000f050 con 0x7f453c076f80 2024-04-20T11:12:26.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.285+0000 7f4541200700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4524039840 0x7f452403bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:26.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.286+0000 7f4541200700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4524039840 0x7f452403bd00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4528006fd0 tx=0x7f4528006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:26.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.291+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f453001c5e0 con 0x7f453c076f80 2024-04-20T11:12:26.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:26.527+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}) -- 0x7f453c19f9a0 con 0x7f4524039840 2024-04-20T11:12:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:27 smithi044 ceph-mon[16855]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:12:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:27 smithi044 ceph-mon[16855]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:27 smithi044 ceph-mon[16855]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:28.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:28 smithi044 ceph-mon[16855]: Deploying cephadm binary to smithi165 2024-04-20T11:12:29.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:29 smithi044 ceph-mon[16855]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:31.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:31 smithi044 ceph-mon[16855]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:31.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.547+0000 7f453a000700 1 -- 172.21.15.44:0/3693816961 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f453c19f9a0 con 0x7f4524039840 2024-04-20T11:12:31.547 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-04-20T11:12:31.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4524039840 msgr2=0x7f452403bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:31.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4524039840 0x7f452403bd00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4528006fd0 tx=0x7f4528006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:31.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 msgr2=0x7f453c19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:31.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c19e640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4530000f80 tx=0x7f453000bb10 comp rx=0 tx=0).stop 2024-04-20T11:12:31.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 shutdown_connections 2024-04-20T11:12:31.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4524039840 0x7f452403bd00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:31.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 --2- 172.21.15.44:0/3693816961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c076f80 0x7f453c19e640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:31.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 >> 172.21.15.44:0/3693816961 conn(0x7f453c100160 msgr2=0x7f453c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:31.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 shutdown_connections 2024-04-20T11:12:31.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:31.551+0000 7f4544600700 1 -- 172.21.15.44:0/3693816961 wait complete. 2024-04-20T11:12:32.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T11:12:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:32.907 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:32.908 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:32 smithi044 ceph-mon[16855]: Added host smithi165 2024-04-20T11:12:32.908 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:32.908 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:33.885 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:33 smithi044 ceph-mon[16855]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:33.885 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 -- 172.21.15.44:0/2679368467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 msgr2=0x7f714c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 --2- 172.21.15.44:0/2679368467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c10dd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f713c009910 tx=0x7f713c009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 -- 172.21.15.44:0/2679368467 shutdown_connections 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 --2- 172.21.15.44:0/2679368467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c10dd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 -- 172.21.15.44:0/2679368467 >> 172.21.15.44:0/2679368467 conn(0x7f714c0781f0 msgr2=0x7f714c07a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 -- 172.21.15.44:0/2679368467 shutdown_connections 2024-04-20T11:12:34.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.765+0000 7f7152000700 1 -- 172.21.15.44:0/2679368467 wait complete. 2024-04-20T11:12:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.766+0000 7f7152000700 1 Processor -- start 2024-04-20T11:12:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.767+0000 7f7152000700 1 -- start start 2024-04-20T11:12:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.767+0000 7f7152000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.767+0000 7f7152000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f714c113200 con 0x7f714c10d970 2024-04-20T11:12:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.767+0000 7f714b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.767+0000 7f714b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44468/0 (socket says 172.21.15.44:44468) 2024-04-20T11:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.767+0000 7f714b400700 1 -- 172.21.15.44:0/2096514141 learned_addr learned my addr 172.21.15.44:0/2096514141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.768+0000 7f714b400700 1 -- 172.21.15.44:0/2096514141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c19eae0 con 0x7f714c10d970 2024-04-20T11:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.768+0000 7f714b400700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c19e5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f713c006010 tx=0x7f713c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.769+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f713c026070 con 0x7f714c10d970 2024-04-20T11:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.769+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f713c0044a0 con 0x7f714c10d970 2024-04-20T11:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.769+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c19edd0 con 0x7f714c10d970 2024-04-20T11:12:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.769+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f713c003af0 con 0x7f714c10d970 2024-04-20T11:12:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.769+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1a2e20 con 0x7f714c10d970 2024-04-20T11:12:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.770+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c10a190 con 0x7f714c10d970 2024-04-20T11:12:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.770+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f713c029070 con 0x7f714c10d970 2024-04-20T11:12:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.771+0000 7f7143e00700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7130039650 0x7f713003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.771+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f713c01e070 con 0x7f714c10d970 2024-04-20T11:12:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.771+0000 7f714aa00700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7130039650 0x7f713003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.772+0000 7f714aa00700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7130039650 0x7f713003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7138006fd0 tx=0x7f7138006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:34.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:34.777+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f713c013070 con 0x7f714c10d970 2024-04-20T11:12:35.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.014+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f714c19f8b0 con 0x7f7130039650 2024-04-20T11:12:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.016+0000 7f7143e00700 1 -- 172.21.15.44:0/2096514141 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f714c19f8b0 con 0x7f7130039650 2024-04-20T11:12:35.015 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:12:35.016 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.018+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7130039650 msgr2=0x7f713003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7130039650 0x7f713003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7138006fd0 tx=0x7f7138006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 msgr2=0x7f714c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c19e5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f713c006010 tx=0x7f713c004cc0 comp rx=0 tx=0).stop 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 shutdown_connections 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7130039650 0x7f713003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 --2- 172.21.15.44:0/2096514141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f714c10d970 0x7f714c19e5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 >> 172.21.15.44:0/2096514141 conn(0x7f714c0781f0 msgr2=0x7f714c110450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 shutdown_connections 2024-04-20T11:12:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:35.019+0000 7f7152000700 1 -- 172.21.15.44:0/2096514141 wait complete. 2024-04-20T11:12:35.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:35 smithi044 ceph-mon[16855]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:35.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:35 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:35.803 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-20T11:12:35.803 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd crush tunables default 2024-04-20T11:12:36.253 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:12:36.590 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:36 smithi044 ceph-mon[16855]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:12:37.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:37 smithi044 ceph-mon[16855]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:38.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 -- 172.21.15.44:0/762081721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 msgr2=0x7f0750101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:38.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 --2- 172.21.15.44:0/762081721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750101810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0740009910 tx=0x7f0740009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:38.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 -- 172.21.15.44:0/762081721 shutdown_connections 2024-04-20T11:12:38.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 --2- 172.21.15.44:0/762081721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750101810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:38.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 -- 172.21.15.44:0/762081721 >> 172.21.15.44:0/762081721 conn(0x7f07500fcce0 msgr2=0x7f07500ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:38.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 -- 172.21.15.44:0/762081721 shutdown_connections 2024-04-20T11:12:38.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.460+0000 7f074f200700 1 -- 172.21.15.44:0/762081721 wait complete. 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074f200700 1 Processor -- start 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074f200700 1 -- start start 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074f200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074f200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f075010b940 con 0x7f0750101430 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074de00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074de00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50762/0 (socket says 172.21.15.44:50762) 2024-04-20T11:12:38.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.462+0000 7f074de00700 1 -- 172.21.15.44:0/3916880844 learned_addr learned my addr 172.21.15.44:0/3916880844 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:12:38.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.463+0000 7f074de00700 1 -- 172.21.15.44:0/3916880844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0750198310 con 0x7f0750101430 2024-04-20T11:12:38.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.463+0000 7f074de00700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750197dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0740006010 tx=0x7f0740004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:38.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.463+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0740026070 con 0x7f0750101430 2024-04-20T11:12:38.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.463+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07400044a0 con 0x7f0750101430 2024-04-20T11:12:38.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.464+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0750198600 con 0x7f0750101430 2024-04-20T11:12:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.464+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0740003a70 con 0x7f0750101430 2024-04-20T11:12:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.464+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075019c6a0 con 0x7f0750101430 2024-04-20T11:12:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.465+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0740029070 con 0x7f0750101430 2024-04-20T11:12:38.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.465+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0750102910 con 0x7f0750101430 2024-04-20T11:12:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.465+0000 7f0746a00700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0734039650 0x7f073403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.465+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f074001e070 con 0x7f0750101430 2024-04-20T11:12:38.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.466+0000 7f074d400700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0734039650 0x7f073403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:38.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.466+0000 7f074d400700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0734039650 0x7f073403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f073c006fd0 tx=0x7f073c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.471+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0740013070 con 0x7f0750101430 2024-04-20T11:12:38.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:38 smithi044 ceph-mon[16855]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:12:38.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:38.713+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0750199130 con 0x7f0750101430 2024-04-20T11:12:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.613+0000 7f0746a00700 1 -- 172.21.15.44:0/3916880844 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f0740003ce0 con 0x7f0750101430 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.617+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0734039650 msgr2=0x7f073403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.617+0000 7f074f200700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0734039650 0x7f073403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f073c006fd0 tx=0x7f073c006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 msgr2=0x7f0750197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750197dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0740006010 tx=0x7f0740004cc0 comp rx=0 tx=0).stop 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 shutdown_connections 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0734039650 0x7f073403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 --2- 172.21.15.44:0/3916880844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0750101430 0x7f0750197dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:39.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 >> 172.21.15.44:0/3916880844 conn(0x7f07500fcce0 msgr2=0x7f075006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:39.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 shutdown_connections 2024-04-20T11:12:39.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:12:39.618+0000 7f074f200700 1 -- 172.21.15.44:0/3916880844 wait complete. 2024-04-20T11:12:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:39 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3916880844' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T11:12:40.416 INFO:tasks.cephadm:Adding mon.a on smithi044 2024-04-20T11:12:40.416 INFO:tasks.cephadm:Adding mon.b on smithi078 2024-04-20T11:12:40.416 INFO:tasks.cephadm:Adding mon.c on smithi165 2024-04-20T11:12:40.416 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch apply mon '3;smithi044:172.21.15.44=a;smithi078:172.21.15.78=b;smithi165:172.21.15.165=c' 2024-04-20T11:12:40.654 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:12:40.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3916880844' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T11:12:40.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:40 smithi044 ceph-mon[16855]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:12:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:41 smithi044 ceph-mon[16855]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:42.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 -- 172.21.15.165:0/3319439186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 msgr2=0x7f51b41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 --2- 172.21.15.165:0/3319439186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b41019a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009910 tx=0x7f51a4009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:42.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 -- 172.21.15.165:0/3319439186 shutdown_connections 2024-04-20T11:12:42.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 --2- 172.21.15.165:0/3319439186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b41019a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 -- 172.21.15.165:0/3319439186 >> 172.21.15.165:0/3319439186 conn(0x7f51b40fce70 msgr2=0x7f51b40ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:42.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 -- 172.21.15.165:0/3319439186 shutdown_connections 2024-04-20T11:12:42.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.145+0000 7f51ba6d0700 1 -- 172.21.15.165:0/3319439186 wait complete. 2024-04-20T11:12:42.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.146+0000 7f51ba6d0700 1 Processor -- start 2024-04-20T11:12:42.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.146+0000 7f51ba6d0700 1 -- start start 2024-04-20T11:12:42.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.146+0000 7f51ba6d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:42.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.146+0000 7f51ba6d0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51b410bad0 con 0x7f51b41015c0 2024-04-20T11:12:42.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.147+0000 7f51b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:42.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.147+0000 7f51b3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b4198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.165:48492/0 (socket says 172.21.15.165:48492) 2024-04-20T11:12:42.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.147+0000 7f51b3fff700 1 -- 172.21.15.165:0/2385246373 learned_addr learned my addr 172.21.15.165:0/2385246373 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:12:42.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.148+0000 7f51b3fff700 1 -- 172.21.15.165:0/2385246373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b41993a0 con 0x7f51b41015c0 2024-04-20T11:12:42.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.148+0000 7f51b3fff700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b4198e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51a40052a0 tx=0x7f51a4004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:42.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.149+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a4026070 con 0x7f51b41015c0 2024-04-20T11:12:42.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.149+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b4199690 con 0x7f51b41015c0 2024-04-20T11:12:42.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.149+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b4199b50 con 0x7f51b41015c0 2024-04-20T11:12:42.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.149+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51a400bbf0 con 0x7f51b41015c0 2024-04-20T11:12:42.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.149+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a4030060 con 0x7f51b41015c0 2024-04-20T11:12:42.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.149+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a0005330 con 0x7f51b41015c0 2024-04-20T11:12:42.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.150+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f51a4029070 con 0x7f51b41015c0 2024-04-20T11:12:42.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.150+0000 7f51b17fa700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f519c039650 0x7f519c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:42.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.150+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51a401e070 con 0x7f51b41015c0 2024-04-20T11:12:42.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.151+0000 7f51b37fe700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f519c039650 0x7f519c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:42.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.152+0000 7f51b37fe700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f519c039650 0x7f519c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006fd0 tx=0x7f51a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:42.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.165+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f51a4013070 con 0x7f51b41015c0 2024-04-20T11:12:42.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.377+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi078:172.21.15.78=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}) -- 0x7f51a0000c80 con 0x7f519c039650 2024-04-20T11:12:42.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.392+0000 7f51b17fa700 1 -- 172.21.15.165:0/2385246373 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f51a0000c80 con 0x7f519c039650 2024-04-20T11:12:42.392 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mon update... 2024-04-20T11:12:42.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.394+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f519c039650 msgr2=0x7f519c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f519c039650 0x7f519c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006fd0 tx=0x7f51a8006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:42.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 msgr2=0x7f51b4198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:42.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b4198e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51a40052a0 tx=0x7f51a4004cc0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 shutdown_connections 2024-04-20T11:12:42.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f519c039650 0x7f519c03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 --2- 172.21.15.165:0/2385246373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b41015c0 0x7f51b4198e60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:42.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 >> 172.21.15.165:0/2385246373 conn(0x7f51b40fce70 msgr2=0x7f51b4104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:42.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 shutdown_connections 2024-04-20T11:12:42.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:42.395+0000 7f51ba6d0700 1 -- 172.21.15.165:0/2385246373 wait complete. 2024-04-20T11:12:43.023 DEBUG:teuthology.orchestra.run.smithi078:mon.b> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.b.service 2024-04-20T11:12:43.025 DEBUG:teuthology.orchestra.run.smithi165:mon.c> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.c.service 2024-04-20T11:12:43.028 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T11:12:43.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T11:12:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi078:172.21.15.78=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:12:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi078:172.21.15.78=b;smithi165:172.21.15.165=c;count:3 2024-04-20T11:12:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:12:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:12:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:12:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:43 smithi044 ceph-mon[16855]: Deploying daemon mon.c on smithi165 2024-04-20T11:12:43.779 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:12:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:45 smithi044 ceph-mon[16855]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 -- 172.21.15.165:0/4210839319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 msgr2=0x7facac1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 --2- 172.21.15.165:0/4210839319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac1019a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faca8009910 tx=0x7faca8009c20 comp rx=0 tx=0).stop 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 -- 172.21.15.165:0/4210839319 shutdown_connections 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 --2- 172.21.15.165:0/4210839319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac1019a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 -- 172.21.15.165:0/4210839319 >> 172.21.15.165:0/4210839319 conn(0x7facac0fce70 msgr2=0x7facac0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 -- 172.21.15.165:0/4210839319 shutdown_connections 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.770+0000 7facb327a700 1 -- 172.21.15.165:0/4210839319 wait complete. 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.771+0000 7facb327a700 1 Processor -- start 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.771+0000 7facb327a700 1 -- start start 2024-04-20T11:12:46.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.771+0000 7facb327a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:46.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.771+0000 7facb327a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7facac10bad0 con 0x7facac1015c0 2024-04-20T11:12:46.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.772+0000 7facb1016700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:46.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.772+0000 7facb1016700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.165:48508/0 (socket says 172.21.15.165:48508) 2024-04-20T11:12:46.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.772+0000 7facb1016700 1 -- 172.21.15.165:0/166845175 learned_addr learned my addr 172.21.15.165:0/166845175 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.772+0000 7facb1016700 1 -- 172.21.15.165:0/166845175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac1984c0 con 0x7facac1015c0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.773+0000 7facb1016700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac197f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faca80052a0 tx=0x7faca8004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.773+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faca8026070 con 0x7facac1015c0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.773+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac1987b0 con 0x7facac1015c0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.773+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac19c850 con 0x7facac1015c0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.773+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faca8004830 con 0x7facac1015c0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.773+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faca8030060 con 0x7facac1015c0 2024-04-20T11:12:46.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.774+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac90005330 con 0x7facac1015c0 2024-04-20T11:12:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.774+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7faca8029070 con 0x7facac1015c0 2024-04-20T11:12:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.775+0000 7faca27fc700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fac98039650 0x7fac9803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.775+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faca801e070 con 0x7facac1015c0 2024-04-20T11:12:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.775+0000 7facb0815700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fac98039650 0x7fac9803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:46.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.776+0000 7facb0815700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fac98039650 0x7fac9803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac9c006fd0 tx=0x7fac9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:46.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:46.790+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faca8013070 con 0x7facac1015c0 2024-04-20T11:12:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.087+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fac900058f0 con 0x7facac1015c0 2024-04-20T11:12:47.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.089+0000 7faca27fc700 1 -- 172.21.15.165:0/166845175 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7faca8017440 con 0x7facac1015c0 2024-04-20T11:12:47.088 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:12:47.088 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":1,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","modified":"2024-04-20T11:10:35.587741Z","created":"2024-04-20T11:10:35.587741Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T11:12:47.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fac98039650 msgr2=0x7fac9803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fac98039650 0x7fac9803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac9c006fd0 tx=0x7fac9c006e40 comp rx=0 tx=0).stop 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 msgr2=0x7facac197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac197f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faca80052a0 tx=0x7faca8004cc0 comp rx=0 tx=0).stop 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 shutdown_connections 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fac98039650 0x7fac9803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 --2- 172.21.15.165:0/166845175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facac1015c0 0x7facac197f80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 >> 172.21.15.165:0/166845175 conn(0x7facac0fce70 msgr2=0x7facac06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 shutdown_connections 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:47.091+0000 7facb327a700 1 -- 172.21.15.165:0/166845175 wait complete. 2024-04-20T11:12:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 1 2024-04-20T11:12:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:47 smithi165 systemd[1]: Started Ceph mon.c for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:12:48.921 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T11:12:48.922 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T11:12:49.168 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:12:52.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: mon.a calling monitor election 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: mon.c calling monitor election 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: monmap epoch 2 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: last_changed 2024-04-20T11:12:47.283993+0000 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: min_mon_release 19 (squid) 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: election_strategy: 1 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: fsmap 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: mgrmap e13: x(active, since 62s) 2024-04-20T11:12:52.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: overall HEALTH_OK 2024-04-20T11:12:52.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:52.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:52.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:12:52.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: mon.a calling monitor election 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: mon.c calling monitor election 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: monmap epoch 2 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:12:52.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: last_changed 2024-04-20T11:12:47.283993+0000 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: min_mon_release 19 (squid) 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: election_strategy: 1 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: fsmap 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: mgrmap e13: x(active, since 62s) 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: overall HEALTH_OK 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:52.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 -- 172.21.15.165:0/2090453088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd740035c0 msgr2=0x7fbd74005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 --2- 172.21.15.165:0/2090453088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd740035c0 0x7fbd74005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd88078b40 tx=0x7fbd7800feb0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 -- 172.21.15.165:0/2090453088 shutdown_connections 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 --2- 172.21.15.165:0/2090453088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd740035c0 0x7fbd74005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 -- 172.21.15.165:0/2090453088 >> 172.21.15.165:0/2090453088 conn(0x7fbd88077930 msgr2=0x7fbd88077d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 -- 172.21.15.165:0/2090453088 shutdown_connections 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.555+0000 7fbd8f998700 1 -- 172.21.15.165:0/2090453088 wait complete. 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.556+0000 7fbd8f998700 1 Processor -- start 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.556+0000 7fbd8f998700 1 -- start start 2024-04-20T11:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.556+0000 7fbd8f998700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:53.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.556+0000 7fbd8f998700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd881986d0 0x7fbd8819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:53.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.556+0000 7fbd8f998700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8810a730 con 0x7fbd881986d0 2024-04-20T11:12:53.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.556+0000 7fbd8f998700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8810a5b0 con 0x7fbd740035c0 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57006/0 (socket says 172.21.15.165:57006) 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 -- 172.21.15.165:0/1723580885 learned_addr learned my addr 172.21.15.165:0/1723580885 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 -- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 msgr2=0x7fbd88198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 -- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 msgr2=0x7fbd88198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8d734700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:12:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.557+0000 7fbd8cf33700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd881986d0 0x7fbd8819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:53.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.558+0000 7fbd8cf33700 1 -- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 msgr2=0x7fbd88198190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:12:53.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.558+0000 7fbd8cf33700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.558+0000 7fbd8cf33700 1 -- 172.21.15.165:0/1723580885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd88198d10 con 0x7fbd881986d0 2024-04-20T11:12:53.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.558+0000 7fbd8cf33700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd881986d0 0x7fbd8819ca60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400b4d0 tx=0x7fbd8400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:53.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.559+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbd84011630 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.559+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8819cfa0 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.559+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd84011cd0 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.559+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbd8400f2e0 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.559+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8819d500 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.560+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8806a970 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.561+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbd84018020 con 0x7fbd881986d0 2024-04-20T11:12:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.561+0000 7fbd7e7fc700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbd6c0393f0 0x7fbd6c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:12:53.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.561+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd8404c1b0 con 0x7fbd881986d0 2024-04-20T11:12:53.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.561+0000 7fbd8d734700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbd6c0393f0 0x7fbd6c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:12:53.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.562+0000 7fbd8d734700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbd6c0393f0 0x7fbd6c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800fa60 tx=0x7fbd78005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:12:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.579+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbd8401f020 con 0x7fbd881986d0 2024-04-20T11:12:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:53 smithi044 ceph-mon[16855]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:53 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:53 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:12:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:53 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:53 smithi044 ceph-mon[16855]: Deploying daemon mon.b on smithi078 2024-04-20T11:12:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:53 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:53 smithi165 ceph-mon[21975]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:53 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:12:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:53 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:12:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:53 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:12:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:53 smithi165 ceph-mon[21975]: Deploying daemon mon.b on smithi078 2024-04-20T11:12:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:53 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:12:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.863+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbd8819d810 con 0x7fbd881986d0 2024-04-20T11:12:53.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.864+0000 7fbd7e7fc700 1 -- 172.21.15.165:0/1723580885 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fbd84017520 con 0x7fbd881986d0 2024-04-20T11:12:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:12:53.865 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","modified":"2024-04-20T11:12:47.283993Z","created":"2024-04-20T11:10:35.587741Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T11:12:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbd6c0393f0 msgr2=0x7fbd6c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbd6c0393f0 0x7fbd6c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800fa60 tx=0x7fbd78005a90 comp rx=0 tx=0).stop 2024-04-20T11:12:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd881986d0 msgr2=0x7fbd8819ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd881986d0 0x7fbd8819ca60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400b4d0 tx=0x7fbd8400b890 comp rx=0 tx=0).stop 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 shutdown_connections 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbd6c0393f0 0x7fbd6c03b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd740035c0 0x7fbd88198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 --2- 172.21.15.165:0/1723580885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd881986d0 0x7fbd8819ca60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 >> 172.21.15.165:0/1723580885 conn(0x7fbd88077930 msgr2=0x7fbd880ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 shutdown_connections 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:12:53.867+0000 7fbd8f998700 1 -- 172.21.15.165:0/1723580885 wait complete. 2024-04-20T11:12:53.868 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-04-20T11:12:54.409 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:54 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/1723580885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T11:12:54.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:54 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/1723580885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T11:12:55.411 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T11:12:55.411 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T11:12:55.654 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:12:55.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:12:55 smithi044 ceph-mon[16855]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:12:55 smithi165 ceph-mon[21975]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:12:56.321 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:12:56 smithi078 ceph-mon[21464]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-20T11:12:56.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:12:56 smithi078 bash[21445]: b8f80487171f94dd18fa186d04c68465a06f3806f99cb15e75e20a5275b01d39 2024-04-20T11:12:56.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:12:56 smithi078 systemd[1]: Started Ceph mon.b for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:13:01.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.467+0000 7f79c1ffd700 1 -- 172.21.15.165:0/277654688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79a80057d0 msgr2=0x7f79a8007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:01.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.467+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/277654688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79a80057d0 0x7f79a8007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79bc101370 tx=0x7f79ac00fe60 comp rx=0 tx=0).stop 2024-04-20T11:13:01.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.467+0000 7f79c1ffd700 1 -- 172.21.15.165:0/277654688 shutdown_connections 2024-04-20T11:13:01.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.467+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/277654688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79a80057d0 0x7f79a8007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.467+0000 7f79c1ffd700 1 -- 172.21.15.165:0/277654688 >> 172.21.15.165:0/277654688 conn(0x7f79bc0fcf30 msgr2=0x7f79bc0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:01.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.468+0000 7f79c1ffd700 1 -- 172.21.15.165:0/277654688 shutdown_connections 2024-04-20T11:13:01.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.468+0000 7f79c1ffd700 1 -- 172.21.15.165:0/277654688 wait complete. 2024-04-20T11:13:01.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.468+0000 7f79c1ffd700 1 Processor -- start 2024-04-20T11:13:01.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.468+0000 7f79c1ffd700 1 -- start start 2024-04-20T11:13:01.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79c1ffd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:01.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79c1ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79bc1965d0 0x7f79bc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79c1ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 0x7f79bc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79c1ffd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1084f0 con 0x7f79bc1965d0 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79c1ffd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc108370 con 0x7f79bc19aea0 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79c1ffd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc108670 con 0x7f79a80057d0 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79bb7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79bb7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48348/0 (socket says 172.21.15.165:48348) 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79bb7fe700 1 -- 172.21.15.165:0/444904533 learned_addr learned my addr 172.21.15.165:0/444904533 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.469+0000 7f79bbfff700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 0x7f79bc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bb7fe700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 msgr2=0x7f79bc196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bb7fe700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 msgr2=0x7f79bc196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bb7fe700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79bc1965d0 0x7f79bc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:01.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bb7fe700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bbfff700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 msgr2=0x7f79bc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bbfff700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 msgr2=0x7f79bc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bbfff700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 0x7f79bc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79bbfff700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 0x7f79bc19b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 msgr2=0x7f79bc196090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 msgr2=0x7f79bc19b350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 0x7f79bc19b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 -- 172.21.15.165:0/444904533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc1a2c10 con 0x7f79bc1965d0 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.470+0000 7f79b2ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79bc1965d0 0x7f79bc19a960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f79a400b550 tx=0x7f79a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.471+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4004230 con 0x7f79bc1965d0 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.471+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc1a2e40 con 0x7f79bc1965d0 2024-04-20T11:13:01.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.471+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f79a40043d0 con 0x7f79bc1965d0 2024-04-20T11:13:01.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.471+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4011c70 con 0x7f79bc1965d0 2024-04-20T11:13:01.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.471+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc1a3370 con 0x7f79bc1965d0 2024-04-20T11:13:01.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.472+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79bc0640b0 con 0x7f79bc1965d0 2024-04-20T11:13:01.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.472+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f79a40036a0 con 0x7f79bc1965d0 2024-04-20T11:13:01.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.472+0000 7f79b97fa700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f799c039ae0 0x7f799c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:01.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.473+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79a404c320 con 0x7f79bc1965d0 2024-04-20T11:13:01.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.473+0000 7f79bb7fe700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f799c039ae0 0x7f799c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:01.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.474+0000 7f79bb7fe700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f799c039ae0 0x7f799c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00fa10 tx=0x7f79ac005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:01.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.480+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f79a4003de0 con 0x7f79bc1965d0 2024-04-20T11:13:01.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: mon.c calling monitor election 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: mon.a calling monitor election 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T11:13:01.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: monmap epoch 3 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: last_changed 2024-04-20T11:12:56.310188+0000 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: min_mon_release 19 (squid) 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: election_strategy: 1 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: fsmap 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: mgrmap e13: x(active, since 71s) 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: overall HEALTH_OK 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:01.700 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: mon.c calling monitor election 2024-04-20T11:13:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: mon.a calling monitor election 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: monmap epoch 3 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: last_changed 2024-04-20T11:12:56.310188+0000 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: min_mon_release 19 (squid) 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: election_strategy: 1 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: fsmap 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: mgrmap e13: x(active, since 71s) 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: overall HEALTH_OK 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:01.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:01.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.776+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f79bc02e150 con 0x7f79bc1965d0 2024-04-20T11:13:01.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.776+0000 7f79b97fa700 1 -- 172.21.15.165:0/444904533 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f79a4021e00 con 0x7f79bc1965d0 2024-04-20T11:13:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T11:13:01.777 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","modified":"2024-04-20T11:12:56.310188Z","created":"2024-04-20T11:10:35.587741Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f799c039ae0 msgr2=0x7f799c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f799c039ae0 0x7f799c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00fa10 tx=0x7f79ac005ad0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79bc1965d0 msgr2=0x7f79bc19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79bc1965d0 0x7f79bc19a960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f79a400b550 tx=0x7f79a400b910 comp rx=0 tx=0).stop 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 shutdown_connections 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f799c039ae0 0x7f799c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79bc19aea0 0x7f79bc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79a80057d0 0x7f79bc196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 --2- 172.21.15.165:0/444904533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79bc1965d0 0x7f79bc19a960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 >> 172.21.15.165:0/444904533 conn(0x7f79bc0fcf30 msgr2=0x7f79bc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:01.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 shutdown_connections 2024-04-20T11:13:01.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:01.779+0000 7f79c1ffd700 1 -- 172.21.15.165:0/444904533 wait complete. 2024-04-20T11:13:01.781 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 3 2024-04-20T11:13:02.412 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-20T11:13:02.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph config generate-minimal-conf 2024-04-20T11:13:02.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:02.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:02.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:02 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/444904533' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T11:13:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:02 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/444904533' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T11:13:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:02.847 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: mon.c calling monitor election 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: mon.a calling monitor election 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: monmap epoch 3 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: last_changed 2024-04-20T11:12:56.310188+0000 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: min_mon_release 19 (squid) 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: election_strategy: 1 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-20T11:13:03.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: fsmap 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: mgrmap e13: x(active, since 71s) 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: overall HEALTH_OK 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/444904533' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T11:13:03.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: mon.b calling monitor election 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: mon.b calling monitor election 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: mon.c calling monitor election 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: mon.a calling monitor election 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T11:13:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: monmap epoch 3 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: last_changed 2024-04-20T11:12:56.310188+0000 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: min_mon_release 19 (squid) 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: election_strategy: 1 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: fsmap 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: mgrmap e13: x(active, since 73s) 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: overall HEALTH_OK 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:04.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:04.482 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.482 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.482 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.482 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: mon.b calling monitor election 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: mon.b calling monitor election 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: mon.c calling monitor election 2024-04-20T11:13:04.526 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: mon.a calling monitor election 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: monmap epoch 3 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: last_changed 2024-04-20T11:12:56.310188+0000 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: min_mon_release 19 (squid) 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: election_strategy: 1 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: fsmap 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: mgrmap e13: x(active, since 73s) 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: overall HEALTH_OK 2024-04-20T11:13:04.527 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:04.528 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:04.528 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.528 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.528 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.528 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: mon.b calling monitor election 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: mon.b calling monitor election 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: mon.c calling monitor election 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: mon.a calling monitor election 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: monmap epoch 3 2024-04-20T11:13:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: last_changed 2024-04-20T11:12:56.310188+0000 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: created 2024-04-20T11:10:35.587741+0000 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: min_mon_release 19 (squid) 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: election_strategy: 1 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: 2: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: fsmap 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: mgrmap e13: x(active, since 73s) 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: overall HEALTH_OK 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:04.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.083+0000 7fad4be00700 1 -- 172.21.15.44:0/3133374611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c102500 msgr2=0x7fad4c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.083+0000 7fad4be00700 1 --2- 172.21.15.44:0/3133374611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c102500 0x7fad4c1028e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009910 tx=0x7fad3c009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.084+0000 7fad4be00700 1 -- 172.21.15.44:0/3133374611 shutdown_connections 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.084+0000 7fad4be00700 1 --2- 172.21.15.44:0/3133374611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c102500 0x7fad4c1028e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.084+0000 7fad4be00700 1 -- 172.21.15.44:0/3133374611 >> 172.21.15.44:0/3133374611 conn(0x7fad4c0fddb0 msgr2=0x7fad4c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.084+0000 7fad4be00700 1 -- 172.21.15.44:0/3133374611 shutdown_connections 2024-04-20T11:13:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.084+0000 7fad4be00700 1 -- 172.21.15.44:0/3133374611 wait complete. 2024-04-20T11:13:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.085+0000 7fad4be00700 1 Processor -- start 2024-04-20T11:13:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.085+0000 7fad4be00700 1 -- start start 2024-04-20T11:13:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.085+0000 7fad4be00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 0x7fad4c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.085+0000 7fad4be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad4c19a860 0x7fad4c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.085+0000 7fad4be00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c19f130 0x7fad4c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4be00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c9d0 con 0x7fad4c19f130 2024-04-20T11:13:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4be00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c850 con 0x7fad4c102500 2024-04-20T11:13:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4be00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10cb50 con 0x7fad4c19a860 2024-04-20T11:13:05.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 0x7fad4c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 0x7fad4c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55520/0 (socket says 172.21.15.44:55520) 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 -- 172.21.15.44:0/409596781 learned_addr learned my addr 172.21.15.44:0/409596781 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad43e00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad4c19a860 0x7fad4c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4b400700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c19f130 0x7fad4c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 -- 172.21.15.44:0/409596781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad4c19a860 msgr2=0x7fad4c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad4c19a860 0x7fad4c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 -- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c19f130 msgr2=0x7fad4c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c19f130 0x7fad4c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.086+0000 7fad4aa00700 1 -- 172.21.15.44:0/409596781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c1a6ea0 con 0x7fad4c102500 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.087+0000 7fad4aa00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 0x7fad4c19a320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006010 tx=0x7fad3c0056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.087+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c026070 con 0x7fad4c102500 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.087+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad3c00bcd0 con 0x7fad4c102500 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.087+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c1a70d0 con 0x7fad4c102500 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.087+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00fa90 con 0x7fad4c102500 2024-04-20T11:13:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.088+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c1a7660 con 0x7fad4c102500 2024-04-20T11:13:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.088+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fad3c029070 con 0x7fad4c102500 2024-04-20T11:13:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.089+0000 7fad43400700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad30039ae0 0x7fad3003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.089+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad3c003cf0 con 0x7fad4c102500 2024-04-20T11:13:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.089+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c0640b0 con 0x7fad4c102500 2024-04-20T11:13:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.089+0000 7fad43e00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad30039ae0 0x7fad3003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.090+0000 7fad43e00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad30039ae0 0x7fad3003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad34009ad0 tx=0x7fad34009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:05.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.095+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fad3c030070 con 0x7fad4c102500 2024-04-20T11:13:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.329+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fad4c1a78c0 con 0x7fad4c102500 2024-04-20T11:13:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.330+0000 7fad43400700 1 -- 172.21.15.44:0/409596781 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fad3c013070 con 0x7fad4c102500 2024-04-20T11:13:05.331 INFO:teuthology.orchestra.run.smithi044.stdout:# minimal ceph.conf for 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:05.331 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-04-20T11:13:05.331 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:13:05.331 INFO:teuthology.orchestra.run.smithi044.stdout: mon_host = [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-04-20T11:13:05.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad30039ae0 msgr2=0x7fad3003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:05.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad30039ae0 0x7fad3003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad34009ad0 tx=0x7fad34009210 comp rx=0 tx=0).stop 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 msgr2=0x7fad4c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 0x7fad4c19a320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006010 tx=0x7fad3c0056f0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 shutdown_connections 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad30039ae0 0x7fad3003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4c19f130 0x7fad4c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c102500 0x7fad4c19a320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 --2- 172.21.15.44:0/409596781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad4c19a860 0x7fad4c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 >> 172.21.15.44:0/409596781 conn(0x7fad4c0fddb0 msgr2=0x7fad4c074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 shutdown_connections 2024-04-20T11:13:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:05.334+0000 7fad4be00700 1 -- 172.21.15.44:0/409596781 wait complete. 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.337 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:05.337 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:05.337 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:05.337 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:05.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:05.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:05.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:05.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:05.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:06.123 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-20T11:13:06.123 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:13:06.124 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T11:13:06.218 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:13:06.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:13:06.334 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:06 smithi044 ceph-mon[16855]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T11:13:06.335 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:06 smithi044 ceph-mon[16855]: Reconfiguring daemon mon.a on smithi044 2024-04-20T11:13:06.335 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:06 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/409596781' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:06.349 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:13:06.350 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T11:13:06.381 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:13:06.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:13:06.450 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:13:06.450 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T11:13:06.476 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:06 smithi165 ceph-mon[21975]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T11:13:06.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:06 smithi165 ceph-mon[21975]: Reconfiguring daemon mon.a on smithi044 2024-04-20T11:13:06.477 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:06 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/409596781' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:06.482 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:13:06.482 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:13:06.550 INFO:tasks.cephadm:Adding mgr.x on smithi044 2024-04-20T11:13:06.550 INFO:tasks.cephadm:Adding mgr.y on smithi078 2024-04-20T11:13:06.550 INFO:tasks.cephadm:Adding mgr.z on smithi165 2024-04-20T11:13:06.550 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch apply mgr '3;smithi044=x;smithi078=y;smithi165=z' 2024-04-20T11:13:06.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:06 smithi078 ceph-mon[21464]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T11:13:06.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:06 smithi078 ceph-mon[21464]: Reconfiguring daemon mon.a on smithi044 2024-04-20T11:13:06.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:06 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/409596781' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:06.826 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:13:07.361 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:07 smithi044 ceph-mon[16855]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:07.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:07 smithi165 ceph-mon[21975]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:07.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:07 smithi078 ceph-mon[21464]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.301+0000 7f69c77dc700 1 -- 172.21.15.165:0/2617450323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0103140 msgr2=0x7f69c0105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.301+0000 7f69c77dc700 1 --2- 172.21.15.165:0/2617450323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0103140 0x7f69c0105530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009910 tx=0x7f69bc009c20 comp rx=0 tx=0).stop 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.301+0000 7f69c77dc700 1 -- 172.21.15.165:0/2617450323 shutdown_connections 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.301+0000 7f69c77dc700 1 --2- 172.21.15.165:0/2617450323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0103140 0x7f69c0105530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.301+0000 7f69c77dc700 1 -- 172.21.15.165:0/2617450323 >> 172.21.15.165:0/2617450323 conn(0x7f69c00fca90 msgr2=0x7f69c00feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 -- 172.21.15.165:0/2617450323 shutdown_connections 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 -- 172.21.15.165:0/2617450323 wait complete. 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 Processor -- start 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 -- start start 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 0x7f69c0102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69c01010f0 0x7f69c0101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0101ab0 0x7f69c01a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0108770 con 0x7f69c0101ab0 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01085f0 con 0x7f69c01010f0 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c77dc700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01088f0 con 0x7f69c0103140 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c5578700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 0x7f69c0102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c5578700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 0x7f69c0102aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33870/0 (socket says 172.21.15.165:33870) 2024-04-20T11:13:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.302+0000 7f69c5578700 1 -- 172.21.15.165:0/131929123 learned_addr learned my addr 172.21.15.165:0/131929123 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:13:08.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c4d77700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69c01010f0 0x7f69c0101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c5578700 1 -- 172.21.15.165:0/131929123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69c01010f0 msgr2=0x7f69c0101570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c5578700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69c01010f0 0x7f69c0101570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c5578700 1 -- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0101ab0 msgr2=0x7f69c01a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c5578700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0101ab0 0x7f69c01a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c5578700 1 -- 172.21.15.165:0/131929123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01a2500 con 0x7f69c0103140 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c5578700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 0x7f69c0102aa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0057f0 tx=0x7f69bc00ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc026070 con 0x7f69c0103140 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01a2790 con 0x7f69c0103140 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f69bc003b80 con 0x7f69c0103140 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00fd40 con 0x7f69c0103140 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.303+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c01a2d40 con 0x7f69c0103140 2024-04-20T11:13:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.304+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f69bc029070 con 0x7f69c0103140 2024-04-20T11:13:08.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.304+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c00640b0 con 0x7f69c0103140 2024-04-20T11:13:08.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.304+0000 7f69b67fc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69ac039ae0 0x7f69ac03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:08.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.304+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f69bc04d680 con 0x7f69c0103140 2024-04-20T11:13:08.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.305+0000 7f69c4d77700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69ac039ae0 0x7f69ac03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:08.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.306+0000 7f69c4d77700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69ac039ae0 0x7f69ac03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009850 tx=0x7f69b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:08.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.310+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f69bc013070 con 0x7f69c0103140 2024-04-20T11:13:08.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.521+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi078=y;smithi165=z", "target": ["mon-mgr", ""]}) -- 0x7f69c0062f20 con 0x7f69ac039ae0 2024-04-20T11:13:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.533+0000 7f69b67fc700 1 -- 172.21.15.165:0/131929123 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f69c0062f20 con 0x7f69ac039ae0 2024-04-20T11:13:08.534 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mgr update... 2024-04-20T11:13:08.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:08.534 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:08.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: Reconfiguring mon.b (monmap changed)... 2024-04-20T11:13:08.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:08.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:08.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:08.535 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:08 smithi165 ceph-mon[21975]: Reconfiguring daemon mon.b on smithi078 2024-04-20T11:13:08.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69ac039ae0 msgr2=0x7f69ac03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69ac039ae0 0x7f69ac03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009850 tx=0x7f69b0008040 comp rx=0 tx=0).stop 2024-04-20T11:13:08.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 msgr2=0x7f69c0102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:08.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 0x7f69c0102aa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0057f0 tx=0x7f69bc00ba70 comp rx=0 tx=0).stop 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 shutdown_connections 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69ac039ae0 0x7f69ac03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c0101ab0 0x7f69c01a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69c0103140 0x7f69c0102aa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 --2- 172.21.15.165:0/131929123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69c01010f0 0x7f69c0101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 >> 172.21.15.165:0/131929123 conn(0x7f69c00fca90 msgr2=0x7f69c00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 shutdown_connections 2024-04-20T11:13:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:13:08.536+0000 7f69c77dc700 1 -- 172.21.15.165:0/131929123 wait complete. 2024-04-20T11:13:08.800 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:08.800 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:08.801 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: Reconfiguring mon.b (monmap changed)... 2024-04-20T11:13:08.801 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:08.801 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:08.801 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:08.801 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:08 smithi078 ceph-mon[21464]: Reconfiguring daemon mon.b on smithi078 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: Reconfiguring mon.b (monmap changed)... 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:08 smithi044 ceph-mon[16855]: Reconfiguring daemon mon.b on smithi078 2024-04-20T11:13:09.103 DEBUG:teuthology.orchestra.run.smithi078:mgr.y> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.y.service 2024-04-20T11:13:09.105 DEBUG:teuthology.orchestra.run.smithi165:mgr.z> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.z.service 2024-04-20T11:13:09.107 INFO:tasks.cephadm:Deploying OSDs... 2024-04-20T11:13:09.107 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:13:09.107 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T11:13:09.166 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T11:13:09.166 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 11:12:06.706563638 +0000 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 11:04:53.965025492 +0000 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 11:04:53.965025492 +0000 2024-04-20T11:13:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-20 11:04:53.965025492 +0000 2024-04-20T11:13:09.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T11:13:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T11:13:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T11:13:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000332409 s, 1.5 MB/s 2024-04-20T11:13:09.304 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T11:13:09.360 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-20T11:13:09.414 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T11:13:09.414 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:09.414 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-20T11:13:09.414 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:09.414 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:09.415 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 11:12:06.775562584 +0000 2024-04-20T11:13:09.415 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 11:04:54.608015949 +0000 2024-04-20T11:13:09.415 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 11:04:54.608015949 +0000 2024-04-20T11:13:09.415 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-20 11:04:54.608015949 +0000 2024-04-20T11:13:09.415 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T11:13:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T11:13:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T11:13:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000391522 s, 1.3 MB/s 2024-04-20T11:13:09.503 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T11:13:09.562 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-20T11:13:09.616 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T11:13:09.616 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:09.616 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-20T11:13:09.616 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:09.616 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:09.616 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 11:12:06.847561483 +0000 2024-04-20T11:13:09.617 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 11:04:55.226006776 +0000 2024-04-20T11:13:09.617 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 11:04:55.226006776 +0000 2024-04-20T11:13:09.617 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-20 11:04:55.226006776 +0000 2024-04-20T11:13:09.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T11:13:09.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:09.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi078=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:09.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: Saving service mgr spec with placement smithi044=x;smithi078=y;smithi165=z;count:3 2024-04-20T11:13:09.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:09.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:09.691 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T11:13:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T11:13:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000390157 s, 1.3 MB/s 2024-04-20T11:13:09.703 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T11:13:09.760 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 11:12:06.912560490 +0000 2024-04-20T11:13:09.814 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 11:04:55.862997322 +0000 2024-04-20T11:13:09.815 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 11:04:55.862997322 +0000 2024-04-20T11:13:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-20 11:04:55.862997322 +0000 2024-04-20T11:13:09.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi078=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: Saving service mgr spec with placement smithi044=x;smithi078=y;smithi165=z;count:3 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:09.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T11:13:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T11:13:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000375279 s, 1.4 MB/s 2024-04-20T11:13:09.900 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T11:13:09.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:09.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi044=x;smithi078=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: Saving service mgr spec with placement smithi044=x;smithi078=y;smithi165=z;count:3 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T11:13:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:09.957 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-20T11:13:09.957 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T11:13:09.975 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T11:13:09.976 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-04-20T11:13:10.033 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T11:13:10.033 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:10.033 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-20T11:13:10.033 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:10.033 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:10.034 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-20 11:12:55.763673198 +0000 2024-04-20T11:13:10.034 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-20 11:04:46.390986948 +0000 2024-04-20T11:13:10.034 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-20 11:04:46.390986948 +0000 2024-04-20T11:13:10.034 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-20 11:04:46.390986948 +0000 2024-04-20T11:13:10.034 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T11:13:10.100 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-20T11:13:10.100 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-20T11:13:10.100 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000255366 s, 2.0 MB/s 2024-04-20T11:13:10.101 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T11:13:10.160 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-20 11:12:55.764673184 +0000 2024-04-20T11:13:10.218 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-20 11:04:46.737981909 +0000 2024-04-20T11:13:10.219 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-20 11:04:46.737981909 +0000 2024-04-20T11:13:10.219 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-20 11:04:46.737981909 +0000 2024-04-20T11:13:10.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T11:13:10.285 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-20T11:13:10.285 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-20T11:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.00019879 s, 2.6 MB/s 2024-04-20T11:13:10.287 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T11:13:10.345 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-04-20T11:13:10.403 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T11:13:10.403 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:10.403 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-20T11:13:10.403 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:10.403 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:10.404 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-20 11:12:55.764673184 +0000 2024-04-20T11:13:10.404 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-20 11:04:47.069977088 +0000 2024-04-20T11:13:10.404 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-20 11:04:47.069977088 +0000 2024-04-20T11:13:10.404 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-20 11:04:47.069977088 +0000 2024-04-20T11:13:10.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T11:13:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-20T11:13:10.470 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-20T11:13:10.470 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000257797 s, 2.0 MB/s 2024-04-20T11:13:10.471 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T11:13:10.530 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-04-20T11:13:10.588 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-20 11:12:55.765673169 +0000 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-20 11:04:47.399972296 +0000 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-20 11:04:47.399972296 +0000 2024-04-20T11:13:10.589 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-20 11:04:47.399972296 +0000 2024-04-20T11:13:10.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T11:13:10.656 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-20T11:13:10.656 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-20T11:13:10.656 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000231225 s, 2.2 MB/s 2024-04-20T11:13:10.658 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T11:13:10.716 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T11:13:10.716 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T11:13:10.730 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T11:13:10.731 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-04-20T11:13:10.786 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T11:13:10.786 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-20T11:13:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 11:12:46.264088829 +0000 2024-04-20T11:13:10.787 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 11:04:46.396027938 +0000 2024-04-20T11:13:10.787 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 11:04:46.396027938 +0000 2024-04-20T11:13:10.787 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-20 11:04:46.396027938 +0000 2024-04-20T11:13:10.787 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T11:13:10.846 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:10 smithi165 ceph-mon[21975]: Reconfiguring mon.c (monmap changed)... 2024-04-20T11:13:10.846 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:10 smithi165 ceph-mon[21975]: Reconfiguring daemon mon.c on smithi165 2024-04-20T11:13:10.849 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T11:13:10.849 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T11:13:10.849 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000192293 s, 2.7 MB/s 2024-04-20T11:13:10.850 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T11:13:10.906 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-04-20T11:13:10.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:10 smithi078 ceph-mon[21464]: Reconfiguring mon.c (monmap changed)... 2024-04-20T11:13:10.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:10 smithi078 ceph-mon[21464]: Reconfiguring daemon mon.c on smithi165 2024-04-20T11:13:10.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:10 smithi044 ceph-mon[16855]: Reconfiguring mon.c (monmap changed)... 2024-04-20T11:13:10.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:10 smithi044 ceph-mon[16855]: Reconfiguring daemon mon.c on smithi165 2024-04-20T11:13:10.961 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T11:13:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:11.026 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-20T11:13:11.026 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:11.026 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:11.027 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 11:12:46.265088814 +0000 2024-04-20T11:13:11.027 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 11:04:46.738022675 +0000 2024-04-20T11:13:11.027 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 11:04:46.738022675 +0000 2024-04-20T11:13:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-20 11:04:46.738022675 +0000 2024-04-20T11:13:11.027 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T11:13:11.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T11:13:11.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T11:13:11.050 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000201167 s, 2.5 MB/s 2024-04-20T11:13:11.051 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T11:13:11.107 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-04-20T11:13:11.163 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T11:13:11.163 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:11.163 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-20T11:13:11.163 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:11.164 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:11.164 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 11:12:46.265088814 +0000 2024-04-20T11:13:11.164 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 11:04:47.091017243 +0000 2024-04-20T11:13:11.164 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 11:04:47.091017243 +0000 2024-04-20T11:13:11.164 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-20 11:04:47.091017243 +0000 2024-04-20T11:13:11.164 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T11:13:11.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T11:13:11.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T11:13:11.227 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000219284 s, 2.3 MB/s 2024-04-20T11:13:11.228 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T11:13:11.283 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 11:12:46.266088800 +0000 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 11:04:47.419012196 +0000 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 11:04:47.419012196 +0000 2024-04-20T11:13:11.338 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-20 11:04:47.419012196 +0000 2024-04-20T11:13:11.339 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T11:13:11.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T11:13:11.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T11:13:11.400 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000208391 s, 2.5 MB/s 2024-04-20T11:13:11.401 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T11:13:11.457 INFO:tasks.cephadm:Deploying osd.0 on smithi044 with /dev/vg_nvme/lv_4... 2024-04-20T11:13:11.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T11:13:11.914 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:13:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:11 smithi078 ceph-mon[21464]: Deploying daemon mgr.z on smithi165 2024-04-20T11:13:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:11 smithi044 ceph-mon[16855]: Deploying daemon mgr.z on smithi165 2024-04-20T11:13:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:11.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:11 smithi165 ceph-mon[21975]: Deploying daemon mgr.z on smithi165 2024-04-20T11:13:12.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:12 smithi165 systemd[1]: Starting Ceph mgr.z for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:13:13.114 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:12 smithi165 podman[23819]: 2024-04-20 11:13:12.836348363 +0000 UTC m=+0.104734842 container create caa509b16bb3a51174d99a3ea16b6055fd4dd96dacee58349c965ed115f5fba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:13:13.114 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:12 smithi165 podman[23819]: 2024-04-20 11:13:12.749458777 +0000 UTC m=+0.017845270 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:13:13.114 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 podman[23819]: 2024-04-20 11:13:13.031142859 +0000 UTC m=+0.299529338 container init caa509b16bb3a51174d99a3ea16b6055fd4dd96dacee58349c965ed115f5fba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-20T11:13:13.114 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 podman[23819]: 2024-04-20 11:13:13.033855721 +0000 UTC m=+0.302242201 container start caa509b16bb3a51174d99a3ea16b6055fd4dd96dacee58349c965ed115f5fba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T11:13:13.114 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mgr[23838]: -- 172.21.15.165:0/1549788588 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b75fc805a0 con 0x55b75fc54800 2024-04-20T11:13:13.114 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 bash[23819]: caa509b16bb3a51174d99a3ea16b6055fd4dd96dacee58349c965ed115f5fba9 2024-04-20T11:13:13.371 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 systemd[1]: Started Ceph mgr.z for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:13:13.371 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:13.185+0000 7fba998ac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:13:13.371 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:13.370+0000 7fba998ac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:13.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:13.471+0000 7fba998ac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:13:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:13.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T11:13:13.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:13.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:13.901 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:14.229 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:13.779+0000 7fba998ac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:13:14.229 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:13.865+0000 7fba998ac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:13:14.715 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:14 smithi165 ceph-mon[21975]: Deploying daemon mgr.y on smithi078 2024-04-20T11:13:14.890 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:14 smithi078 ceph-mon[21464]: Deploying daemon mgr.y on smithi078 2024-04-20T11:13:14.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:14 smithi044 ceph-mon[16855]: Deploying daemon mgr.y on smithi078 2024-04-20T11:13:15.100 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T11:13:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T11:13:15.101 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-20T11:13:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-20T11:13:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0395077 s, 265 MB/s 2024-04-20T11:13:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-20T11:13:15.454 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:15.109+0000 7fba998ac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:13:15.454 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:15.280+0000 7fba998ac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:13:15.454 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:15.453+0000 7fba998ac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:13:15.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:15 smithi165 ceph-mon[21975]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:15.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:15 smithi044 ceph-mon[16855]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:15.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-04-20T11:13:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:15 smithi078 ceph-mon[21464]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:15.937 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:15.786+0000 7ffbe031c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:13:16.046 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:15.746+0000 7fba998ac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:13:16.046 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:15.839+0000 7fba998ac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:13:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:13:16.397 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:15.970+0000 7ffbe031c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:13:16.398 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:16.075+0000 7ffbe031c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:13:16.398 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:16.397+0000 7ffbe031c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:13:16.479 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:16.045+0000 7fba998ac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:13:16.479 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:16.246+0000 7fba998ac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:13:16.687 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:16.485+0000 7ffbe031c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:13:17.035 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.035 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.035 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.035 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.035 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-mon[21464]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:17.035 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:17.045 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:16.882+0000 7fba998ac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:13:17.045 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.045 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.045 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-mon[21975]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:17.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:16 smithi044 ceph-mon[16855]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:17.647 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:17.340+0000 7fba998ac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:13:17.648 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:17.445+0000 7fba998ac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:13:17.648 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:17.647+0000 7fba998ac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:13:18.074 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:17.739+0000 7ffbe031c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:13:18.074 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:17.905+0000 7ffbe031c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:13:18.074 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:18.074+0000 7ffbe031c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:13:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 -- 172.21.15.44:0/317848270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 msgr2=0x7f23ec10ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 --2- 172.21.15.44:0/317848270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 0x7f23ec10ae30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009fc0 tx=0x7f23e400b330 comp rx=0 tx=0).stop 2024-04-20T11:13:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 -- 172.21.15.44:0/317848270 shutdown_connections 2024-04-20T11:13:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 --2- 172.21.15.44:0/317848270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 0x7f23ec10ae30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:18.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 --2- 172.21.15.44:0/317848270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:18.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 --2- 172.21.15.44:0/317848270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23ec103be0 0x7f23ec104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:18.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.272+0000 7f23f2200700 1 -- 172.21.15.44:0/317848270 >> 172.21.15.44:0/317848270 conn(0x7f23ec0fdf80 msgr2=0x7f23ec1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:18.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.273+0000 7f23f2200700 1 -- 172.21.15.44:0/317848270 shutdown_connections 2024-04-20T11:13:18.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.273+0000 7f23f2200700 1 -- 172.21.15.44:0/317848270 wait complete. 2024-04-20T11:13:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.274+0000 7f23f2200700 1 Processor -- start 2024-04-20T11:13:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.274+0000 7f23f2200700 1 -- start start 2024-04-20T11:13:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f2200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f2200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23ec103be0 0x7f23ec074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f2200700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 0x7f23ec075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f2200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec10e740 con 0x7f23ec103be0 2024-04-20T11:13:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f2200700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec10e5c0 con 0x7f23ec104580 2024-04-20T11:13:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f2200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec10e8c0 con 0x7f23ec1029e0 2024-04-20T11:13:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f0e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f0e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec0767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35342/0 (socket says 172.21.15.44:35342) 2024-04-20T11:13:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23ebe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23ec103be0 0x7f23ec074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f0e00700 1 -- 172.21.15.44:0/1714042385 learned_addr learned my addr 172.21.15.44:0/1714042385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.275+0000 7f23f1800700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 0x7f23ec075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f0e00700 1 -- 172.21.15.44:0/1714042385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 msgr2=0x7f23ec075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f0e00700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 0x7f23ec075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f0e00700 1 -- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23ec103be0 msgr2=0x7f23ec074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f0e00700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23ec103be0 0x7f23ec074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f0e00700 1 -- 172.21.15.44:0/1714042385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec075c30 con 0x7f23ec1029e0 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f0e00700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec0767f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23e000e970 tx=0x7f23e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0004220 con 0x7f23ec1029e0 2024-04-20T11:13:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23e00043c0 con 0x7f23ec1029e0 2024-04-20T11:13:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0003e60 con 0x7f23ec1029e0 2024-04-20T11:13:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.276+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec1a2d70 con 0x7f23ec1029e0 2024-04-20T11:13:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.277+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1a3170 con 0x7f23ec1029e0 2024-04-20T11:13:18.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.278+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f23e000c7c0 con 0x7f23ec1029e0 2024-04-20T11:13:18.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.278+0000 7f23e9600700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f23d4039890 0x7f23d403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:18.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.278+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23e004ccb0 con 0x7f23ec1029e0 2024-04-20T11:13:18.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.278+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec069740 con 0x7f23ec1029e0 2024-04-20T11:13:18.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.278+0000 7f23ebe00700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f23d4039890 0x7f23d403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:18.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.279+0000 7f23ebe00700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f23d4039890 0x7f23d403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005fd0 tx=0x7f23d8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:18.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.288+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f23e0016330 con 0x7f23ec1029e0 2024-04-20T11:13:18.425 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:18.167+0000 7fba998ac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:13:18.426 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:18.257+0000 7fba998ac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:13:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:18.587+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f23ec062f50 con 0x7f23d4039890 2024-04-20T11:13:18.668 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:18.376+0000 7ffbe031c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:13:18.668 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:18.468+0000 7ffbe031c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:13:18.668 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:18.668+0000 7ffbe031c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:18.669 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:18.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:18.725 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.725 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:18.502+0000 7fba998ac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:18.632+0000 7fba998ac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:13:18.726 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:18.725+0000 7fba998ac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:13:18.937 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:18.861+0000 7ffbe031c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:13:18.979 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:18.838+0000 7fba998ac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:13:19.479 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:19.137+0000 7fba998ac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:13:19.850 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T11:13:19.850 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: Reconfiguring daemon mgr.x on smithi044 2024-04-20T11:13:19.850 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:19.851 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:19.851 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:13:19.851 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:13:19.851 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:19 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:19.866 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:19.451+0000 7ffbe031c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: Reconfiguring daemon mgr.x on smithi044 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:13:19.867 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: Reconfiguring daemon mgr.x on smithi044 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:19.946 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:19.733+0000 7fba998ac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:13:19.947 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:13:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:13:19.945+0000 7fba998ac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:13:20.161 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:19.866+0000 7ffbe031c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:13:20.161 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:19.961+0000 7ffbe031c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:13:20.161 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:20.161+0000 7ffbe031c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:13:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:20.508+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f23e0018cd0 con 0x7f23ec1029e0 2024-04-20T11:13:20.770 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:20.680+0000 7ffbe031c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:20.770+0000 7ffbe031c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: Standby manager daemon z started 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.771 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:20.772 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:20.772 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:20.772 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: Standby manager daemon z started 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:20.824 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:20.825 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: Standby manager daemon z started 2024-04-20T11:13:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T11:13:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/2320760570' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:20.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:21.138 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:21.008+0000 7ffbe031c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:13:21.138 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:21.138+0000 7ffbe031c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:13:21.437 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:21.232+0000 7ffbe031c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:13:21.437 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:21.346+0000 7ffbe031c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:13:21.704 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:21 smithi044 ceph-mon[16855]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:21.705 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:21 smithi044 ceph-mon[16855]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-20T11:13:21.705 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:21 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T11:13:21.937 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:21.642+0000 7ffbe031c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:13:21.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-mon[21464]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:21.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-mon[21464]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-20T11:13:21.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:21 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T11:13:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:21 smithi165 ceph-mon[21975]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:21 smithi165 ceph-mon[21975]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-20T11:13:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:21 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T11:13:22.511 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:22.233+0000 7ffbe031c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:13:22.511 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:13:22.446+0000 7ffbe031c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:13:22.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:22.525+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mon.1 v2:172.21.15.165:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f23e000cab0 con 0x7f23ec1029e0 2024-04-20T11:13:22.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-mon[21464]: Standby manager daemon y started 2024-04-20T11:13:22.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T11:13:22.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:22.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T11:13:22.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:22 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:22.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:22 smithi044 ceph-mon[16855]: Standby manager daemon y started 2024-04-20T11:13:22.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:22 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T11:13:22.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:22 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:22.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:22 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T11:13:22.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:22 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:22 smithi165 ceph-mon[21975]: Standby manager daemon y started 2024-04-20T11:13:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:22 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T11:13:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:22 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:13:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:22 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T11:13:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:22 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/3489524964' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/321255393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d4d49b-1218-474f-8e22-6d528d61f28d"}]: dispatch 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/321255393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d4d49b-1218-474f-8e22-6d528d61f28d"}]': finished 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T11:13:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-20T11:13:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T11:13:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/321255393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d4d49b-1218-474f-8e22-6d528d61f28d"}]: dispatch 2024-04-20T11:13:23.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/321255393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d4d49b-1218-474f-8e22-6d528d61f28d"}]': finished 2024-04-20T11:13:23.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T11:13:23.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-20T11:13:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T11:13:23.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/321255393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d4d49b-1218-474f-8e22-6d528d61f28d"}]: dispatch 2024-04-20T11:13:23.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/321255393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d4d49b-1218-474f-8e22-6d528d61f28d"}]': finished 2024-04-20T11:13:23.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T11:13:23.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:24.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:24 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1679888364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:13:24.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:24 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1679888364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:13:24.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:24 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1679888364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:13:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:25 smithi078 ceph-mon[21464]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:25.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:25 smithi044 ceph-mon[16855]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:25 smithi165 ceph-mon[21975]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:27 smithi078 ceph-mon[21464]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:27 smithi044 ceph-mon[16855]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:27 smithi165 ceph-mon[21975]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:29 smithi078 ceph-mon[21464]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:29 smithi044 ceph-mon[16855]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:29 smithi165 ceph-mon[21975]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:31.894 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:31 smithi044 ceph-mon[16855]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:31.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:31 smithi078 ceph-mon[21464]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:31 smithi165 ceph-mon[21975]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:32.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:13:32.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:32.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:13:32.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:32.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T11:13:32.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:33.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:33 smithi044 ceph-mon[16855]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:33.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:33 smithi044 ceph-mon[16855]: Deploying daemon osd.0 on smithi044 2024-04-20T11:13:33.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:33 smithi078 ceph-mon[21464]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:33.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:33 smithi078 ceph-mon[21464]: Deploying daemon osd.0 on smithi044 2024-04-20T11:13:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:33 smithi165 ceph-mon[21975]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:33 smithi165 ceph-mon[21975]: Deploying daemon osd.0 on smithi044 2024-04-20T11:13:35.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:35 smithi044 ceph-mon[16855]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:35.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:35 smithi078 ceph-mon[21464]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:35.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:35 smithi165 ceph-mon[21975]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:37.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:37 smithi044 ceph-mon[16855]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:37 smithi078 ceph-mon[21464]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:37 smithi165 ceph-mon[21975]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:39.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:39 smithi078 ceph-mon[21464]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:39 smithi044 ceph-mon[16855]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:39 smithi165 ceph-mon[21975]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:41.834 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:41 smithi044 ceph-mon[16855]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:41.835 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:41.835 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:41.835 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:41 smithi078 ceph-mon[21464]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:41 smithi165 ceph-mon[21975]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:41.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:43.801 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:43 smithi044 ceph-mon[16855]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:43.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:43 smithi078 ceph-mon[21464]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:43 smithi165 ceph-mon[21975]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:13:45.166 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:45 smithi044 ceph-mon[16855]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:13:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:45 smithi078 ceph-mon[21464]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:13:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:13:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:45 smithi165 ceph-mon[21975]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T11:13:45.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.982+0000 7f23e9600700 1 -- 172.21.15.44:0/1714042385 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f23ec062f50 con 0x7f23d4039890 2024-04-20T11:13:45.984 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 0 on host 'smithi044' 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f23d4039890 msgr2=0x7f23d403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f23d4039890 0x7f23d403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005fd0 tx=0x7f23d8009310 comp rx=0 tx=0).stop 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 msgr2=0x7f23ec0767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec0767f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23e000e970 tx=0x7f23e000ec80 comp rx=0 tx=0).stop 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 shutdown_connections 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f23d4039890 0x7f23d403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ec104580 0x7f23ec075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23ec1029e0 0x7f23ec0767f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.986+0000 7f23f2200700 1 --2- 172.21.15.44:0/1714042385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23ec103be0 0x7f23ec074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.987+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 >> 172.21.15.44:0/1714042385 conn(0x7f23ec0fdf80 msgr2=0x7f23ec106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.987+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 shutdown_connections 2024-04-20T11:13:45.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:13:45.987+0000 7f23f2200700 1 -- 172.21.15.44:0/1714042385 wait complete. 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:46.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:13:46.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:47.011 DEBUG:teuthology.orchestra.run.smithi044:osd.0> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.0.service 2024-04-20T11:13:47.016 INFO:tasks.cephadm:Deploying osd.1 on smithi078 with /dev/vg_nvme/lv_4... 2024-04-20T11:13:47.016 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T11:13:47.289 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:13:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:47 smithi078 ceph-mon[21464]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:47 smithi078 ceph-mon[21464]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:13:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:47 smithi078 ceph-mon[21464]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T11:13:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:47 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:47 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:47 smithi044 ceph-mon[16855]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:47 smithi044 ceph-mon[16855]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:13:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:47 smithi044 ceph-mon[16855]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T11:13:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:47 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:47 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:47 smithi165 ceph-mon[21975]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:47 smithi165 ceph-mon[21975]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:13:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:47 smithi165 ceph-mon[21975]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T11:13:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:47 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:47 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:48 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:13:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:48 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:13:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:48 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:48 smithi078 ceph-mon[21464]: osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] boot 2024-04-20T11:13:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:48 smithi078 ceph-mon[21464]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T11:13:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:48 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:48.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:48 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:13:48.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:48 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:13:48.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:48 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:48 smithi044 ceph-mon[16855]: osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] boot 2024-04-20T11:13:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:48 smithi044 ceph-mon[16855]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T11:13:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:48 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:48.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:48 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:13:48.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:48 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:13:48.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:48 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:48.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:48 smithi165 ceph-mon[21975]: osd.0 [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] boot 2024-04-20T11:13:48.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:48 smithi165 ceph-mon[21975]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T11:13:48.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:48 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:13:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T11:13:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T11:13:49.008 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-20T11:13:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-20T11:13:49.008 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114952 s, 912 MB/s 2024-04-20T11:13:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-20T11:13:49.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-04-20T11:13:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:13:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:49 smithi078 ceph-mon[21464]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:49 smithi078 ceph-mon[21464]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T11:13:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:49 smithi044 ceph-mon[16855]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:49 smithi044 ceph-mon[16855]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T11:13:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:49 smithi165 ceph-mon[21975]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T11:13:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:49 smithi165 ceph-mon[21975]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 -- 172.21.15.78:0/2746106051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8103ca0 msgr2=0x7f0df8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/2746106051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8103ca0 0x7f0df8104100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0099c0 tx=0x7f0dec009cd0 comp rx=0 tx=0).stop 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 -- 172.21.15.78:0/2746106051 shutdown_connections 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/2746106051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0df8104640 0x7f0df810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/2746106051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8102aa0 0x7f0df8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/2746106051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8103ca0 0x7f0df8104100 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 -- 172.21.15.78:0/2746106051 >> 172.21.15.78:0/2746106051 conn(0x7f0df80fe040 msgr2=0x7f0df8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 -- 172.21.15.78:0/2746106051 shutdown_connections 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 -- 172.21.15.78:0/2746106051 wait complete. 2024-04-20T11:13:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 Processor -- start 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.313+0000 7f0dfe17b700 1 -- start start 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0df8102aa0 0x7f0df80767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8103ca0 0x7f0df8074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 0x7f0df8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0df810e7a0 con 0x7f0df8102aa0 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0df810e620 con 0x7f0df8104640 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0df810e920 con 0x7f0df8103ca0 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 0x7f0df8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 0x7f0df8075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50548/0 (socket says 172.21.15.78:50548) 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 -- 172.21.15.78:0/1509124413 learned_addr learned my addr 172.21.15.78:0/1509124413 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df77fe700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0df8102aa0 0x7f0df80767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df6ffd700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8103ca0 0x7f0df8074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 -- 172.21.15.78:0/1509124413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8103ca0 msgr2=0x7f0df8074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8103ca0 0x7f0df8074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 -- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0df8102aa0 msgr2=0x7f0df80767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0df8102aa0 0x7f0df80767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 -- 172.21.15.78:0/1509124413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df8075c30 con 0x7f0df8104640 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df6ffd700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8103ca0 0x7f0df8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df7fff700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 0x7f0df8075380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0df0007ee0 tx=0x7f0df000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df000dbd0 con 0x7f0df8104640 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0df000dd70 con 0x7f0df8104640 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df81a2f40 con 0x7f0df8104640 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.314+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00133f0 con 0x7f0df8104640 2024-04-20T11:13:51.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.315+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df81a33a0 con 0x7f0df8104640 2024-04-20T11:13:51.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.315+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df8069740 con 0x7f0df8104640 2024-04-20T11:13:51.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.316+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0df001a020 con 0x7f0df8104640 2024-04-20T11:13:51.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.317+0000 7f0df4ff9700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0de00a3bb0 0x7f0de00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:13:51.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.317+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f0df00ce1a0 con 0x7f0df8104640 2024-04-20T11:13:51.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.317+0000 7f0df77fe700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0de00a3bb0 0x7f0de00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:13:51.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.318+0000 7f0df77fe700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0de00a3bb0 0x7f0de00a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0df8103b00 tx=0x7f0de800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:13:51.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.322+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0df0010070 con 0x7f0df8104640 2024-04-20T11:13:51.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:13:51.525+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0df8062e90 con 0x7f0de00a3bb0 2024-04-20T11:13:51.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:51 smithi044 ceph-mon[16855]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:51.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:13:51.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:13:51.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:51 smithi078 ceph-mon[21464]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:13:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:13:51.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:51 smithi165 ceph-mon[21975]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:13:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:13:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:52.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:52.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:52.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:13:52.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:52.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:52.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:13:52.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:13:52.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:52.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:13:52.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:13:52.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:13:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:53 smithi078 ceph-mon[21464]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:53.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:53 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi044 2024-04-20T11:13:53.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:53 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi044 to 17094M 2024-04-20T11:13:53.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:53 smithi044 ceph-mon[16855]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:53.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:53 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi044 2024-04-20T11:13:53.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:53 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi044 to 17094M 2024-04-20T11:13:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:53 smithi165 ceph-mon[21975]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:53 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi044 2024-04-20T11:13:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:53 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi044 to 17094M 2024-04-20T11:13:54.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:54 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/4233824322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]: dispatch 2024-04-20T11:13:54.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:54 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]: dispatch 2024-04-20T11:13:54.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:54 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]': finished 2024-04-20T11:13:54.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:54 smithi078 ceph-mon[21464]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T11:13:54.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:54 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:13:54.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:54 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/4233824322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]: dispatch 2024-04-20T11:13:54.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:54 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]: dispatch 2024-04-20T11:13:54.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:54 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]': finished 2024-04-20T11:13:54.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:54 smithi044 ceph-mon[16855]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T11:13:54.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:54 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:13:54.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:54 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/4233824322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]: dispatch 2024-04-20T11:13:54.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:54 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]: dispatch 2024-04-20T11:13:54.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:54 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e99f802-a969-42c1-8c47-75ac8cc78349"}]': finished 2024-04-20T11:13:54.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:54 smithi165 ceph-mon[21975]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T11:13:54.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:54 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:13:55.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:55 smithi078 ceph-mon[21464]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:55.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:55 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/3790457392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:13:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:55 smithi044 ceph-mon[16855]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:55 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/3790457392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:13:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:55 smithi165 ceph-mon[21975]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:55 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/3790457392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:13:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:57 smithi078 ceph-mon[21464]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:57.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:57 smithi044 ceph-mon[16855]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:57 smithi165 ceph-mon[21975]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:59.809 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:13:59 smithi078 ceph-mon[21464]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:59.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:13:59 smithi044 ceph-mon[16855]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:13:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:13:59 smithi165 ceph-mon[21975]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:01 smithi078 ceph-mon[21464]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:14:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:01.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:01 smithi044 ceph-mon[16855]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:01.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:14:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:01 smithi165 ceph-mon[21975]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T11:14:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:02 smithi078 ceph-mon[21464]: Deploying daemon osd.1 on smithi078 2024-04-20T11:14:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:02 smithi044 ceph-mon[16855]: Deploying daemon osd.1 on smithi078 2024-04-20T11:14:02.951 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:02 smithi165 ceph-mon[21975]: Deploying daemon osd.1 on smithi078 2024-04-20T11:14:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:03 smithi078 ceph-mon[21464]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:03.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:03 smithi044 ceph-mon[16855]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:03 smithi165 ceph-mon[21975]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:05.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:05 smithi078 ceph-mon[21464]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:05.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:05 smithi044 ceph-mon[16855]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:05 smithi165 ceph-mon[21975]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:07 smithi044 ceph-mon[16855]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:07.708 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:07 smithi078 ceph-mon[21464]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:07.709 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:07.709 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:07.709 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:07 smithi165 ceph-mon[21975]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:09.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:09 smithi165 ceph-mon[21975]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:09.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:09.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:09.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.793 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:09 smithi078 ceph-mon[21464]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:09.793 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.793 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.793 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:09.793 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:09.793 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:09 smithi044 ceph-mon[16855]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.118+0000 7f0df4ff9700 1 -- 172.21.15.78:0/1509124413 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0df8062e90 con 0x7f0de00a3bb0 2024-04-20T11:14:10.119 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 1 on host 'smithi078' 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0de00a3bb0 msgr2=0x7f0de00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0de00a3bb0 0x7f0de00a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0df8103b00 tx=0x7f0de800b5b0 comp rx=0 tx=0).stop 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 msgr2=0x7f0df8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 0x7f0df8075380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0df0007ee0 tx=0x7f0df000d3b0 comp rx=0 tx=0).stop 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 shutdown_connections 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0de00a3bb0 0x7f0de00a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df8104640 0x7f0df8075380 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0df8102aa0 0x7f0df80767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 --2- 172.21.15.78:0/1509124413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0df8103ca0 0x7f0df8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:10.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 >> 172.21.15.78:0/1509124413 conn(0x7f0df80fe040 msgr2=0x7f0df8106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:10.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 shutdown_connections 2024-04-20T11:14:10.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:14:10.121+0000 7f0dfe17b700 1 -- 172.21.15.78:0/1509124413 wait complete. 2024-04-20T11:14:10.747 DEBUG:teuthology.orchestra.run.smithi078:osd.1> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.1.service 2024-04-20T11:14:10.749 INFO:tasks.cephadm:Deploying osd.2 on smithi165 with /dev/vg_nvme/lv_4... 2024-04-20T11:14:10.750 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T11:14:10.997 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:14:11.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:11 smithi165 ceph-mon[21975]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:11.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:11.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:11.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:11.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:11 smithi165 ceph-mon[21975]: from='osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:14:11.426 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:11 smithi165 ceph-mon[21975]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:14:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:11 smithi078 ceph-mon[21464]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:11 smithi078 ceph-mon[21464]: from='osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:14:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:11 smithi078 ceph-mon[21464]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:14:11.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:11 smithi044 ceph-mon[16855]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:11.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:11.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:11.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:11.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:11 smithi044 ceph-mon[16855]: from='osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:14:11.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:11 smithi044 ceph-mon[16855]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T11:14:12.421 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:12 smithi165 ceph-mon[21975]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:14:12.594 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:12 smithi165 ceph-mon[21975]: from='osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:14:12.594 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:12 smithi165 ceph-mon[21975]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T11:14:12.594 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:12 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:12.595 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:12 smithi165 ceph-mon[21975]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:14:12.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:12 smithi044 ceph-mon[16855]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:14:12.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:12 smithi044 ceph-mon[16855]: from='osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:14:12.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:12 smithi044 ceph-mon[16855]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:12 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:12 smithi044 ceph-mon[16855]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-mon[21464]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-mon[21464]: from='osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-mon[21464]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:12.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-mon[21464]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:14:12.883 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1[26080]: 2024-04-20T11:14:12.555+0000 7ff8b78cc700 -1 osd.1 0 waiting for initial osdmap 2024-04-20T11:14:12.883 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1[26080]: 2024-04-20T11:14:12.559+0000 7ff8af6b7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:14:13.121 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T11:14:13.121 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T11:14:13.122 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-20T11:14:13.122 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-20T11:14:13.122 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115043 s, 911 MB/s 2024-04-20T11:14:13.122 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-20T11:14:13.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:13 smithi078 ceph-mon[21464]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:13.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:13 smithi078 ceph-mon[21464]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:14:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:13 smithi078 ceph-mon[21464]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T11:14:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:13 smithi078 ceph-mon[21464]: from='osd.1 ' entity='osd.1' 2024-04-20T11:14:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:13 smithi044 ceph-mon[16855]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:13 smithi044 ceph-mon[16855]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:14:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:13 smithi044 ceph-mon[16855]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T11:14:13.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:13.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:13 smithi044 ceph-mon[16855]: from='osd.1 ' entity='osd.1' 2024-04-20T11:14:13.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:13.452 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:13 smithi165 ceph-mon[21975]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:13.452 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:13 smithi165 ceph-mon[21975]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:14:13.452 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:13 smithi165 ceph-mon[21975]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T11:14:13.452 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:13.453 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:13 smithi165 ceph-mon[21975]: from='osd.1 ' entity='osd.1' 2024-04-20T11:14:13.453 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:13.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-04-20T11:14:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi078 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] boot 2024-04-20T11:14:14.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T11:14:14.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:14.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:14.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:14.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi078 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] boot 2024-04-20T11:14:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T11:14:14.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:14.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:14.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:14.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi078 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-20T11:14:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: osd.1 [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] boot 2024-04-20T11:14:14.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T11:14:14.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:14:14.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:14.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:14.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:14.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:15.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 -- 172.21.15.165:0/1290899394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204102ab0 msgr2=0x7f0204102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:15.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 --2- 172.21.15.165:0/1290899394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204102ab0 0x7f0204102ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009990 tx=0x7f01f4009ca0 comp rx=0 tx=0).stop 2024-04-20T11:14:15.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 -- 172.21.15.165:0/1290899394 shutdown_connections 2024-04-20T11:14:15.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 --2- 172.21.15.165:0/1290899394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0204104650 0x7f020410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:15.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 --2- 172.21.15.165:0/1290899394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204102ab0 0x7f0204102ed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:15.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 --2- 172.21.15.165:0/1290899394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 0x7f0204104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:15.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 -- 172.21.15.165:0/1290899394 >> 172.21.15.165:0/1290899394 conn(0x7f02040fe050 msgr2=0x7f0204100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:15.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 -- 172.21.15.165:0/1290899394 shutdown_connections 2024-04-20T11:14:15.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.522+0000 7f020c26f700 1 -- 172.21.15.165:0/1290899394 wait complete. 2024-04-20T11:14:15.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 Processor -- start 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 -- start start 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0204102ab0 0x7f02040767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 0x7f0204074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 0x7f0204075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f020410e8b0 con 0x7f0204102ab0 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f020410e730 con 0x7f0204103cb0 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020c26f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f020410ea30 con 0x7f0204104650 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 0x7f0204075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 0x7f0204075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35180/0 (socket says 172.21.15.165:35180) 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 -- 172.21.15.165:0/2556929989 learned_addr learned my addr 172.21.15.165:0/2556929989 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020980a700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 0x7f0204074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:15.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 -- 172.21.15.165:0/2556929989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 msgr2=0x7f0204074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 0x7f0204074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 -- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0204102ab0 msgr2=0x7f02040767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0204102ab0 0x7f02040767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020a80c700 1 -- 172.21.15.165:0/2556929989 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0204075c30 con 0x7f0204104650 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.523+0000 7f020980a700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 0x7f0204074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.524+0000 7f020a80c700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 0x7f0204075380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b970 tx=0x7f01fc00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.524+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc019070 con 0x7f0204104650 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.524+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f01fc014440 con 0x7f0204104650 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.524+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02041a2ff0 con 0x7f0204104650 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.524+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0075d0 con 0x7f0204104650 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.524+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a34d0 con 0x7f0204104650 2024-04-20T11:14:15.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.525+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0204069740 con 0x7f0204104650 2024-04-20T11:14:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.526+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f01fc01c070 con 0x7f0204104650 2024-04-20T11:14:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.526+0000 7f01fb7fe700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01f00a3bb0 0x7f01f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.527+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f01fc0ce450 con 0x7f0204104650 2024-04-20T11:14:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.527+0000 7f020a00b700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01f00a3bb0 0x7f01f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:15.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.528+0000 7f020a00b700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01f00a3bb0 0x7f01f00a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005f50 tx=0x7f01f4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:15.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.530+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f01fc09c680 con 0x7f0204104650 2024-04-20T11:14:15.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:15.738+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0204062e90 con 0x7f01f00a3bb0 2024-04-20T11:14:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:15 smithi078 ceph-mon[21464]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:15.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:15 smithi078 ceph-mon[21464]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T11:14:15.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:15 smithi044 ceph-mon[16855]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:15.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:15 smithi044 ceph-mon[16855]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T11:14:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:15 smithi165 ceph-mon[21975]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T11:14:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:15 smithi165 ceph-mon[21975]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T11:14:16.669 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:14:16.669 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:14:16.669 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:16.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:14:16.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:14:16.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:16.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:14:16.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:14:16.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:17 smithi165 ceph-mon[21975]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:17 smithi165 ceph-mon[21975]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:17 smithi078 ceph-mon[21464]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:17 smithi078 ceph-mon[21464]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:17.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:17 smithi044 ceph-mon[16855]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:17.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:17 smithi044 ceph-mon[16855]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:18.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:18 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/1329720030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]: dispatch 2024-04-20T11:14:18.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:18 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]: dispatch 2024-04-20T11:14:18.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:18 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]': finished 2024-04-20T11:14:18.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:18 smithi078 ceph-mon[21464]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T11:14:18.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:18 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:18.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:18 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/1329720030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]: dispatch 2024-04-20T11:14:18.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:18 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]: dispatch 2024-04-20T11:14:18.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:18 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]': finished 2024-04-20T11:14:18.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:18 smithi044 ceph-mon[16855]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T11:14:18.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:18 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:18.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:18 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/1329720030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]: dispatch 2024-04-20T11:14:18.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:18 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]: dispatch 2024-04-20T11:14:18.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:18 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25612bf3-e5b7-49e1-bf3c-929ddf669fc4"}]': finished 2024-04-20T11:14:18.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:18 smithi165 ceph-mon[21975]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T11:14:18.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:18 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:19 smithi078 ceph-mon[21464]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:19.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:19 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/3117138756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:14:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:19 smithi044 ceph-mon[16855]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:19 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/3117138756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:14:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:19 smithi165 ceph-mon[21975]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:19 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/3117138756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:14:21.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:21 smithi165 ceph-mon[21975]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:21 smithi078 ceph-mon[21464]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:21 smithi044 ceph-mon[16855]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:23 smithi078 ceph-mon[21464]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:23 smithi044 ceph-mon[16855]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:23 smithi165 ceph-mon[21975]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:25 smithi078 ceph-mon[21464]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:25 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:14:25.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:25 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:25.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:25 smithi044 ceph-mon[16855]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:25 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:14:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:25 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:25 smithi165 ceph-mon[21975]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:25 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T11:14:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:25 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:26.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:26 smithi165 ceph-mon[21975]: Deploying daemon osd.2 on smithi165 2024-04-20T11:14:26.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:26 smithi078 ceph-mon[21464]: Deploying daemon osd.2 on smithi165 2024-04-20T11:14:26.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:26 smithi044 ceph-mon[16855]: Deploying daemon osd.2 on smithi165 2024-04-20T11:14:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:27 smithi078 ceph-mon[21464]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:27 smithi044 ceph-mon[16855]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:27.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:27 smithi165 ceph-mon[21975]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:29.908 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:29 smithi165 ceph-mon[21975]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:29 smithi078 ceph-mon[21464]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:29.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:29 smithi044 ceph-mon[16855]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:31 smithi165 ceph-mon[21975]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:32.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:31 smithi078 ceph-mon[21464]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:32.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:32.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:32.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:32.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:31 smithi044 ceph-mon[16855]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:32.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:32.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:32.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:33 smithi165 ceph-mon[21975]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:33 smithi078 ceph-mon[21464]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:33 smithi044 ceph-mon[16855]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:34.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:34.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:34.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:34.275 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 2 on host 'smithi165' 2024-04-20T11:14:34.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.275+0000 7f01fb7fe700 1 -- 172.21.15.165:0/2556929989 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0204062e90 con 0x7f01f00a3bb0 2024-04-20T11:14:34.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.277+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01f00a3bb0 msgr2=0x7f01f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:34.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.277+0000 7f020c26f700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01f00a3bb0 0x7f01f00a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005f50 tx=0x7f01f4005e80 comp rx=0 tx=0).stop 2024-04-20T11:14:34.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.277+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 msgr2=0x7f0204075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:34.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.277+0000 7f020c26f700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 0x7f0204075380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b970 tx=0x7f01fc00bd30 comp rx=0 tx=0).stop 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 shutdown_connections 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f01f00a3bb0 0x7f01f00a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0204104650 0x7f0204075380 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0204102ab0 0x7f02040767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 --2- 172.21.15.165:0/2556929989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204103cb0 0x7f0204074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 >> 172.21.15.165:0/2556929989 conn(0x7f02040fe050 msgr2=0x7f0204106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 shutdown_connections 2024-04-20T11:14:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:14:34.278+0000 7f020c26f700 1 -- 172.21.15.165:0/2556929989 wait complete. 2024-04-20T11:14:34.871 DEBUG:teuthology.orchestra.run.smithi165:osd.2> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.2.service 2024-04-20T11:14:34.874 INFO:tasks.cephadm:Deploying osd.3 on smithi044 with /dev/vg_nvme/lv_3... 2024-04-20T11:14:34.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T11:14:35.344 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:14:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:35 smithi165 ceph-mon[21975]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:35 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:35 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:35 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:35 smithi165 ceph-mon[21975]: from='osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:14:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:35 smithi165 ceph-mon[21975]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:14:35.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:35 smithi044 ceph-mon[16855]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:35.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:35 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:35.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:35 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:35.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:35 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:35.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:35 smithi044 ceph-mon[16855]: from='osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:14:35.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:35 smithi044 ceph-mon[16855]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:14:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:35 smithi078 ceph-mon[21464]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:35.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:35 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:35.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:35 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:35.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:35 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:14:35.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:35 smithi078 ceph-mon[21464]: from='osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:14:35.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:35 smithi078 ceph-mon[21464]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T11:14:36.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-mon[21975]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:14:36.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-mon[21975]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T11:14:36.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:36.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-mon[21975]: from='osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:14:36.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-mon[21975]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:14:36.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:36 smithi044 ceph-mon[16855]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:14:36.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:36 smithi044 ceph-mon[16855]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T11:14:36.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:36 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:36.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:36 smithi044 ceph-mon[16855]: from='osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:14:36.578 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:36 smithi044 ceph-mon[16855]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:14:36.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:36 smithi078 ceph-mon[21464]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T11:14:36.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:36 smithi078 ceph-mon[21464]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T11:14:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:36 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:36 smithi078 ceph-mon[21464]: from='osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:14:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:36 smithi078 ceph-mon[21464]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:14:36.980 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2[27116]: 2024-04-20T11:14:36.706+0000 7feb8a5ac700 -1 osd.2 0 waiting for initial osdmap 2024-04-20T11:14:36.980 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2[27116]: 2024-04-20T11:14:36.710+0000 7feb82397700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:14:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:37 smithi165 ceph-mon[21975]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:37 smithi165 ceph-mon[21975]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:14:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:37 smithi165 ceph-mon[21975]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T11:14:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:37.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:37 smithi165 ceph-mon[21975]: from='osd.2 ' entity='osd.2' 2024-04-20T11:14:37.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:37 smithi044 ceph-mon[16855]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:37.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:37 smithi044 ceph-mon[16855]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:14:37.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:37 smithi044 ceph-mon[16855]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T11:14:37.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:37.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:37.507 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:37 smithi044 ceph-mon[16855]: from='osd.2 ' entity='osd.2' 2024-04-20T11:14:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:37 smithi078 ceph-mon[21464]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:37 smithi078 ceph-mon[21464]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:14:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:37 smithi078 ceph-mon[21464]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T11:14:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:37 smithi078 ceph-mon[21464]: from='osd.2 ' entity='osd.2' 2024-04-20T11:14:38.529 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T11:14:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T11:14:38.530 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-20T11:14:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-20T11:14:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0390285 s, 269 MB/s 2024-04-20T11:14:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-20T11:14:38.579 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:14:38.579 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:14:38.579 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] boot 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.580 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:14:38.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] boot 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:38.692 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:38.693 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.693 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: osd.2 [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] boot 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:14:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:38.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:14:38.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:38.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:14:39.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:39 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi165 2024-04-20T11:14:39.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:39 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-20T11:14:39.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:39 smithi044 ceph-mon[16855]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:39.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T11:14:39.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:39 smithi044 ceph-mon[16855]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T11:14:39.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:14:39.568 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-04-20T11:14:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:39 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi165 2024-04-20T11:14:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:39 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-20T11:14:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:39 smithi078 ceph-mon[21464]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:39 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T11:14:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:39 smithi078 ceph-mon[21464]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T11:14:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:39 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:14:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:39 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi165 2024-04-20T11:14:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:39 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-20T11:14:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:39 smithi165 ceph-mon[21975]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T11:14:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:39 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T11:14:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:39 smithi165 ceph-mon[21975]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T11:14:39.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:39 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:14:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:14:40.967 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:14:40.967 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:40 smithi044 ceph-mon[16855]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T11:14:40.967 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:40 smithi044 ceph-mon[16855]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:14:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:40 smithi078 ceph-mon[21464]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T11:14:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:40 smithi078 ceph-mon[21464]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:41.188 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:14:41.188 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.188 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29776]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.188 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29776]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.188 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:14:41.189 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.189 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29779]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.189 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29779]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.198 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30744]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:14:41.198 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30744]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.198 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30744]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.198 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30744]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.199 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:14:41.199 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:40 smithi165 ceph-mon[21975]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T11:14:41.199 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:40 smithi165 ceph-mon[21975]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:40 smithi044 sudo[36159]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:40 smithi044 sudo[36159]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:40 smithi044 sudo[36159]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:40 smithi044 sudo[36159]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36165]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36165]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36165]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.240 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36165]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:14:41.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30750]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30750]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30753]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:14:41.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30753]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30753]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.480 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30747]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:14:41.480 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.480 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30747]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.480 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:14:41 smithi165 sudo[30747]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36168]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36168]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36168]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36168]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36171]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36171]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36171]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 sudo[36171]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29782]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T11:14:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29782]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29782]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29782]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:41.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29785]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T11:14:41.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29785]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T11:14:41.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29785]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T11:14:41.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 sudo[29785]: pam_unix(sudo:session): session closed for user root 2024-04-20T11:14:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T11:14:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:41 smithi044 ceph-mon[16855]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.231 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.231 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.231 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.231 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.231 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.231 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:41 smithi165 ceph-mon[21975]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:14:42.439 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:41 smithi078 ceph-mon[21464]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T11:14:42.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 -- 172.21.15.44:0/3242428783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07381019b0 msgr2=0x7f0738101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:42.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 --2- 172.21.15.44:0/3242428783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07381019b0 0x7f0738101dd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f07280098f0 tx=0x7f0728009c00 comp rx=0 tx=0).stop 2024-04-20T11:14:42.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 -- 172.21.15.44:0/3242428783 shutdown_connections 2024-04-20T11:14:42.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 --2- 172.21.15.44:0/3242428783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0738103550 0x7f0738109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 --2- 172.21.15.44:0/3242428783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07381019b0 0x7f0738101dd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 --2- 172.21.15.44:0/3242428783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0738102bb0 0x7f0738103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.466+0000 7f073ec00700 1 -- 172.21.15.44:0/3242428783 >> 172.21.15.44:0/3242428783 conn(0x7f07380fcf50 msgr2=0x7f07380ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.467+0000 7f073ec00700 1 -- 172.21.15.44:0/3242428783 shutdown_connections 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.467+0000 7f073ec00700 1 -- 172.21.15.44:0/3242428783 wait complete. 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.468+0000 7f073ec00700 1 Processor -- start 2024-04-20T11:14:42.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.468+0000 7f073ec00700 1 -- start start 2024-04-20T11:14:42.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.468+0000 7f073ec00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 0x7f073819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f073ec00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0738102bb0 0x7f073819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f073ec00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738103550 0x7f07381a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f073ec00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f073810d700 con 0x7f0738102bb0 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f073ec00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f073810d580 con 0x7f07381019b0 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f073ec00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f073810d880 con 0x7f0738103550 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 0x7f073819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f072f400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0738102bb0 0x7f073819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 0x7f073819b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60860/0 (socket says 172.21.15.44:60860) 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 -- 172.21.15.44:0/2016668928 learned_addr learned my addr 172.21.15.44:0/2016668928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:14:42.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f073cc00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738103550 0x7f07381a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:42.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 -- 172.21.15.44:0/2016668928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738103550 msgr2=0x7f07381a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738103550 0x7f07381a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 -- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0738102bb0 msgr2=0x7f073819b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0738102bb0 0x7f073819b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.469+0000 7f0737e00700 1 -- 172.21.15.44:0/2016668928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07381a2cd0 con 0x7f07381019b0 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.470+0000 7f0737e00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 0x7f073819b070 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f072800c010 tx=0x7f072800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.470+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072800fe60 con 0x7f07381019b0 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.470+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f072800fac0 con 0x7f07381019b0 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.470+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381a2f60 con 0x7f07381019b0 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.470+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728017400 con 0x7f07381019b0 2024-04-20T11:14:42.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.470+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381a3300 con 0x7f07381019b0 2024-04-20T11:14:42.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.472+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07381a3530 con 0x7f07381019b0 2024-04-20T11:14:42.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.473+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0728003680 con 0x7f07381019b0 2024-04-20T11:14:42.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.475+0000 7f0735600700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f071c0a3bb0 0x7f071c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:14:42.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.475+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f07280cfc00 con 0x7f07381019b0 2024-04-20T11:14:42.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.475+0000 7f072f400700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f071c0a3bb0 0x7f071c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:14:42.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.476+0000 7f072f400700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f071c0a3bb0 0x7f071c0a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0720009c60 tx=0x7f0720009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:14:42.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.480+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0728013030 con 0x7f07381019b0 2024-04-20T11:14:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.720+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f07380ff160 con 0x7f071c0a3bb0 2024-04-20T11:14:42.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:14:42.963+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f07280999f0 con 0x7f07381019b0 2024-04-20T11:14:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:43 smithi078 ceph-mon[21464]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:43.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:14:43.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:14:43.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:43.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:43 smithi078 ceph-mon[21464]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:43 smithi165 ceph-mon[21975]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:43 smithi165 ceph-mon[21975]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:43 smithi044 ceph-mon[16855]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:14:43.889 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:43.890 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:43 smithi044 ceph-mon[16855]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T11:14:44.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:44 smithi044 ceph-mon[16855]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:44.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:44 smithi078 ceph-mon[21464]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:44.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:44 smithi165 ceph-mon[21975]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:14:45.503 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:45 smithi044 ceph-mon[16855]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:45.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:45 smithi078 ceph-mon[21464]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:45 smithi165 ceph-mon[21975]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:47 smithi078 ceph-mon[21464]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:47 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2685940329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]: dispatch 2024-04-20T11:14:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:47 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]: dispatch 2024-04-20T11:14:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:47 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]': finished 2024-04-20T11:14:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:47 smithi078 ceph-mon[21464]: osdmap e22: 4 total, 3 up, 4 in 2024-04-20T11:14:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:47 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:14:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:47 smithi044 ceph-mon[16855]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:47 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2685940329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]: dispatch 2024-04-20T11:14:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:47 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]: dispatch 2024-04-20T11:14:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:47 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]': finished 2024-04-20T11:14:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:47 smithi044 ceph-mon[16855]: osdmap e22: 4 total, 3 up, 4 in 2024-04-20T11:14:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:47 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:14:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:47 smithi165 ceph-mon[21975]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:47 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2685940329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]: dispatch 2024-04-20T11:14:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:47 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]: dispatch 2024-04-20T11:14:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:47 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e081b5a8-ec52-40c2-bebd-3dd47ac73986"}]': finished 2024-04-20T11:14:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:47 smithi165 ceph-mon[21975]: osdmap e22: 4 total, 3 up, 4 in 2024-04-20T11:14:47.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:47 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:14:48.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:48 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2623961682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:14:48.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:48 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2623961682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:14:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:48 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2623961682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:14:49.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:49 smithi078 ceph-mon[21464]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:49.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:49 smithi044 ceph-mon[16855]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:49 smithi165 ceph-mon[21975]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:51.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:51 smithi044 ceph-mon[16855]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:51 smithi078 ceph-mon[21464]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:51 smithi165 ceph-mon[21975]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:53 smithi078 ceph-mon[21464]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:53.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:53 smithi044 ceph-mon[16855]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:53 smithi165 ceph-mon[21975]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:55.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:55 smithi078 ceph-mon[21464]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:55 smithi044 ceph-mon[16855]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:55 smithi165 ceph-mon[21975]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:56.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:56 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:14:56.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:56 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:56.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:56 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:14:56.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:56 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:56.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:56 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T11:14:56.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:56 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:14:57.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:57 smithi044 ceph-mon[16855]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:57.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:57 smithi044 ceph-mon[16855]: Deploying daemon osd.3 on smithi044 2024-04-20T11:14:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:57 smithi078 ceph-mon[21464]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:57 smithi078 ceph-mon[21464]: Deploying daemon osd.3 on smithi044 2024-04-20T11:14:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:57 smithi165 ceph-mon[21975]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:57 smithi165 ceph-mon[21975]: Deploying daemon osd.3 on smithi044 2024-04-20T11:14:59.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:14:59 smithi078 ceph-mon[21464]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:59.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:14:59 smithi044 ceph-mon[16855]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:14:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:14:59 smithi165 ceph-mon[21975]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:01.651 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:01 smithi044 ceph-mon[16855]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:01.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:01 smithi078 ceph-mon[21464]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:01 smithi165 ceph-mon[21975]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:03.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:03 smithi078 ceph-mon[21464]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:03.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:03 smithi044 ceph-mon[16855]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:03 smithi165 ceph-mon[21975]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:05.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:05 smithi044 ceph-mon[16855]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:05.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:05.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:05.459 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:05.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:05 smithi078 ceph-mon[21464]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:05.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:05 smithi165 ceph-mon[21975]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:07.431 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:07 smithi044 ceph-mon[16855]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:07.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:07 smithi078 ceph-mon[21464]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:07 smithi165 ceph-mon[21975]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:09.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:09 smithi078 ceph-mon[21464]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:09.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:09.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:09.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:09 smithi078 ceph-mon[21464]: from='osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:15:09.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:09 smithi078 ceph-mon[21464]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:15:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:09 smithi044 ceph-mon[16855]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:09 smithi044 ceph-mon[16855]: from='osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:15:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:09 smithi044 ceph-mon[16855]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:15:09.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:09 smithi165 ceph-mon[21975]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:09.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:09.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:09.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:09 smithi165 ceph-mon[21975]: from='osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:15:09.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:09 smithi165 ceph-mon[21975]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T11:15:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.374+0000 7f0735600700 1 -- 172.21.15.44:0/2016668928 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f07380ff160 con 0x7f071c0a3bb0 2024-04-20T11:15:10.376 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 3 on host 'smithi044' 2024-04-20T11:15:10.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f071c0a3bb0 msgr2=0x7f071c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:10.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f071c0a3bb0 0x7f071c0a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0720009c60 tx=0x7f0720009260 comp rx=0 tx=0).stop 2024-04-20T11:15:10.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 msgr2=0x7f073819b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 0x7f073819b070 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f072800c010 tx=0x7f072800f690 comp rx=0 tx=0).stop 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 shutdown_connections 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f071c0a3bb0 0x7f071c0a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738103550 0x7f07381a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07381019b0 0x7f073819b070 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 --2- 172.21.15.44:0/2016668928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0738102bb0 0x7f073819b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 >> 172.21.15.44:0/2016668928 conn(0x7f07380fcf50 msgr2=0x7f07380fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 shutdown_connections 2024-04-20T11:15:10.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:15:10.378+0000 7f073ec00700 1 -- 172.21.15.44:0/2016668928 wait complete. 2024-04-20T11:15:11.138 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:11.138 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:15:11.138 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:15:11.138 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.139 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:11.204 DEBUG:teuthology.orchestra.run.smithi044:osd.3> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.3.service 2024-04-20T11:15:11.209 INFO:tasks.cephadm:Deploying osd.4 on smithi078 with /dev/vg_nvme/lv_3... 2024-04-20T11:15:11.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:15:11.237 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:11.238 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:11.238 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.238 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.238 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.238 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:15:11.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:11.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T11:15:11.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:15:11.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:11.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:12.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:15:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T11:15:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] boot 2024-04-20T11:15:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: osdmap e25: 4 total, 4 up, 4 in 2024-04-20T11:15:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:11 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] boot 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: osdmap e25: 4 total, 4 up, 4 in 2024-04-20T11:15:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:11 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:15:12.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T11:15:12.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:12.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: osd.3 [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] boot 2024-04-20T11:15:12.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: osdmap e25: 4 total, 4 up, 4 in 2024-04-20T11:15:12.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:11 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:15:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:12 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:15:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:12 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:15:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:12 smithi078 ceph-mon[21464]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:12 smithi078 ceph-mon[21464]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T11:15:13.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:12 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:15:13.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:12 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:15:13.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:12 smithi044 ceph-mon[16855]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:13.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:12 smithi044 ceph-mon[16855]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T11:15:13.461 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T11:15:13.461 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T11:15:13.461 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-20T11:15:13.461 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-20T11:15:13.461 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114462 s, 916 MB/s 2024-04-20T11:15:13.461 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-20T11:15:13.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:12 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:15:13.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:12 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:15:13.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:12 smithi165 ceph-mon[21975]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T11:15:13.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:12 smithi165 ceph-mon[21975]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T11:15:14.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-04-20T11:15:14.419 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:15:14.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:14 smithi078 ceph-mon[21464]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T11:15:14.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:14 smithi165 ceph-mon[21975]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T11:15:15.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:14 smithi044 ceph-mon[16855]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T11:15:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:15 smithi165 ceph-mon[21975]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:15 smithi165 ceph-mon[21975]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T11:15:16.016 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:15 smithi078 ceph-mon[21464]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:16.016 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:15 smithi078 ceph-mon[21464]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 -- 172.21.15.78:0/2166976016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c108880 msgr2=0x7fc60c10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 --2- 172.21.15.78:0/2166976016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c108880 0x7fc60c10ac70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc604009990 tx=0x7fc604009ca0 comp rx=0 tx=0).stop 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 -- 172.21.15.78:0/2166976016 shutdown_connections 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 --2- 172.21.15.78:0/2166976016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c108880 0x7fc60c10ac70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 --2- 172.21.15.78:0/2166976016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60c103620 0x7fc60c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 --2- 172.21.15.78:0/2166976016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c105f50 0x7fc60c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 -- 172.21.15.78:0/2166976016 >> 172.21.15.78:0/2166976016 conn(0x7fc60c0fcfb0 msgr2=0x7fc60c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 -- 172.21.15.78:0/2166976016 shutdown_connections 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.118+0000 7fc611b24700 1 -- 172.21.15.78:0/2166976016 wait complete. 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 Processor -- start 2024-04-20T11:15:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 -- start start 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 0x7fc60c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60c105f50 0x7fc60c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c108880 0x7fc60c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10e860 con 0x7fc60c105f50 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10e6e0 con 0x7fc60c103620 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc611b24700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10e9e0 con 0x7fc60c108880 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 0x7fc60c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 0x7fc60c198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34212/0 (socket says 172.21.15.78:34212) 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 -- 172.21.15.78:0/2678725648 learned_addr learned my addr 172.21.15.78:0/2678725648 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60bfff700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c108880 0x7fc60c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 -- 172.21.15.78:0/2678725648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c108880 msgr2=0x7fc60c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c108880 0x7fc60c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 -- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60c105f50 msgr2=0x7fc60c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60c105f50 0x7fc60c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60b7fe700 1 -- 172.21.15.78:0/2678725648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c1a0930 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.119+0000 7fc60bfff700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c108880 0x7fc60c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.120+0000 7fc60b7fe700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 0x7fc60c198ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00ba50 tx=0x7fc5fc00be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.120+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc019070 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.120+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5fc014440 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.120+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0075b0 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.120+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c1a0c20 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.120+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c1a1160 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.121+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c069740 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.122+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc5fc01c070 con 0x7fc60c103620 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.122+0000 7fc608ff9700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5f80a3c40 0x7fc5f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:16.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.123+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fc5fc0ceb50 con 0x7fc60c103620 2024-04-20T11:15:16.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.123+0000 7fc60affd700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5f80a3c40 0x7fc5f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:16.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.124+0000 7fc60affd700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5f80a3c40 0x7fc5f80a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005fd0 tx=0x7fc5f4005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:16.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.126+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc5fc09c750 con 0x7fc60c103620 2024-04-20T11:15:16.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:15 smithi044 ceph-mon[16855]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:16.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:15 smithi044 ceph-mon[16855]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T11:15:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:16.332+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc60c062e90 con 0x7fc5f80a3c40 2024-04-20T11:15:16.976 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:16.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T11:15:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:16.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T11:15:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:17.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:17.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:17 smithi165 ceph-mon[21975]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2024-04-20T11:15:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:17 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi044 2024-04-20T11:15:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:17 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi044 to 8547M 2024-04-20T11:15:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:17 smithi165 ceph-mon[21975]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:18.129 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:17 smithi078 ceph-mon[21464]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2024-04-20T11:15:18.129 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:17 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi044 2024-04-20T11:15:18.129 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:17 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi044 to 8547M 2024-04-20T11:15:18.129 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:17 smithi078 ceph-mon[21464]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:18.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:17 smithi044 ceph-mon[16855]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2024-04-20T11:15:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:17 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi044 2024-04-20T11:15:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:17 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi044 to 8547M 2024-04-20T11:15:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:17 smithi044 ceph-mon[16855]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:19.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:18 smithi078 ceph-mon[21464]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-20T11:15:19.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:18 smithi044 ceph-mon[16855]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-20T11:15:19.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:18 smithi165 ceph-mon[21975]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-20T11:15:20.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:19 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/1752250708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]: dispatch 2024-04-20T11:15:20.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:19 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]: dispatch 2024-04-20T11:15:20.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:19 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]': finished 2024-04-20T11:15:20.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:19 smithi078 ceph-mon[21464]: osdmap e30: 5 total, 4 up, 5 in 2024-04-20T11:15:20.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:19 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:20.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:19 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/1752250708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]: dispatch 2024-04-20T11:15:20.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:19 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]: dispatch 2024-04-20T11:15:20.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:19 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]': finished 2024-04-20T11:15:20.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:19 smithi044 ceph-mon[16855]: osdmap e30: 5 total, 4 up, 5 in 2024-04-20T11:15:20.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:19 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:20.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:19 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/1752250708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]: dispatch 2024-04-20T11:15:20.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:19 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]: dispatch 2024-04-20T11:15:20.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:19 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97110c0d-1f21-4f21-be1f-9b026c6d37ba"}]': finished 2024-04-20T11:15:20.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:19 smithi165 ceph-mon[21975]: osdmap e30: 5 total, 4 up, 5 in 2024-04-20T11:15:20.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:19 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:21.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:20 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/329171599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:15:21.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:20 smithi078 ceph-mon[21464]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:15:21.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:20 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/329171599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:15:21.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:20 smithi044 ceph-mon[16855]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:15:21.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:20 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/329171599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:15:21.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:20 smithi165 ceph-mon[21975]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:15:23.390 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:23 smithi078 ceph-mon[21464]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:23.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:23 smithi044 ceph-mon[16855]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:23.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:23 smithi165 ceph-mon[21975]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:25.347 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:25 smithi078 ceph-mon[21464]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:25.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:25 smithi044 ceph-mon[16855]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:25 smithi165 ceph-mon[21975]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:27.353 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:27 smithi078 ceph-mon[21464]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:27.353 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:15:27.353 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:27.353 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:27 smithi078 ceph-mon[21464]: Deploying daemon osd.4 on smithi078 2024-04-20T11:15:27.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:27 smithi044 ceph-mon[16855]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:15:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:27 smithi044 ceph-mon[16855]: Deploying daemon osd.4 on smithi078 2024-04-20T11:15:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:27 smithi165 ceph-mon[21975]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T11:15:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:27 smithi165 ceph-mon[21975]: Deploying daemon osd.4 on smithi078 2024-04-20T11:15:29.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:29 smithi078 ceph-mon[21464]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:29.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:29 smithi044 ceph-mon[16855]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:29 smithi165 ceph-mon[21975]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:31.301 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:31 smithi078 ceph-mon[21464]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:31.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:31 smithi044 ceph-mon[16855]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:31 smithi165 ceph-mon[21975]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:33 smithi078 ceph-mon[21464]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:33 smithi165 ceph-mon[21975]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:33.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:33 smithi044 ceph-mon[16855]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:33.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:33.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:33.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:35.221 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:35 smithi078 ceph-mon[21464]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.320+0000 7fc608ff9700 1 -- 172.21.15.78:0/2678725648 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc60c062e90 con 0x7fc5f80a3c40 2024-04-20T11:15:35.321 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 4 on host 'smithi078' 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5f80a3c40 msgr2=0x7fc5f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5f80a3c40 0x7fc5f80a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005fd0 tx=0x7fc5f4005d00 comp rx=0 tx=0).stop 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 msgr2=0x7fc60c198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 0x7fc60c198ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00ba50 tx=0x7fc5fc00be10 comp rx=0 tx=0).stop 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 shutdown_connections 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5f80a3c40 0x7fc5f80a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc60c108880 0x7fc60c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60c103620 0x7fc60c198ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 --2- 172.21.15.78:0/2678725648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60c105f50 0x7fc60c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 >> 172.21.15.78:0/2678725648 conn(0x7fc60c0fcfb0 msgr2=0x7fc60c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 shutdown_connections 2024-04-20T11:15:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:15:35.323+0000 7fc611b24700 1 -- 172.21.15.78:0/2678725648 wait complete. 2024-04-20T11:15:35.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:35 smithi165 ceph-mon[21975]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:35.497 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:35 smithi044 ceph-mon[16855]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:36.335 DEBUG:teuthology.orchestra.run.smithi078:osd.4> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.4.service 2024-04-20T11:15:36.337 INFO:tasks.cephadm:Deploying osd.5 on smithi165 with /dev/vg_nvme/lv_3... 2024-04-20T11:15:36.338 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T11:15:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:15:36.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:36 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:36.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:36 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:36 smithi078 ceph-mon[21464]: from='osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:15:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:36 smithi078 ceph-mon[21464]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:15:36.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:36 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:36.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:36 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:36.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:36 smithi044 ceph-mon[16855]: from='osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:15:36.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:36 smithi044 ceph-mon[16855]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:15:36.706 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:36 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:36.707 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:36 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:36.707 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:36 smithi165 ceph-mon[21975]: from='osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:15:36.707 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:36 smithi165 ceph-mon[21975]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T11:15:37.346 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:37.346 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:15:37.346 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: from='osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:15:37.346 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T11:15:37.346 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:37.347 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:15:37.347 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:37.347 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:15:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: from='osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:15:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T11:15:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:15:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:37.650 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:37.650 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T11:15:37.650 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: from='osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:15:37.650 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T11:15:37.650 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:37.651 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:15:37.651 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:37.651 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:37.938 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:15:37.678+0000 7f382fa5f700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T11:15:37.938 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:15:37 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:15:37.685+0000 7f382784a700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:15:38.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:38 smithi165 ceph-mon[21975]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:15:38.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:38 smithi165 ceph-mon[21975]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T11:15:38.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:38.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:38.538 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:38 smithi165 ceph-mon[21975]: from='osd.4 ' entity='osd.4' 2024-04-20T11:15:38.538 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T11:15:38.538 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T11:15:38.538 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-20T11:15:38.538 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-20T11:15:38.538 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125442 s, 836 MB/s 2024-04-20T11:15:38.539 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-20T11:15:38.554 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:38 smithi078 ceph-mon[21464]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:15:38.554 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:38 smithi078 ceph-mon[21464]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T11:15:38.554 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:38.554 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:38.554 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:38 smithi078 ceph-mon[21464]: from='osd.4 ' entity='osd.4' 2024-04-20T11:15:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:38 smithi044 ceph-mon[16855]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:15:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:38 smithi044 ceph-mon[16855]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T11:15:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:38 smithi044 ceph-mon[16855]: from='osd.4 ' entity='osd.4' 2024-04-20T11:15:39.192 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-04-20T11:15:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:15:39.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:15:39.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:15:39.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:39.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:39.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] boot 2024-04-20T11:15:39.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: osdmap e33: 5 total, 5 up, 5 in 2024-04-20T11:15:39.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:39 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:39.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:15:39.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:15:39.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:39.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:39.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] boot 2024-04-20T11:15:39.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: osdmap e33: 5 total, 5 up, 5 in 2024-04-20T11:15:39.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:39 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:39.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:15:39.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:15:39.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T11:15:39.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:39.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: osd.4 [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] boot 2024-04-20T11:15:39.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: osdmap e33: 5 total, 5 up, 5 in 2024-04-20T11:15:39.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:15:40.952 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T11:15:40.952 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:40.952 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:40.952 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:40.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:40.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 -- 172.21.15.165:0/3981553576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 msgr2=0x7ff8c4109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 --2- 172.21.15.165:0/3981553576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c4109e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009990 tx=0x7ff8b4009ca0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 -- 172.21.15.165:0/3981553576 shutdown_connections 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 --2- 172.21.15.165:0/3981553576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c4109e00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 --2- 172.21.15.165:0/3981553576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c41019b0 0x7ff8c4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 --2- 172.21.15.165:0/3981553576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8c4102bb0 0x7ff8c4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 -- 172.21.15.165:0/3981553576 >> 172.21.15.165:0/3981553576 conn(0x7ff8c40fcf50 msgr2=0x7ff8c40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 -- 172.21.15.165:0/3981553576 shutdown_connections 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.947+0000 7ff8ca009700 1 -- 172.21.15.165:0/3981553576 wait complete. 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 Processor -- start 2024-04-20T11:15:40.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 -- start start 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c41019b0 0x7ff8c4105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8c4102bb0 0x7ff8c4103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c41044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c410d750 con 0x7ff8c41019b0 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c410d5d0 con 0x7ff8c4102bb0 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8ca009700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c410d8d0 con 0x7ff8c4103550 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8c3fff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c41044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8c3fff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c41044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:46710/0 (socket says 172.21.15.165:46710) 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8c3fff700 1 -- 172.21.15.165:0/2831689486 learned_addr learned my addr 172.21.15.165:0/2831689486 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.948+0000 7ff8c2ffd700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8c4102bb0 0x7ff8c4103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:40.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c3fff700 1 -- 172.21.15.165:0/2831689486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8c4102bb0 msgr2=0x7ff8c4103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c3fff700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8c4102bb0 0x7ff8c4103f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c3fff700 1 -- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c41019b0 msgr2=0x7ff8c4105940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c3fff700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c41019b0 0x7ff8c4105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c3fff700 1 -- 172.21.15.165:0/2831689486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c4104d80 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c3fff700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c41044d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40059d0 tx=0x7ff8b400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4027070 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8b400fb40 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c41a2f50 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40173f0 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.949+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c41a32f0 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.950+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c4069740 con 0x7ff8c4103550 2024-04-20T11:15:40.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.951+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff8b402a070 con 0x7ff8c4103550 2024-04-20T11:15:40.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.951+0000 7ff8c0ff9700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff8a40a3c40 0x7ff8a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:15:40.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.951+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7ff8b40d0080 con 0x7ff8c4103550 2024-04-20T11:15:40.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.952+0000 7ff8c37fe700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff8a40a3c40 0x7ff8a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:15:40.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.953+0000 7ff8c37fe700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff8a40a3c40 0x7ff8a40a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4102a10 tx=0x7ff8bc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:15:40.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:40.957+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff8b4013070 con 0x7ff8c4103550 2024-04-20T11:15:41.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:15:41.163+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff8c40ff060 con 0x7ff8a40a3c40 2024-04-20T11:15:41.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T11:15:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:15:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:41 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi078 2024-04-20T11:15:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:41 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-20T11:15:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:41 smithi165 ceph-mon[21975]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:15:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:15:41.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:41 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:42.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:41 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi078 2024-04-20T11:15:42.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:41 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-20T11:15:42.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:41 smithi078 ceph-mon[21464]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:42.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:15:42.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:15:42.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:41 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:42.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:41 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi078 2024-04-20T11:15:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:41 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-20T11:15:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:41 smithi044 ceph-mon[16855]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:15:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:15:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:41 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:42.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:42 smithi165 ceph-mon[21975]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:43.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:42 smithi078 ceph-mon[21464]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:43.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:42 smithi044 ceph-mon[16855]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:15:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:43 smithi165 ceph-mon[21975]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:44.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:43 smithi078 ceph-mon[21464]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:44.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:43 smithi044 ceph-mon[16855]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:44.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/3313762173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]: dispatch 2024-04-20T11:15:44.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:44 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]: dispatch 2024-04-20T11:15:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:44 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]': finished 2024-04-20T11:15:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:44 smithi165 ceph-mon[21975]: osdmap e35: 6 total, 5 up, 6 in 2024-04-20T11:15:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:44 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:15:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/1098340519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:15:45.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/3313762173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]: dispatch 2024-04-20T11:15:45.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:44 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]: dispatch 2024-04-20T11:15:45.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:44 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]': finished 2024-04-20T11:15:45.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:44 smithi078 ceph-mon[21464]: osdmap e35: 6 total, 5 up, 6 in 2024-04-20T11:15:45.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:44 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:15:45.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/1098340519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:15:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/3313762173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]: dispatch 2024-04-20T11:15:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:44 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]: dispatch 2024-04-20T11:15:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:44 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e45eea6-98f1-48a0-aac0-8836c0e46fd1"}]': finished 2024-04-20T11:15:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:44 smithi044 ceph-mon[16855]: osdmap e35: 6 total, 5 up, 6 in 2024-04-20T11:15:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:44 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:15:45.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/1098340519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:15:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:45 smithi165 ceph-mon[21975]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:46.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:45 smithi078 ceph-mon[21464]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:46.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:45 smithi044 ceph-mon[16855]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:47.908 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:47 smithi165 ceph-mon[21975]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:48.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:47 smithi078 ceph-mon[21464]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:48.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:47 smithi044 ceph-mon[16855]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:49.033 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:48 smithi165 ceph-mon[21975]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:49.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:48 smithi078 ceph-mon[21464]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:49.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:48 smithi044 ceph-mon[16855]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:51.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:51 smithi078 ceph-mon[21464]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:51.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:51 smithi044 ceph-mon[16855]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:51 smithi165 ceph-mon[21975]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:52.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:15:52.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:52 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:52.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:52 smithi165 ceph-mon[21975]: Deploying daemon osd.5 on smithi165 2024-04-20T11:15:52.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:15:52.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:52 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:52.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:52 smithi078 ceph-mon[21464]: Deploying daemon osd.5 on smithi165 2024-04-20T11:15:52.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T11:15:52.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:52 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:15:52.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:52 smithi044 ceph-mon[16855]: Deploying daemon osd.5 on smithi165 2024-04-20T11:15:53.337 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:53 smithi165 ceph-mon[21975]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:53.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:53 smithi078 ceph-mon[21464]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:53.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:53 smithi044 ceph-mon[16855]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:55.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:55 smithi165 ceph-mon[21975]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:55.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:55 smithi078 ceph-mon[21464]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:55.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:55 smithi044 ceph-mon[16855]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:57.415 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:57 smithi165 ceph-mon[21975]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:57.416 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:57 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:57.416 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:57 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:57.416 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:57 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:57.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:57 smithi078 ceph-mon[21464]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:57.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:57 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:57.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:57 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:57.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:57 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:57.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:57 smithi044 ceph-mon[16855]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:57.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:57 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:57.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:57 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:15:57.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:57 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:15:59.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:15:59 smithi165 ceph-mon[21975]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:59.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:15:59 smithi078 ceph-mon[21464]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:15:59.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:15:59 smithi044 ceph-mon[16855]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:00.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.752+0000 7ff8c0ff9700 1 -- 172.21.15.165:0/2831689486 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8c40ff060 con 0x7ff8a40a3c40 2024-04-20T11:16:00.752 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 5 on host 'smithi165' 2024-04-20T11:16:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff8a40a3c40 msgr2=0x7ff8a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff8a40a3c40 0x7ff8a40a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4102a10 tx=0x7ff8bc006d20 comp rx=0 tx=0).stop 2024-04-20T11:16:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 msgr2=0x7ff8c41044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c41044d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40059d0 tx=0x7ff8b400f690 comp rx=0 tx=0).stop 2024-04-20T11:16:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 shutdown_connections 2024-04-20T11:16:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff8a40a3c40 0x7ff8a40a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c4103550 0x7ff8c41044d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c41019b0 0x7ff8c4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 --2- 172.21.15.165:0/2831689486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8c4102bb0 0x7ff8c4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 >> 172.21.15.165:0/2831689486 conn(0x7ff8c40fcf50 msgr2=0x7ff8c40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 shutdown_connections 2024-04-20T11:16:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:16:00.754+0000 7ff8ca009700 1 -- 172.21.15.165:0/2831689486 wait complete. 2024-04-20T11:16:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:00.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:00.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:00.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:00.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:01.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:01.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:01.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:01.417 DEBUG:teuthology.orchestra.run.smithi165:osd.5> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.5.service 2024-04-20T11:16:01.423 INFO:tasks.cephadm:Deploying osd.6 on smithi044 with /dev/vg_nvme/lv_2... 2024-04-20T11:16:01.423 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T11:16:01.888 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:16:02.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:01 smithi078 ceph-mon[21464]: from='osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:16:02.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:01 smithi078 ceph-mon[21464]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:16:02.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:01 smithi044 ceph-mon[16855]: from='osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:16:02.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:01 smithi044 ceph-mon[16855]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:16:02.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:01 smithi165 ceph-mon[21975]: from='osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:16:02.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:01 smithi165 ceph-mon[21975]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T11:16:03.176 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:16:03.176 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T11:16:03.177 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: from='osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:16:03.177 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:03.177 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:16:03.177 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:03.177 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-mon[21975]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:16:03.177 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:16:02.902+0000 7f3fa524f700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T11:16:03.177 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:16:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:16:02.912+0000 7f3fa1042700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:16:03.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:16:03.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T11:16:03.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: from='osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:16:03.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:03.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:16:03.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:03.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:02 smithi078 ceph-mon[21464]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:16:03.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T11:16:03.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T11:16:03.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: from='osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:16:03.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:03.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:16:03.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-20T11:16:03.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:02 smithi044 ceph-mon[16855]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:16:04.013 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] boot 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: osdmap e38: 6 total, 6 up, 6 in 2024-04-20T11:16:04.014 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] boot 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: osdmap e38: 6 total, 6 up, 6 in 2024-04-20T11:16:04.172 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:16:04.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:16:04.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T11:16:04.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: osd.5 [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] boot 2024-04-20T11:16:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: osdmap e38: 6 total, 6 up, 6 in 2024-04-20T11:16:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:16:05.025 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T11:16:05.025 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T11:16:05.025 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-20T11:16:05.025 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-20T11:16:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0352123 s, 298 MB/s 2024-04-20T11:16:05.026 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi165 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-20T11:16:05.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:05.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:05.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.773 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:05 smithi044 ceph-mon[16855]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi165 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:05 smithi078 ceph-mon[21464]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T11:16:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi165 2024-04-20T11:16:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:05 smithi165 ceph-mon[21975]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T11:16:06.071 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-04-20T11:16:06.518 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:16:06.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:06 smithi044 ceph-mon[16855]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T11:16:06.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:06 smithi044 ceph-mon[16855]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:07.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:06 smithi078 ceph-mon[21464]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T11:16:07.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:06 smithi078 ceph-mon[21464]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:07.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:06 smithi165 ceph-mon[21975]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T11:16:07.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:06 smithi165 ceph-mon[21975]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:08.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:07 smithi078 ceph-mon[21464]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T11:16:08.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:07 smithi044 ceph-mon[16855]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T11:16:08.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:07 smithi165 ceph-mon[21975]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T11:16:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 -- 172.21.15.44:0/2470985979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 msgr2=0x7f30b810cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 --2- 172.21.15.44:0/2470985979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b810cd40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b0a0 tx=0x7f30b000b3b0 comp rx=0 tx=0).stop 2024-04-20T11:16:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 -- 172.21.15.44:0/2470985979 shutdown_connections 2024-04-20T11:16:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 --2- 172.21.15.44:0/2470985979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b810cd40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 --2- 172.21.15.44:0/2470985979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30b8076a80 0x7f30b8074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:08.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 --2- 172.21.15.44:0/2470985979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 0x7f30b8075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:08.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 -- 172.21.15.44:0/2470985979 >> 172.21.15.44:0/2470985979 conn(0x7f30b80fdf80 msgr2=0x7f30b81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:08.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 -- 172.21.15.44:0/2470985979 shutdown_connections 2024-04-20T11:16:08.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.918+0000 7f30bd600700 1 -- 172.21.15.44:0/2470985979 wait complete. 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.919+0000 7f30bd600700 1 Processor -- start 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 -- start start 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 0x7f30b819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30b8076a80 0x7f30b819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b81a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8110900 con 0x7f30b80754b0 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8110780 con 0x7f30b810a8b0 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bd600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8110a80 con 0x7f30b8076a80 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30b7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 0x7f30b819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bcc00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b81a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bcc00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b81a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45110/0 (socket says 172.21.15.44:45110) 2024-04-20T11:16:08.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.920+0000 7f30bcc00700 1 -- 172.21.15.44:0/1100515269 learned_addr learned my addr 172.21.15.44:0/1100515269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30b7400700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30b8076a80 0x7f30b819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30bcc00700 1 -- 172.21.15.44:0/1100515269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30b8076a80 msgr2=0x7f30b819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30bcc00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30b8076a80 0x7f30b819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30bcc00700 1 -- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 msgr2=0x7f30b819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30bcc00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 0x7f30b819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30bcc00700 1 -- 172.21.15.44:0/1100515269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b81a2a60 con 0x7f30b810a8b0 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30b7e00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 0x7f30b819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.921+0000 7f30bcc00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b81a23c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b780 tx=0x7f30b0004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.922+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0021070 con 0x7f30b810a8b0 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.922+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f30b0007c80 con 0x7f30b810a8b0 2024-04-20T11:16:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.922+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b001d450 con 0x7f30b810a8b0 2024-04-20T11:16:08.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.922+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b81a2cf0 con 0x7f30b810a8b0 2024-04-20T11:16:08.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.922+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b81a3150 con 0x7f30b810a8b0 2024-04-20T11:16:08.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.923+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b8069740 con 0x7f30b810a8b0 2024-04-20T11:16:08.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.928+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f30b0004000 con 0x7f30b810a8b0 2024-04-20T11:16:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.929+0000 7f30b4c00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f309c0a3cc0 0x7f309c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.930+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f30b00d0630 con 0x7f30b810a8b0 2024-04-20T11:16:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.930+0000 7f30b7e00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f309c0a3cc0 0x7f309c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.931+0000 7f30b7e00700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f309c0a3cc0 0x7f309c0a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f30b8076e60 tx=0x7f30a800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:08.931+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f30b009dec0 con 0x7f30b810a8b0 2024-04-20T11:16:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:09.175+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f30b8062e90 con 0x7f309c0a3cc0 2024-04-20T11:16:09.176 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:08 smithi044 ceph-mon[16855]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T11:16:09.176 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:08 smithi044 ceph-mon[16855]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:09.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:08 smithi078 ceph-mon[21464]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T11:16:09.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:08 smithi078 ceph-mon[21464]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:09.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:08 smithi165 ceph-mon[21975]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T11:16:09.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:08 smithi165 ceph-mon[21975]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:09 smithi078 ceph-mon[21464]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:16:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:16:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:09 smithi044 ceph-mon[16855]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:16:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:16:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:09 smithi165 ceph-mon[21975]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:16:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:16:10.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:10 smithi165 ceph-mon[21975]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-20T11:16:11.259 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:10 smithi044 ceph-mon[16855]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-20T11:16:11.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:10 smithi078 ceph-mon[21464]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-20T11:16:13.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:13 smithi078 ceph-mon[21464]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-20T11:16:13.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:13 smithi044 ceph-mon[16855]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-20T11:16:13.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:13 smithi165 ceph-mon[21975]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-20T11:16:14.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:14 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3266535870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]: dispatch 2024-04-20T11:16:14.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:14 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]: dispatch 2024-04-20T11:16:14.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:14 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]': finished 2024-04-20T11:16:14.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:14 smithi165 ceph-mon[21975]: osdmap e43: 7 total, 6 up, 7 in 2024-04-20T11:16:14.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:14.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:14 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3266535870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]: dispatch 2024-04-20T11:16:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:14 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]: dispatch 2024-04-20T11:16:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:14 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]': finished 2024-04-20T11:16:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:14 smithi078 ceph-mon[21464]: osdmap e43: 7 total, 6 up, 7 in 2024-04-20T11:16:14.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:14.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:14 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3266535870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]: dispatch 2024-04-20T11:16:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:14 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]: dispatch 2024-04-20T11:16:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:14 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4fd9d25-8aa7-45e3-a5da-50ccddec8555"}]': finished 2024-04-20T11:16:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:14 smithi044 ceph-mon[16855]: osdmap e43: 7 total, 6 up, 7 in 2024-04-20T11:16:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:15.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:15 smithi078 ceph-mon[21464]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-20T11:16:15.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:15 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4017600980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:16:15.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:15 smithi044 ceph-mon[16855]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-20T11:16:15.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:15 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4017600980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:16:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:15 smithi165 ceph-mon[21975]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-20T11:16:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:15 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4017600980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:16:17.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:17 smithi078 ceph-mon[21464]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:16:17.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:17 smithi044 ceph-mon[16855]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:16:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:17 smithi165 ceph-mon[21975]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:16:19.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:19 smithi078 ceph-mon[21464]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:16:19.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:19 smithi044 ceph-mon[16855]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:16:19.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:19 smithi165 ceph-mon[21975]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:16:21.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:21 smithi078 ceph-mon[21464]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:21.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:21 smithi044 ceph-mon[16855]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:21.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:21 smithi165 ceph-mon[21975]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:23.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:23 smithi044 ceph-mon[16855]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:23.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:23 smithi078 ceph-mon[21464]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:23.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:23 smithi165 ceph-mon[21975]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:24.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:24 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T11:16:24.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:24 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:24.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:24 smithi078 ceph-mon[21464]: Deploying daemon osd.6 on smithi044 2024-04-20T11:16:24.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T11:16:24.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:24 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:24.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:24 smithi044 ceph-mon[16855]: Deploying daemon osd.6 on smithi044 2024-04-20T11:16:24.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:24 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T11:16:24.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:24 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:24.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:24 smithi165 ceph-mon[21975]: Deploying daemon osd.6 on smithi044 2024-04-20T11:16:25.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:25 smithi044 ceph-mon[16855]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:25.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:25 smithi078 ceph-mon[21464]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:25 smithi165 ceph-mon[21975]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:27.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:27 smithi078 ceph-mon[21464]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:27 smithi044 ceph-mon[16855]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:27 smithi165 ceph-mon[21975]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:29.432 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:29 smithi044 ceph-mon[16855]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:29.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:29 smithi078 ceph-mon[21464]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:29 smithi165 ceph-mon[21975]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:31.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:31 smithi078 ceph-mon[21464]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:31.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:31 smithi044 ceph-mon[16855]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:31 smithi165 ceph-mon[21975]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:33 smithi078 ceph-mon[21464]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:33.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:33 smithi044 ceph-mon[16855]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:33.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:33.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:33.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:33 smithi165 ceph-mon[21975]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:35.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:35 smithi078 ceph-mon[21464]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:35.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:35 smithi044 ceph-mon[16855]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:35 smithi165 ceph-mon[21975]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:37.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:37 smithi078 ceph-mon[21464]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:37.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:37.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:37.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:37 smithi078 ceph-mon[21464]: from='osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T11:16:37.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:37 smithi078 ceph-mon[21464]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T11:16:37.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:37 smithi044 ceph-mon[16855]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:37.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:37 smithi044 ceph-mon[16855]: from='osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T11:16:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:37 smithi044 ceph-mon[16855]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T11:16:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:37 smithi165 ceph-mon[21975]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:37.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:37 smithi165 ceph-mon[21975]: from='osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T11:16:37.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:37 smithi165 ceph-mon[21975]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T11:16:38.395 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 6 on host 'smithi044' 2024-04-20T11:16:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.395+0000 7f30b4c00700 1 -- 172.21.15.44:0/1100515269 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30b8062e90 con 0x7f309c0a3cc0 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.398+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f309c0a3cc0 msgr2=0x7f309c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.398+0000 7f30bd600700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f309c0a3cc0 0x7f309c0a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f30b8076e60 tx=0x7f30a800b5f0 comp rx=0 tx=0).stop 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.398+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 msgr2=0x7f30b81a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.398+0000 7f30bd600700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b81a23c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b780 tx=0x7f30b0004e40 comp rx=0 tx=0).stop 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 shutdown_connections 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f309c0a3cc0 0x7f309c0a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b810a8b0 0x7f30b81a23c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30b80754b0 0x7f30b819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 --2- 172.21.15.44:0/1100515269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30b8076a80 0x7f30b819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 >> 172.21.15.44:0/1100515269 conn(0x7f30b80fdf80 msgr2=0x7f30b81067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 shutdown_connections 2024-04-20T11:16:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:16:38.399+0000 7f30bd600700 1 -- 172.21.15.44:0/1100515269 wait complete. 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: from='osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:39.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: from='osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: from='osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:39.366 DEBUG:teuthology.orchestra.run.smithi044:osd.6> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.6.service 2024-04-20T11:16:39.370 INFO:tasks.cephadm:Deploying osd.7 on smithi078 with /dev/vg_nvme/lv_2... 2024-04-20T11:16:39.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T11:16:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:16:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:39 smithi044 ceph-mon[16855]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:16:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:39 smithi044 ceph-mon[16855]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T11:16:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:39 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:39 smithi044 ceph-mon[16855]: from='osd.6 ' entity='osd.6' 2024-04-20T11:16:40.076 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:39 smithi078 ceph-mon[21464]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:16:40.076 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:39 smithi078 ceph-mon[21464]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T11:16:40.076 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:39 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:40.076 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:39 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:40.077 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:39 smithi078 ceph-mon[21464]: from='osd.6 ' entity='osd.6' 2024-04-20T11:16:40.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:39 smithi165 ceph-mon[21975]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:16:40.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:39 smithi165 ceph-mon[21975]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T11:16:40.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:39 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:40.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:39 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:40.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:39 smithi165 ceph-mon[21975]: from='osd.6 ' entity='osd.6' 2024-04-20T11:16:40.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:16:40.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:16:40.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:40.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:40.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] boot 2024-04-20T11:16:40.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: osdmap e46: 7 total, 7 up, 7 in 2024-04-20T11:16:40.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] boot 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: osdmap e46: 7 total, 7 up, 7 in 2024-04-20T11:16:41.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:16:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:16:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-20T11:16:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: osd.6 [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] boot 2024-04-20T11:16:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: osdmap e46: 7 total, 7 up, 7 in 2024-04-20T11:16:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:16:41.577 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T11:16:41.577 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T11:16:41.577 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-20T11:16:41.578 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-20T11:16:41.578 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01154 s, 909 MB/s 2024-04-20T11:16:41.578 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-20T11:16:42.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-04-20T11:16:42.497 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:16:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:42 smithi078 ceph-mon[21464]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T11:16:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:42 smithi078 ceph-mon[21464]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:43.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:42 smithi044 ceph-mon[16855]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T11:16:43.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:42 smithi044 ceph-mon[16855]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:43.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:42 smithi165 ceph-mon[21975]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T11:16:43.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:42 smithi165 ceph-mon[21975]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.111+0000 7f83239ad700 1 -- 172.21.15.78:0/1599340186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101340 msgr2=0x7f831c103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.111+0000 7f83239ad700 1 --2- 172.21.15.78:0/1599340186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101340 0x7f831c103770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8318009990 tx=0x7f8318009ca0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- 172.21.15.78:0/1599340186 shutdown_connections 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 --2- 172.21.15.78:0/1599340186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831c106680 0x7f831c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 --2- 172.21.15.78:0/1599340186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101340 0x7f831c103770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 --2- 172.21.15.78:0/1599340186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f831c103cb0 0x7f831c106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- 172.21.15.78:0/1599340186 >> 172.21.15.78:0/1599340186 conn(0x7f831c0fcf80 msgr2=0x7f831c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- 172.21.15.78:0/1599340186 shutdown_connections 2024-04-20T11:16:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- 172.21.15.78:0/1599340186 wait complete. 2024-04-20T11:16:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 Processor -- start 2024-04-20T11:16:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- start start 2024-04-20T11:16:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f831c101340 0x7f831c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831c103cb0 0x7f831c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 0x7f831c1a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f831c114dc0 con 0x7f831c101340 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f831c114c40 con 0x7f831c106680 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.112+0000 7f83239ad700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f831c114f40 con 0x7f831c103cb0 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 0x7f831c1a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 0x7f831c1a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45400/0 (socket says 172.21.15.78:45400) 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 -- 172.21.15.78:0/1263043015 learned_addr learned my addr 172.21.15.78:0/1263043015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8320f48700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831c103cb0 0x7f831c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 -- 172.21.15.78:0/1263043015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831c103cb0 msgr2=0x7f831c19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831c103cb0 0x7f831c19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 -- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f831c101340 msgr2=0x7f831c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:16:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f831c101340 0x7f831c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 -- 172.21.15.78:0/1263043015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1a7000 con 0x7f831c106680 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f8321f4a700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 0x7f831c1a6960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8314007d70 tx=0x7f831400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314016020 con 0x7f831c106680 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f831400ad40 con 0x7f831c106680 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1a72f0 con 0x7f831c106680 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140116b0 con 0x7f831c106680 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.113+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a7830 con 0x7f831c106680 2024-04-20T11:16:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.114+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c069740 con 0x7f831c106680 2024-04-20T11:16:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.115+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8314019020 con 0x7f831c106680 2024-04-20T11:16:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.116+0000 7f83127fc700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f83080a3c40 0x7f83080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:16:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.116+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f83140cf620 con 0x7f831c106680 2024-04-20T11:16:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.116+0000 7f8321749700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f83080a3c40 0x7f83080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:16:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.117+0000 7f8321749700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f83080a3c40 0x7f83080a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8318003ad0 tx=0x7f8318005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:16:44.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.120+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8314020020 con 0x7f831c106680 2024-04-20T11:16:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:16:44.327+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f831c062e90 con 0x7f83080a3c40 2024-04-20T11:16:45.276 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:45.276 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:45.276 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi044 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi044 to 5698M 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:45.277 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:45 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi044 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi044 to 5698M 2024-04-20T11:16:45.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:45.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:45.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:45 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:16:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi044 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi044 to 5698M 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:45.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:45 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:46.217 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:46.217 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:46.217 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:46.217 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:46.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:46.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:46.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:46.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:46.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:16:46.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:46.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:16:46.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:16:47.435 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:47 smithi078 ceph-mon[21464]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:47.435 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:47 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/3713575175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]: dispatch 2024-04-20T11:16:47.436 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:47 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]: dispatch 2024-04-20T11:16:47.436 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:47 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]': finished 2024-04-20T11:16:47.436 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:47 smithi078 ceph-mon[21464]: osdmap e48: 8 total, 7 up, 8 in 2024-04-20T11:16:47.436 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:47 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:16:47.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:47 smithi044 ceph-mon[16855]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:47.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:47 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/3713575175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]: dispatch 2024-04-20T11:16:47.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:47 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]: dispatch 2024-04-20T11:16:47.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:47 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]': finished 2024-04-20T11:16:47.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:47 smithi044 ceph-mon[16855]: osdmap e48: 8 total, 7 up, 8 in 2024-04-20T11:16:47.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:47 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:16:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:47 smithi165 ceph-mon[21975]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:47 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/3713575175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]: dispatch 2024-04-20T11:16:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:47 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]: dispatch 2024-04-20T11:16:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:47 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29b9c0d1-e314-4337-b6cd-0a7c7514f8dc"}]': finished 2024-04-20T11:16:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:47 smithi165 ceph-mon[21975]: osdmap e48: 8 total, 7 up, 8 in 2024-04-20T11:16:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:47 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:16:48.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:48 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/2220417511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:16:48.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:48 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/2220417511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:16:48.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:48 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/2220417511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:16:49.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:49 smithi078 ceph-mon[21464]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:49.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:49 smithi044 ceph-mon[16855]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:49.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:49 smithi165 ceph-mon[21975]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:51.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:51 smithi078 ceph-mon[21464]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:51.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:51 smithi044 ceph-mon[16855]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:51 smithi165 ceph-mon[21975]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:53.364 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:53 smithi078 ceph-mon[21464]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:53.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:53 smithi044 ceph-mon[16855]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:53.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:53 smithi165 ceph-mon[21975]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:55.434 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:55 smithi078 ceph-mon[21464]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:55.434 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T11:16:55.435 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:55.435 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:55 smithi078 ceph-mon[21464]: Deploying daemon osd.7 on smithi078 2024-04-20T11:16:55.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:55 smithi044 ceph-mon[16855]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:55.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T11:16:55.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:55.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:55 smithi044 ceph-mon[16855]: Deploying daemon osd.7 on smithi078 2024-04-20T11:16:55.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:55 smithi165 ceph-mon[21975]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:55.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T11:16:55.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:16:55.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:55 smithi165 ceph-mon[21975]: Deploying daemon osd.7 on smithi078 2024-04-20T11:16:57.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:57 smithi078 ceph-mon[21464]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:57.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:57 smithi044 ceph-mon[16855]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:57.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:57 smithi165 ceph-mon[21975]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:59.411 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:16:59 smithi078 ceph-mon[21464]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:59.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:16:59 smithi044 ceph-mon[16855]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:16:59.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:16:59 smithi165 ceph-mon[21975]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:01 smithi078 ceph-mon[21464]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:01 smithi044 ceph-mon[16855]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:01 smithi165 ceph-mon[21975]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:01.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:03.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:03 smithi078 ceph-mon[21464]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:03.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:03 smithi044 ceph-mon[16855]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:03 smithi165 ceph-mon[21975]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:04.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.271+0000 7f83127fc700 1 -- 172.21.15.78:0/1263043015 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f831c062e90 con 0x7f83080a3c40 2024-04-20T11:17:04.272 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 7 on host 'smithi078' 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f83080a3c40 msgr2=0x7f83080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f83080a3c40 0x7f83080a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8318003ad0 tx=0x7f8318005fb0 comp rx=0 tx=0).stop 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 msgr2=0x7f831c1a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 0x7f831c1a6960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8314007d70 tx=0x7f831400a340 comp rx=0 tx=0).stop 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 shutdown_connections 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f83080a3c40 0x7f83080a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:04.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c106680 0x7f831c1a6960 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:04.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f831c101340 0x7f831c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:04.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 --2- 172.21.15.78:0/1263043015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831c103cb0 0x7f831c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:04.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 >> 172.21.15.78:0/1263043015 conn(0x7f831c0fcf80 msgr2=0x7f831c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:04.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 shutdown_connections 2024-04-20T11:17:04.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:17:04.273+0000 7f83239ad700 1 -- 172.21.15.78:0/1263043015 wait complete. 2024-04-20T11:17:04.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T11:17:04.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T11:17:04.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:04.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:04.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:04.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T11:17:04.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T11:17:04.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:04.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:04.973 DEBUG:teuthology.orchestra.run.smithi078:osd.7> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.7.service 2024-04-20T11:17:04.976 INFO:tasks.cephadm:Deploying osd.8 on smithi165 with /dev/vg_nvme/lv_2... 2024-04-20T11:17:04.976 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T11:17:05.235 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:17:05.541 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-mon[21464]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:05.541 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-mon[21464]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T11:17:05.541 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-mon[21464]: from='osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:17:05.541 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-mon[21464]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T11:17:05.542 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:05.542 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-mon[21464]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:17:05.542 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:17:05.443+0000 7fbf7ce46700 -1 osd.7 0 waiting for initial osdmap 2024-04-20T11:17:05.542 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:17:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:17:05.455+0000 7fbf78c39700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:17:05.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:05 smithi165 ceph-mon[21975]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:05.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:05 smithi165 ceph-mon[21975]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T11:17:05.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:05 smithi165 ceph-mon[21975]: from='osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:17:05.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:05 smithi165 ceph-mon[21975]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T11:17:05.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:05.610 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:05 smithi165 ceph-mon[21975]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:17:05.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:05 smithi044 ceph-mon[16855]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:05.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:05 smithi044 ceph-mon[16855]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T11:17:05.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:05 smithi044 ceph-mon[16855]: from='osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:17:05.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:05 smithi044 ceph-mon[16855]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T11:17:05.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:05.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:05 smithi044 ceph-mon[16855]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:17:06.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:06 smithi078 ceph-mon[21464]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:17:06.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:06 smithi078 ceph-mon[21464]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T11:17:06.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:06.526 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:06.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:06 smithi165 ceph-mon[21975]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:17:06.675 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:06 smithi165 ceph-mon[21975]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T11:17:06.675 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:06.675 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:06 smithi044 ceph-mon[16855]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:17:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:06 smithi044 ceph-mon[16855]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T11:17:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:07.244 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T11:17:07.244 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T11:17:07.244 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-20T11:17:07.244 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-20T11:17:07.244 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117689 s, 891 MB/s 2024-04-20T11:17:07.244 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] boot 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: osdmap e51: 8 total, 8 up, 8 in 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:07.544 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:07 smithi165 ceph-mon[21975]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T11:17:07.635 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:17:07.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:17:07.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:07.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] boot 2024-04-20T11:17:07.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: osdmap e51: 8 total, 8 up, 8 in 2024-04-20T11:17:07.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:07.636 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:07 smithi078 ceph-mon[21464]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: osd.7 [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] boot 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: osdmap e51: 8 total, 8 up, 8 in 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:17:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:07 smithi044 ceph-mon[16855]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T11:17:07.891 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-04-20T11:17:08.140 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:17:09.309 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:09.309 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi078 2024-04-20T11:17:09.309 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.309 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.309 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.309 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.310 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:09 smithi078 ceph-mon[21464]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T11:17:09.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:09.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi078 2024-04-20T11:17:09.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.320 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:09 smithi165 ceph-mon[21975]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T11:17:09.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:09.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi078 2024-04-20T11:17:09.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:09.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:09 smithi044 ceph-mon[16855]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 -- 172.21.15.165:0/1795172164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 msgr2=0x7fae10103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 --2- 172.21.15.165:0/1795172164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10103010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fadf80098f0 tx=0x7fadf8009c00 comp rx=0 tx=0).stop 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 -- 172.21.15.165:0/1795172164 shutdown_connections 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 --2- 172.21.15.165:0/1795172164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae10103550 0x7fae10109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 --2- 172.21.15.165:0/1795172164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae101019b0 0x7fae10101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 --2- 172.21.15.165:0/1795172164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10103010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 -- 172.21.15.165:0/1795172164 >> 172.21.15.165:0/1795172164 conn(0x7fae100fcf50 msgr2=0x7fae100ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 -- 172.21.15.165:0/1795172164 shutdown_connections 2024-04-20T11:17:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.877+0000 7fae1584d700 1 -- 172.21.15.165:0/1795172164 wait complete. 2024-04-20T11:17:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 Processor -- start 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 -- start start 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae101019b0 0x7fae10198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae10103550 0x7fae101a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae1010d750 con 0x7fae10103550 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae1010d5d0 con 0x7fae101019b0 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae1584d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae1010d8d0 con 0x7fae10102bb0 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae101019b0 0x7fae10198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:34096/0 (socket says 172.21.15.165:34096) 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 -- 172.21.15.165:0/1806341052 learned_addr learned my addr 172.21.15.165:0/1806341052 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 -- 172.21.15.165:0/1806341052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae101019b0 msgr2=0x7fae10198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae101019b0 0x7fae10198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 -- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae10103550 msgr2=0x7fae101a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.878+0000 7fae0e7fc700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae10103550 0x7fae101a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae0e7fc700 1 -- 172.21.15.165:0/1806341052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae101a0b00 con 0x7fae10102bb0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae0e7fc700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10199430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadf80059d0 tx=0x7fadf800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf800fe60 con 0x7fae10102bb0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadf800ba60 con 0x7fae10102bb0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae101a0d90 con 0x7fae10102bb0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8017400 con 0x7fae10102bb0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.879+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae101a1210 con 0x7fae10102bb0 2024-04-20T11:17:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.880+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae101a14a0 con 0x7fae10102bb0 2024-04-20T11:17:09.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.881+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fadf8027060 con 0x7fae10102bb0 2024-04-20T11:17:09.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.882+0000 7fae1484b700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadfc0a3cc0 0x7fadfc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:09.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.882+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7fadf80d02d0 con 0x7fae10102bb0 2024-04-20T11:17:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.882+0000 7fae0effd700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadfc0a3cc0 0x7fadfc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.883+0000 7fae0effd700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadfc0a3cc0 0x7fadfc0a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae10102a10 tx=0x7fae0000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:09.886+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fadf8013030 con 0x7fae10102bb0 2024-04-20T11:17:10.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:10.093+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fae100ff140 con 0x7fadfc0a3cc0 2024-04-20T11:17:10.736 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:10 smithi165 ceph-mon[21975]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T11:17:10.736 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:17:10.736 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:17:10.737 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:10 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:10.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:10 smithi078 ceph-mon[21464]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T11:17:10.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:17:10.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:17:10.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:10 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:10.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:10 smithi044 ceph-mon[16855]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T11:17:10.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:17:10.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:17:10.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:10 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:11 smithi165 ceph-mon[21975]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:11 smithi165 ceph-mon[21975]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:11 smithi165 ceph-mon[21975]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T11:17:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:11 smithi078 ceph-mon[21464]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:11 smithi078 ceph-mon[21464]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:11 smithi078 ceph-mon[21464]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T11:17:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:11 smithi044 ceph-mon[16855]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:11 smithi044 ceph-mon[16855]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:11 smithi044 ceph-mon[16855]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T11:17:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/4191072201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]: dispatch 2024-04-20T11:17:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]: dispatch 2024-04-20T11:17:13.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]': finished 2024-04-20T11:17:13.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: osdmap e56: 9 total, 8 up, 9 in 2024-04-20T11:17:13.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:13.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:13 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/2237346939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/4191072201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]: dispatch 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]: dispatch 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]': finished 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: osdmap e56: 9 total, 8 up, 9 in 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:13 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/2237346939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:17:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/4191072201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]: dispatch 2024-04-20T11:17:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]: dispatch 2024-04-20T11:17:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb"}]': finished 2024-04-20T11:17:13.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: osdmap e56: 9 total, 8 up, 9 in 2024-04-20T11:17:13.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:13.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:13 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/2237346939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:17:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:15 smithi078 ceph-mon[21464]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:15 smithi044 ceph-mon[16855]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:15 smithi165 ceph-mon[21975]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:17.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:17 smithi044 ceph-mon[16855]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:17.780 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:17 smithi165 ceph-mon[21975]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:17 smithi078 ceph-mon[21464]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:19.548 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:19 smithi165 ceph-mon[21975]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:19 smithi078 ceph-mon[21464]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:19 smithi044 ceph-mon[16855]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:20.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T11:17:20.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:20.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T11:17:20.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:20.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T11:17:20.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:21.854 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:21 smithi165 ceph-mon[21975]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:21.854 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:21 smithi165 ceph-mon[21975]: Deploying daemon osd.8 on smithi165 2024-04-20T11:17:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:21 smithi078 ceph-mon[21464]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:21 smithi078 ceph-mon[21464]: Deploying daemon osd.8 on smithi165 2024-04-20T11:17:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:21 smithi044 ceph-mon[16855]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:21 smithi044 ceph-mon[16855]: Deploying daemon osd.8 on smithi165 2024-04-20T11:17:23.711 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:23 smithi165 ceph-mon[21975]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:23 smithi078 ceph-mon[21464]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:23.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:23 smithi044 ceph-mon[16855]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:25.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:25 smithi165 ceph-mon[21975]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:25 smithi078 ceph-mon[21464]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:25.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:25 smithi044 ceph-mon[16855]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:27.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:26 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:27.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:26 smithi078 ceph-mon[21464]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:27.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:26 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:27.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:26 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:27.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:26 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:26 smithi044 ceph-mon[16855]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:26 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:26 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:26 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:26 smithi165 ceph-mon[21975]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:26 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:26 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:29.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:29 smithi078 ceph-mon[21464]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:29.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:29 smithi044 ceph-mon[16855]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:29 smithi165 ceph-mon[21975]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.888+0000 7fae1484b700 1 -- 172.21.15.165:0/1806341052 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae100ff140 con 0x7fadfc0a3cc0 2024-04-20T11:17:29.888 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 8 on host 'smithi165' 2024-04-20T11:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadfc0a3cc0 msgr2=0x7fadfc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadfc0a3cc0 0x7fadfc0a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae10102a10 tx=0x7fae0000d040 comp rx=0 tx=0).stop 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 msgr2=0x7fae10199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10199430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadf80059d0 tx=0x7fadf800f690 comp rx=0 tx=0).stop 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 shutdown_connections 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadfc0a3cc0 0x7fadfc0a6180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae10103550 0x7fae101a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae101019b0 0x7fae10198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 --2- 172.21.15.165:0/1806341052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae10102bb0 0x7fae10199430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 >> 172.21.15.165:0/1806341052 conn(0x7fae100fcf50 msgr2=0x7fae100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 shutdown_connections 2024-04-20T11:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:17:29.890+0000 7fae1584d700 1 -- 172.21.15.165:0/1806341052 wait complete. 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T11:17:30.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:30 smithi078 ceph-mon[21464]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T11:17:30.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:30 smithi044 ceph-mon[16855]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:30.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:17:30.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T11:17:30.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:30 smithi165 ceph-mon[21975]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T11:17:30.562 DEBUG:teuthology.orchestra.run.smithi165:osd.8> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.8.service 2024-04-20T11:17:30.564 INFO:tasks.cephadm:Deploying osd.9 on smithi044 with /dev/vg_nvme/lv_1... 2024-04-20T11:17:30.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T11:17:31.030 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:17:31.354 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:31 smithi044 ceph-mon[16855]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:31.355 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:31 smithi044 ceph-mon[16855]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T11:17:31.355 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:31 smithi044 ceph-mon[16855]: from='osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:17:31.355 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:31 smithi044 ceph-mon[16855]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T11:17:31.355 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:31.355 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:31 smithi044 ceph-mon[16855]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:17:31.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:31 smithi078 ceph-mon[21464]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:31.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:31 smithi078 ceph-mon[21464]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T11:17:31.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:31 smithi078 ceph-mon[21464]: from='osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:17:31.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:31 smithi078 ceph-mon[21464]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T11:17:31.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:31.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:31 smithi078 ceph-mon[21464]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:17:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-mon[21975]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-mon[21975]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T11:17:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-mon[21975]: from='osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:17:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-mon[21975]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T11:17:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-mon[21975]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:17:32.125 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:17:31.850+0000 7f531a71a700 -1 osd.8 0 waiting for initial osdmap 2024-04-20T11:17:32.125 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:17:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:17:31.861+0000 7f531650d700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:17:33.107 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:17:33.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T11:17:33.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:33.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:33.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] boot 2024-04-20T11:17:33.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: osdmap e59: 9 total, 9 up, 9 in 2024-04-20T11:17:33.108 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:33.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:17:33.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T11:17:33.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:33.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:33.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] boot 2024-04-20T11:17:33.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: osdmap e59: 9 total, 9 up, 9 in 2024-04-20T11:17:33.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:33.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:17:33.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T11:17:33.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:33.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T11:17:33.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: osd.8 [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] boot 2024-04-20T11:17:33.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: osdmap e59: 9 total, 9 up, 9 in 2024-04-20T11:17:33.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:17:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:17:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:17:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi165 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:33.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:33 smithi165 ceph-mon[21975]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi165 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:34.189 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:34.189 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.189 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:33 smithi078 ceph-mon[21464]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T11:17:34.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi165 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:17:34.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:33 smithi044 ceph-mon[16855]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T11:17:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T11:17:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T11:17:34.217 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-20T11:17:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-20T11:17:34.218 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0375283 s, 279 MB/s 2024-04-20T11:17:34.218 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-20T11:17:35.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-04-20T11:17:35.655 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:17:35.999 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:35 smithi044 ceph-mon[16855]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:36.000 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:35 smithi044 ceph-mon[16855]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T11:17:36.151 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:35 smithi165 ceph-mon[21975]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:36.151 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:35 smithi165 ceph-mon[21975]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T11:17:36.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:35 smithi078 ceph-mon[21464]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:36.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:35 smithi078 ceph-mon[21464]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T11:17:37.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:36 smithi078 ceph-mon[21464]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T11:17:37.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:36 smithi078 ceph-mon[21464]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:37.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:36 smithi044 ceph-mon[16855]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T11:17:37.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:36 smithi044 ceph-mon[16855]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:37.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:36 smithi165 ceph-mon[21975]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T11:17:37.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:36 smithi165 ceph-mon[21975]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:37.920 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:37 smithi044 ceph-mon[16855]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T11:17:38.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:37 smithi078 ceph-mon[21464]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T11:17:38.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:37 smithi165 ceph-mon[21975]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T11:17:38.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 -- 172.21.15.44:0/2529031083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 msgr2=0x7fb27c111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:38.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 --2- 172.21.15.44:0/2529031083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 0x7fb27c111540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb27400b0a0 tx=0x7fb27400b3b0 comp rx=0 tx=0).stop 2024-04-20T11:17:38.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 -- 172.21.15.44:0/2529031083 shutdown_connections 2024-04-20T11:17:38.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 --2- 172.21.15.44:0/2529031083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 0x7fb27c111540 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:38.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 --2- 172.21.15.44:0/2529031083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c1090f0 0x7fb27c109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:38.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 --2- 172.21.15.44:0/2529031083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c10a2f0 0x7fb27c10a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:38.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 -- 172.21.15.44:0/2529031083 >> 172.21.15.44:0/2529031083 conn(0x7fb27c0780c0 msgr2=0x7fb27c07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:17:38.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 -- 172.21.15.44:0/2529031083 shutdown_connections 2024-04-20T11:17:38.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.309+0000 7fb283000700 1 -- 172.21.15.44:0/2529031083 wait complete. 2024-04-20T11:17:38.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.310+0000 7fb283000700 1 Processor -- start 2024-04-20T11:17:38.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 -- start start 2024-04-20T11:17:38.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1090f0 0x7fb27c19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:38.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 0x7fb27c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 0x7fb27c1a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c114e40 con 0x7fb27c1090f0 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c114cc0 con 0x7fb27c10a2f0 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb283000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c114fc0 con 0x7fb27c10ac90 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.311+0000 7fb27be00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1090f0 0x7fb27c19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 0x7fb27c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 0x7fb27c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58300/0 (socket says 172.21.15.44:58300) 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 -- 172.21.15.44:0/1718541358 learned_addr learned my addr 172.21.15.44:0/1718541358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb281000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 0x7fb27c1a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 -- 172.21.15.44:0/1718541358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 msgr2=0x7fb27c1a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 0x7fb27c1a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 -- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1090f0 msgr2=0x7fb27c19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:17:38.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1090f0 0x7fb27c19f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:17:38.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 -- 172.21.15.44:0/1718541358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c1a6f80 con 0x7fb27c10a2f0 2024-04-20T11:17:38.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.312+0000 7fb27b400700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 0x7fb27c19f860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9a0 tx=0x7fb26800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:38.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.313+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800ca10 con 0x7fb27c10a2f0 2024-04-20T11:17:38.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.313+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb26800cbb0 con 0x7fb27c10a2f0 2024-04-20T11:17:38.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.313+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c1a7270 con 0x7fb27c10a2f0 2024-04-20T11:17:38.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.313+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2680187a0 con 0x7fb27c10a2f0 2024-04-20T11:17:38.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.313+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c1a7800 con 0x7fb27c10a2f0 2024-04-20T11:17:38.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.314+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c069740 con 0x7fb27c10a2f0 2024-04-20T11:17:38.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.316+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb268010430 con 0x7fb27c10a2f0 2024-04-20T11:17:38.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.318+0000 7fb278c00700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb2600a3cc0 0x7fb2600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:17:38.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.318+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fb2680cfbd0 con 0x7fb27c10a2f0 2024-04-20T11:17:38.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.318+0000 7fb27be00700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb2600a3cc0 0x7fb2600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:17:38.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.319+0000 7fb27be00700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb2600a3cc0 0x7fb2600a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb27c10a150 tx=0x7fb26c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:17:38.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.322+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb26809cec0 con 0x7fb27c10a2f0 2024-04-20T11:17:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:17:38.554+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb27c062e90 con 0x7fb2600a3cc0 2024-04-20T11:17:39.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:38 smithi078 ceph-mon[21464]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-20T11:17:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:38 smithi078 ceph-mon[21464]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:17:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:17:39.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:39.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:38 smithi044 ceph-mon[16855]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-20T11:17:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:38 smithi044 ceph-mon[16855]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:17:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:17:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:38 smithi165 ceph-mon[21975]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-04-20T11:17:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:38 smithi165 ceph-mon[21975]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:17:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:17:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:17:39.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:41.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:41 smithi044 ceph-mon[16855]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:17:41.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:41 smithi078 ceph-mon[21464]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:17:41.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:41 smithi165 ceph-mon[21975]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T11:17:43.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:43 smithi078 ceph-mon[21464]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-20T11:17:43.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/522207495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e0db795-eae2-41b0-b0f6-38e6fc4b56b9"}]: dispatch 2024-04-20T11:17:43.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/522207495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e0db795-eae2-41b0-b0f6-38e6fc4b56b9"}]': finished 2024-04-20T11:17:43.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:43 smithi078 ceph-mon[21464]: osdmap e64: 10 total, 9 up, 10 in 2024-04-20T11:17:43.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:17:43.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:43 smithi044 ceph-mon[16855]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-20T11:17:43.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/522207495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e0db795-eae2-41b0-b0f6-38e6fc4b56b9"}]: dispatch 2024-04-20T11:17:43.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/522207495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e0db795-eae2-41b0-b0f6-38e6fc4b56b9"}]': finished 2024-04-20T11:17:43.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:43 smithi044 ceph-mon[16855]: osdmap e64: 10 total, 9 up, 10 in 2024-04-20T11:17:43.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:17:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:43 smithi165 ceph-mon[21975]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-20T11:17:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/522207495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e0db795-eae2-41b0-b0f6-38e6fc4b56b9"}]: dispatch 2024-04-20T11:17:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/522207495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e0db795-eae2-41b0-b0f6-38e6fc4b56b9"}]': finished 2024-04-20T11:17:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:43 smithi165 ceph-mon[21975]: osdmap e64: 10 total, 9 up, 10 in 2024-04-20T11:17:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:17:44.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1795450595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:17:44.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1795450595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:17:44.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1795450595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:17:45.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:45 smithi078 ceph-mon[21464]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:17:45.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:45 smithi044 ceph-mon[16855]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:17:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:45 smithi165 ceph-mon[21975]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:17:47.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:47 smithi078 ceph-mon[21464]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-20T11:17:47.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:47 smithi044 ceph-mon[16855]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-20T11:17:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:47 smithi165 ceph-mon[21975]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-20T11:17:49.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:49 smithi078 ceph-mon[21464]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:49.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:49 smithi044 ceph-mon[16855]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:49.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:49 smithi165 ceph-mon[21975]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:51.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:51 smithi078 ceph-mon[21464]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:51.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:51 smithi044 ceph-mon[16855]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:51 smithi165 ceph-mon[21975]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:53.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:53 smithi078 ceph-mon[21464]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:53.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:53 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T11:17:53.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:53 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:53.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:53 smithi078 ceph-mon[21464]: Deploying daemon osd.9 on smithi044 2024-04-20T11:17:53.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:53 smithi044 ceph-mon[16855]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:53.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:53 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T11:17:53.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:53 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:53.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:53 smithi044 ceph-mon[16855]: Deploying daemon osd.9 on smithi044 2024-04-20T11:17:53.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:53 smithi165 ceph-mon[21975]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:53.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:53 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T11:17:53.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:53 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:17:53.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:53 smithi165 ceph-mon[21975]: Deploying daemon osd.9 on smithi044 2024-04-20T11:17:55.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:55 smithi044 ceph-mon[16855]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:55.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:55 smithi078 ceph-mon[21464]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:55.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:55 smithi165 ceph-mon[21975]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:57.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:57 smithi044 ceph-mon[16855]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:57.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:57 smithi078 ceph-mon[21464]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:57.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:57 smithi165 ceph-mon[21975]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:59.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:17:59 smithi078 ceph-mon[21464]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:59.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:17:59 smithi044 ceph-mon[16855]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:17:59.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:17:59 smithi165 ceph-mon[21975]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:01.291 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:01 smithi044 ceph-mon[16855]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:01.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:01 smithi078 ceph-mon[21464]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:01.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:01 smithi165 ceph-mon[21975]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:02.896 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:02.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:02.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:02.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:02 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:02.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:02 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:02.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:02 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:03.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:03 smithi165 ceph-mon[21975]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:03 smithi078 ceph-mon[21464]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:03.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:03 smithi044 ceph-mon[16855]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:05.616 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:05 smithi044 ceph-mon[16855]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:05.617 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:05 smithi044 ceph-mon[16855]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T11:18:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:05 smithi078 ceph-mon[21464]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:05 smithi078 ceph-mon[21464]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T11:18:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:05 smithi165 ceph-mon[21975]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:05 smithi165 ceph-mon[21975]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T11:18:06.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T11:18:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T11:18:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:18:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:06.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:06 smithi044 ceph-mon[16855]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:18:06.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T11:18:06.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T11:18:06.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:18:06.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:06.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:06.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:06.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:06 smithi078 ceph-mon[21464]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:18:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T11:18:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T11:18:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-20T11:18:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:06.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:06.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:06 smithi165 ceph-mon[21975]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-20T11:18:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.317+0000 7fb278c00700 1 -- 172.21.15.44:0/1718541358 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb27c062e90 con 0x7fb2600a3cc0 2024-04-20T11:18:07.318 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 9 on host 'smithi044' 2024-04-20T11:18:07.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb2600a3cc0 msgr2=0x7fb2600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb2600a3cc0 0x7fb2600a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb27c10a150 tx=0x7fb26c009260 comp rx=0 tx=0).stop 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 msgr2=0x7fb27c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 0x7fb27c19f860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9a0 tx=0x7fb26800ecb0 comp rx=0 tx=0).stop 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 shutdown_connections 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb2600a3cc0 0x7fb2600a6180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c10ac90 0x7fb27c1a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1090f0 0x7fb27c19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 --2- 172.21.15.44:0/1718541358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb27c10a2f0 0x7fb27c19f860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 >> 172.21.15.44:0/1718541358 conn(0x7fb27c0780c0 msgr2=0x7fb27c10d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 shutdown_connections 2024-04-20T11:18:07.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:18:07.321+0000 7fb283000700 1 -- 172.21.15.44:0/1718541358 wait complete. 2024-04-20T11:18:07.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:07 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:07 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T11:18:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466]' entity='osd.9' 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:07 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:08.644 DEBUG:teuthology.orchestra.run.smithi044:osd.9> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.9.service 2024-04-20T11:18:08.649 INFO:tasks.cephadm:Deploying osd.10 on smithi078 with /dev/vg_nvme/lv_1... 2024-04-20T11:18:08.649 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T11:18:08.899 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:18:09.578 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:09 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:18:09.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:09 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:18:09.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:09 smithi078 ceph-mon[21464]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:09.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:09 smithi078 ceph-mon[21464]: osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] boot 2024-04-20T11:18:09.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:09 smithi078 ceph-mon[21464]: osdmap e67: 10 total, 10 up, 10 in 2024-04-20T11:18:09.579 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:09 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:09 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:18:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:09 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:18:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:09 smithi044 ceph-mon[16855]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:09 smithi044 ceph-mon[16855]: osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] boot 2024-04-20T11:18:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:09 smithi044 ceph-mon[16855]: osdmap e67: 10 total, 10 up, 10 in 2024-04-20T11:18:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:09 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:09 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:18:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:09 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:18:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:09 smithi165 ceph-mon[21975]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-20T11:18:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:09 smithi165 ceph-mon[21975]: osd.9 [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] boot 2024-04-20T11:18:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:09 smithi165 ceph-mon[21975]: osdmap e67: 10 total, 10 up, 10 in 2024-04-20T11:18:09.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:09 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:18:10.391 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:10 smithi078 ceph-mon[21464]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T11:18:10.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:10 smithi044 ceph-mon[16855]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T11:18:10.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:10 smithi165 ceph-mon[21975]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T11:18:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T11:18:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T11:18:11.249 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-20T11:18:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-20T11:18:11.250 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115002 s, 912 MB/s 2024-04-20T11:18:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-20T11:18:11.554 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:11 smithi078 ceph-mon[21464]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:11.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:11 smithi044 ceph-mon[16855]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:11 smithi165 ceph-mon[21975]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:11.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-04-20T11:18:12.194 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:18:13.553 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:13 smithi078 ceph-mon[21464]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:13.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:13 smithi044 ceph-mon[16855]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:13 smithi165 ceph-mon[21975]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 -- 172.21.15.78:0/790562623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194105e60 msgr2=0x7fb194108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 --2- 172.21.15.78:0/790562623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194105e60 0x7fb194108250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb188009990 tx=0x7fb188009ca0 comp rx=0 tx=0).stop 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 -- 172.21.15.78:0/790562623 shutdown_connections 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 --2- 172.21.15.78:0/790562623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb194108790 0x7fb19410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 --2- 172.21.15.78:0/790562623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb194103530 0x7fb194105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 --2- 172.21.15.78:0/790562623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194105e60 0x7fb194108250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 -- 172.21.15.78:0/790562623 >> 172.21.15.78:0/790562623 conn(0x7fb1940fcf40 msgr2=0x7fb1940ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 -- 172.21.15.78:0/790562623 shutdown_connections 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 -- 172.21.15.78:0/790562623 wait complete. 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 Processor -- start 2024-04-20T11:18:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.778+0000 7fb19a337700 1 -- start start 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb194103530 0x7fb194198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb194105e60 0x7fb194199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 0x7fb1941a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb19410e850 con 0x7fb194103530 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb19410e6d0 con 0x7fb194108790 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb19410e9d0 con 0x7fb194105e60 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 0x7fb1941a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 0x7fb1941a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37996/0 (socket says 172.21.15.78:37996) 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 -- 172.21.15.78:0/4056147961 learned_addr learned my addr 172.21.15.78:0/4056147961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1937fe700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb194105e60 0x7fb194199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 -- 172.21.15.78:0/4056147961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb194105e60 msgr2=0x7fb194199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb194105e60 0x7fb194199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 -- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb194103530 msgr2=0x7fb194198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb193fff700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb194103530 0x7fb194198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb194103530 0x7fb194198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 -- 172.21.15.78:0/4056147961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1941a0a40 con 0x7fb194108790 2024-04-20T11:18:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1988d4700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 0x7fb1941a03a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0094f0 tx=0x7fb18c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00ddd0 con 0x7fb194108790 2024-04-20T11:18:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.779+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1941a0d30 con 0x7fb194108790 2024-04-20T11:18:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.780+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb18c009e20 con 0x7fb194108790 2024-04-20T11:18:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.780+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1941a1260 con 0x7fb194108790 2024-04-20T11:18:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.780+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c015660 con 0x7fb194108790 2024-04-20T11:18:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.781+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb194069740 con 0x7fb194108790 2024-04-20T11:18:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.782+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb18c004ad0 con 0x7fb194108790 2024-04-20T11:18:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.782+0000 7fb1917fa700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb17c0a3c40 0x7fb17c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.782+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fb18c0cf630 con 0x7fb194108790 2024-04-20T11:18:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.783+0000 7fb193fff700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb17c0a3c40 0x7fb17c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.784+0000 7fb193fff700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb17c0a3c40 0x7fb17c0a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb184006fd0 tx=0x7fb184008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:13.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.787+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb18c010070 con 0x7fb194108790 2024-04-20T11:18:13.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:13.996+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb194062e90 con 0x7fb17c0a3c40 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi044 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.888 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:18:14.889 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:14 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:14.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi044 2024-04-20T11:18:14.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:18:14.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:14 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:15.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi044 2024-04-20T11:18:15.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:15.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:15.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:15.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:15.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:15.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:18:15.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:18:15.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:14 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:15.941 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:15 smithi078 ceph-mon[21464]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:15.942 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:15 smithi078 ceph-mon[21464]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:15.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:15 smithi165 ceph-mon[21975]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:15.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:15 smithi165 ceph-mon[21975]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:16.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:15 smithi044 ceph-mon[16855]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:16.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:15 smithi044 ceph-mon[16855]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:18.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:18.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/348687685' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]: dispatch 2024-04-20T11:18:18.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]: dispatch 2024-04-20T11:18:18.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]': finished 2024-04-20T11:18:18.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: osdmap e69: 11 total, 10 up, 11 in 2024-04-20T11:18:18.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:18.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.78:0/3747945557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/348687685' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]: dispatch 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]: dispatch 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]': finished 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: osdmap e69: 11 total, 10 up, 11 in 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.78:0/3747945557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:18:18.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:18.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/348687685' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]: dispatch 2024-04-20T11:18:18.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]: dispatch 2024-04-20T11:18:18.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35e05c9e-6f8c-4dce-ad79-fa9597cb0743"}]': finished 2024-04-20T11:18:18.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: osdmap e69: 11 total, 10 up, 11 in 2024-04-20T11:18:18.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:18.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.78:0/3747945557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:18:19.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:18 smithi078 ceph-mon[21464]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:19.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:18 smithi044 ceph-mon[16855]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:19.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:18 smithi165 ceph-mon[21975]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:21.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:21 smithi078 ceph-mon[21464]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:21.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:21 smithi044 ceph-mon[16855]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:21.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:21 smithi165 ceph-mon[21975]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:23.109 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:23 smithi078 ceph-mon[21464]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:23.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:23 smithi044 ceph-mon[16855]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:23.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:23 smithi165 ceph-mon[21975]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:25.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:25 smithi078 ceph-mon[21464]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:25.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:25 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T11:18:25.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:25 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:25.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:25 smithi078 ceph-mon[21464]: Deploying daemon osd.10 on smithi078 2024-04-20T11:18:25.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:25 smithi044 ceph-mon[16855]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:25.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:25 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T11:18:25.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:25 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:25.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:25 smithi044 ceph-mon[16855]: Deploying daemon osd.10 on smithi078 2024-04-20T11:18:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:25 smithi165 ceph-mon[21975]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:25 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T11:18:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:25 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:25 smithi165 ceph-mon[21975]: Deploying daemon osd.10 on smithi078 2024-04-20T11:18:27.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:27 smithi078 ceph-mon[21464]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:27.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:27 smithi044 ceph-mon[16855]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:27 smithi165 ceph-mon[21975]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:29.323 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:29 smithi078 ceph-mon[21464]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:29.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:29 smithi044 ceph-mon[16855]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:29 smithi165 ceph-mon[21975]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:31.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:31.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:31.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:30 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:31.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:30 smithi078 ceph-mon[21464]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:31.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:31.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:31.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:30 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:31.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:30 smithi044 ceph-mon[16855]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:31.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:31.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:31.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:30 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:31.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:30 smithi165 ceph-mon[21975]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:33.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:33 smithi078 ceph-mon[21464]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:33.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:33.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:33 smithi044 ceph-mon[16855]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:33.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:33.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:33.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:33 smithi165 ceph-mon[21975]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.886+0000 7fb1917fa700 1 -- 172.21.15.78:0/4056147961 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb194062e90 con 0x7fb17c0a3c40 2024-04-20T11:18:33.887 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 10 on host 'smithi078' 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb17c0a3c40 msgr2=0x7fb17c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb17c0a3c40 0x7fb17c0a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb184006fd0 tx=0x7fb184008040 comp rx=0 tx=0).stop 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 msgr2=0x7fb1941a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 0x7fb1941a03a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0094f0 tx=0x7fb18c00d3b0 comp rx=0 tx=0).stop 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 shutdown_connections 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb17c0a3c40 0x7fb17c0a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb194108790 0x7fb1941a03a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb194103530 0x7fb194198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 --2- 172.21.15.78:0/4056147961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb194105e60 0x7fb194199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:33.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 >> 172.21.15.78:0/4056147961 conn(0x7fb1940fcf40 msgr2=0x7fb194103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:33.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 shutdown_connections 2024-04-20T11:18:33.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-20T11:18:33.889+0000 7fb19a337700 1 -- 172.21.15.78:0/4056147961 wait complete. 2024-04-20T11:18:34.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:34 smithi078 ceph-mon[21464]: from='osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T11:18:34.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:34 smithi078 ceph-mon[21464]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T11:18:34.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:34.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:34.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:34.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:34 smithi044 ceph-mon[16855]: from='osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T11:18:34.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:34 smithi044 ceph-mon[16855]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T11:18:34.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:34.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:34.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:34.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:34 smithi165 ceph-mon[21975]: from='osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T11:18:34.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:34 smithi165 ceph-mon[21975]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T11:18:34.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:34.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:34.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:34.571 DEBUG:teuthology.orchestra.run.smithi078:osd.10> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.10.service 2024-04-20T11:18:34.574 INFO:tasks.cephadm:Deploying osd.11 on smithi165 with /dev/vg_nvme/lv_1... 2024-04-20T11:18:34.574 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T11:18:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:18:35.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-mon[21464]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:35.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-mon[21464]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T11:18:35.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-mon[21464]: from='osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:18:35.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-mon[21464]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T11:18:35.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:35.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-mon[21464]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:18:35.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:18:35.093+0000 7fe56ac4b700 -1 osd.10 0 waiting for initial osdmap 2024-04-20T11:18:35.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:18:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:18:35.106+0000 7fe566a3e700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:18:35.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:35 smithi165 ceph-mon[21975]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:35.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:35 smithi165 ceph-mon[21975]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T11:18:35.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:35 smithi165 ceph-mon[21975]: from='osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:18:35.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:35 smithi165 ceph-mon[21975]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T11:18:35.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:35 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:35.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:35 smithi165 ceph-mon[21975]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:18:35.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:35 smithi044 ceph-mon[16855]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:35.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:35 smithi044 ceph-mon[16855]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T11:18:35.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:35 smithi044 ceph-mon[16855]: from='osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:18:35.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:35 smithi044 ceph-mon[16855]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T11:18:35.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:35 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:35.349 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:35 smithi044 ceph-mon[16855]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-20T11:18:36.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:36 smithi165 ceph-mon[21975]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:18:36.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:36 smithi165 ceph-mon[21975]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T11:18:36.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:36 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:36.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:36 smithi165 ceph-mon[21975]: osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] boot 2024-04-20T11:18:36.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:36 smithi165 ceph-mon[21975]: osdmap e72: 11 total, 11 up, 11 in 2024-04-20T11:18:36.236 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:36 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:36.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:36 smithi078 ceph-mon[21464]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:18:36.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:36 smithi078 ceph-mon[21464]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T11:18:36.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:36 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:36.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:36 smithi078 ceph-mon[21464]: osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] boot 2024-04-20T11:18:36.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:36 smithi078 ceph-mon[21464]: osdmap e72: 11 total, 11 up, 11 in 2024-04-20T11:18:36.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:36 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:36.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:36 smithi044 ceph-mon[16855]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-20T11:18:36.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:36 smithi044 ceph-mon[16855]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T11:18:36.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:36 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:36.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:36 smithi044 ceph-mon[16855]: osd.10 [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] boot 2024-04-20T11:18:36.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:36 smithi044 ceph-mon[16855]: osdmap e72: 11 total, 11 up, 11 in 2024-04-20T11:18:36.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:36 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011382 s, 921 MB/s 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 2024-04-20T11:18:36.925 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-20T11:18:37.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:37 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:18:37.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:37 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:18:37.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:37 smithi078 ceph-mon[21464]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:37.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:37 smithi078 ceph-mon[21464]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T11:18:37.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:37 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:18:37.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:37 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:18:37.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:37 smithi165 ceph-mon[21975]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:37.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:37 smithi165 ceph-mon[21975]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T11:18:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:37 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:18:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:37 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:18:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:37 smithi044 ceph-mon[16855]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-20T11:18:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:37 smithi044 ceph-mon[16855]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T11:18:37.739 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-04-20T11:18:37.983 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi078 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-20T11:18:38.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:38.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:38.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:38 smithi078 ceph-mon[21464]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi078 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-20T11:18:38.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:38.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:38.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:38 smithi044 ceph-mon[16855]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi078 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:38.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:38.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:38.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:38 smithi165 ceph-mon[21975]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T11:18:39.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 -- 172.21.15.165:0/2487128092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64102bb0 msgr2=0x7fcb64103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:39.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 --2- 172.21.15.165:0/2487128092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64102bb0 0x7fcb64103010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009990 tx=0x7fcb58009ca0 comp rx=0 tx=0).stop 2024-04-20T11:18:39.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 -- 172.21.15.165:0/2487128092 shutdown_connections 2024-04-20T11:18:39.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 --2- 172.21.15.165:0/2487128092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb64103550 0x7fcb64109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:39.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 --2- 172.21.15.165:0/2487128092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb641019b0 0x7fcb64101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 --2- 172.21.15.165:0/2487128092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64102bb0 0x7fcb64103010 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 -- 172.21.15.165:0/2487128092 >> 172.21.15.165:0/2487128092 conn(0x7fcb640fcf50 msgr2=0x7fcb640ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 -- 172.21.15.165:0/2487128092 shutdown_connections 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.572+0000 7fcb6897e700 1 -- 172.21.15.165:0/2487128092 wait complete. 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 Processor -- start 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 -- start start 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb641019b0 0x7fcb64197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb64102bb0 0x7fcb64198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 0x7fcb6419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d730 con 0x7fcb641019b0 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d5b0 con 0x7fcb64102bb0 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb6897e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d8b0 con 0x7fcb64103550 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb62d9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 0x7fcb6419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb62d9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 0x7fcb6419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52548/0 (socket says 172.21.15.165:52548) 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb62d9d700 1 -- 172.21.15.165:0/3688311185 learned_addr learned my addr 172.21.15.165:0/3688311185 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.573+0000 7fcb61d9b700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb64102bb0 0x7fcb64198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb62d9d700 1 -- 172.21.15.165:0/3688311185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb64102bb0 msgr2=0x7fcb64198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb62d9d700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb64102bb0 0x7fcb64198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb62d9d700 1 -- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb641019b0 msgr2=0x7fcb64197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb62d9d700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb641019b0 0x7fcb64197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb62d9d700 1 -- 172.21.15.165:0/3688311185 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6419f9a0 con 0x7fcb64103550 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb61d9b700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb64102bb0 0x7fcb64198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb62d9d700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 0x7fcb6419f300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00b9a0 tx=0x7fcb5c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c019070 con 0x7fcb64103550 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6419fc90 con 0x7fcb64103550 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcb5c00ce30 con 0x7fcb64103550 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0075d0 con 0x7fcb64103550 2024-04-20T11:18:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.574+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641a0180 con 0x7fcb64103550 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.575+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb64069740 con 0x7fcb64103550 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.576+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb5c01c070 con 0x7fcb64103550 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.577+0000 7fcb537fe700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcb4c0a3c40 0x7fcb4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.577+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fcb5c0cf440 con 0x7fcb64103550 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.577+0000 7fcb6259c700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcb4c0a3c40 0x7fcb4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.578+0000 7fcb6259c700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcb4c0a3c40 0x7fcb4c0a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005fd0 tx=0x7fcb54005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:18:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.582+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcb5c09c470 con 0x7fcb64103550 2024-04-20T11:18:39.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:39.790+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcb64062f50 con 0x7fcb4c0a3c40 2024-04-20T11:18:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:39 smithi165 ceph-mon[21975]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:39 smithi165 ceph-mon[21975]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T11:18:40.132 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:39 smithi078 ceph-mon[21464]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:40.132 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:39 smithi078 ceph-mon[21464]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T11:18:40.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:39 smithi044 ceph-mon[16855]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:40.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:39 smithi044 ceph-mon[16855]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T11:18:40.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:18:40.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:18:40.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:40.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:40 smithi165 ceph-mon[21975]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T11:18:41.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:18:41.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:18:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:40 smithi078 ceph-mon[21464]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T11:18:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T11:18:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T11:18:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:40 smithi044 ceph-mon[16855]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T11:18:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:41 smithi165 ceph-mon[21975]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:41 smithi165 ceph-mon[21975]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:42.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:41 smithi078 ceph-mon[21464]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:42.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:41 smithi078 ceph-mon[21464]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:42.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:41 smithi044 ceph-mon[16855]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:18:42.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:41 smithi044 ceph-mon[16855]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:43.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:42 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/1479266163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]: dispatch 2024-04-20T11:18:43.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:42 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]: dispatch 2024-04-20T11:18:43.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:42 smithi165 ceph-mon[21975]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]': finished 2024-04-20T11:18:43.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:42 smithi165 ceph-mon[21975]: osdmap e77: 12 total, 11 up, 12 in 2024-04-20T11:18:43.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:42 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:18:43.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:42 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/1479266163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]: dispatch 2024-04-20T11:18:43.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:42 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]: dispatch 2024-04-20T11:18:43.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:42 smithi078 ceph-mon[21464]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]': finished 2024-04-20T11:18:43.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:42 smithi078 ceph-mon[21464]: osdmap e77: 12 total, 11 up, 12 in 2024-04-20T11:18:43.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:42 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:18:43.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:42 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/1479266163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]: dispatch 2024-04-20T11:18:43.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:42 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]: dispatch 2024-04-20T11:18:43.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:42 smithi044 ceph-mon[16855]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac"}]': finished 2024-04-20T11:18:43.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:42 smithi044 ceph-mon[16855]: osdmap e77: 12 total, 11 up, 12 in 2024-04-20T11:18:43.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:42 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:18:44.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:43 smithi078 ceph-mon[21464]: pgmap v272: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.165:0/1323693434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:18:44.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:43 smithi044 ceph-mon[16855]: pgmap v272: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:44.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.165:0/1323693434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:18:44.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:43 smithi165 ceph-mon[21975]: pgmap v272: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:44.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.165:0/1323693434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T11:18:46.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:45 smithi078 ceph-mon[21464]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-20T11:18:46.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:45 smithi044 ceph-mon[16855]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-20T11:18:46.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:45 smithi165 ceph-mon[21975]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-20T11:18:47.965 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:47 smithi165 ceph-mon[21975]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-20T11:18:48.190 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:47 smithi078 ceph-mon[21464]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-20T11:18:48.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:47 smithi044 ceph-mon[16855]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-20T11:18:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:49 smithi078 ceph-mon[21464]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:18:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:49 smithi165 ceph-mon[21975]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:18:50.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:49 smithi044 ceph-mon[16855]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T11:18:50.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T11:18:50.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:51.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T11:18:51.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:51.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T11:18:51.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:51 smithi165 ceph-mon[21975]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:18:51.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:51 smithi165 ceph-mon[21975]: Deploying daemon osd.11 on smithi165 2024-04-20T11:18:52.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:51 smithi078 ceph-mon[21464]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:18:52.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:51 smithi078 ceph-mon[21464]: Deploying daemon osd.11 on smithi165 2024-04-20T11:18:52.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:51 smithi044 ceph-mon[16855]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:18:52.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:51 smithi044 ceph-mon[16855]: Deploying daemon osd.11 on smithi165 2024-04-20T11:18:54.046 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:53 smithi165 ceph-mon[21975]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:18:54.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:53 smithi078 ceph-mon[21464]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:18:54.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:53 smithi044 ceph-mon[16855]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T11:18:55.117 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:54 smithi165 ceph-mon[21975]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:55.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:54 smithi078 ceph-mon[21464]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:55.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:54 smithi044 ceph-mon[16855]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:57.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:56 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:57.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:56 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:57.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:56 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:57.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:56 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:57.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:56 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:57.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:56 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:57.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:56 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:57.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:56 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:18:57.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:56 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:57 smithi165 ceph-mon[21975]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:58.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:57 smithi078 ceph-mon[21464]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:58.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:57 smithi044 ceph-mon[16855]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:59.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T11:18:59.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:18:59 smithi165 ceph-mon[21975]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T11:18:59.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.935+0000 7fcb537fe700 1 -- 172.21.15.165:0/3688311185 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcb64062f50 con 0x7fcb4c0a3c40 2024-04-20T11:18:59.936 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 11 on host 'smithi165' 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcb4c0a3c40 msgr2=0x7fcb4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcb4c0a3c40 0x7fcb4c0a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005fd0 tx=0x7fcb54005d00 comp rx=0 tx=0).stop 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 msgr2=0x7fcb6419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 0x7fcb6419f300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00b9a0 tx=0x7fcb5c00bcb0 comp rx=0 tx=0).stop 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 shutdown_connections 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcb4c0a3c40 0x7fcb4c0a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64103550 0x7fcb6419f300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb641019b0 0x7fcb64197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:59.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 --2- 172.21.15.165:0/3688311185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb64102bb0 0x7fcb64198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:18:59.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 >> 172.21.15.165:0/3688311185 conn(0x7fcb640fcf50 msgr2=0x7fcb64105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:18:59.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 shutdown_connections 2024-04-20T11:18:59.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:18:59.937+0000 7fcb6897e700 1 -- 172.21.15.165:0/3688311185 wait complete. 2024-04-20T11:19:00.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T11:19:00.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:18:59 smithi078 ceph-mon[21464]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T11:19:00.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:18:59 smithi044 ceph-mon[16855]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T11:19:00.656 DEBUG:teuthology.orchestra.run.smithi165:osd.11> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.11.service 2024-04-20T11:19:00.658 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-20T11:19:00.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd stat -f json 2024-04-20T11:19:01.130 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:01.174 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:00 smithi165 ceph-mon[21975]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:19:01.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:01.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:01.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:00 smithi078 ceph-mon[21464]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:19:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T11:19:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:19:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T11:19:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:00 smithi044 ceph-mon[16855]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-20T11:19:01.480 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:19:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:19:01.172+0000 7f9ba2d64700 -1 osd.11 0 waiting for initial osdmap 2024-04-20T11:19:01.480 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:19:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:19:01.185+0000 7f9b9eb57700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T11:19:02.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:02 smithi078 ceph-mon[21464]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:19:02.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:02 smithi078 ceph-mon[21464]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T11:19:02.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:02 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:02.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:02 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:02.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:02 smithi044 ceph-mon[16855]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:19:02.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:02 smithi044 ceph-mon[16855]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T11:19:02.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:02.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:02 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:02.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:02 smithi165 ceph-mon[21975]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-20T11:19:02.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:02 smithi165 ceph-mon[21975]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T11:19:02.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:02.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:02 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:03.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:03 smithi165 ceph-mon[21975]: purged_snaps scrub starts 2024-04-20T11:19:03.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:03 smithi165 ceph-mon[21975]: purged_snaps scrub ok 2024-04-20T11:19:03.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:03 smithi165 ceph-mon[21975]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:03.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:03 smithi165 ceph-mon[21975]: osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] boot 2024-04-20T11:19:03.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:03 smithi165 ceph-mon[21975]: osdmap e80: 12 total, 12 up, 12 in 2024-04-20T11:19:03.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:03.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:03 smithi078 ceph-mon[21464]: purged_snaps scrub starts 2024-04-20T11:19:03.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:03 smithi078 ceph-mon[21464]: purged_snaps scrub ok 2024-04-20T11:19:03.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:03 smithi078 ceph-mon[21464]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:03.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:03 smithi078 ceph-mon[21464]: osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] boot 2024-04-20T11:19:03.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:03 smithi078 ceph-mon[21464]: osdmap e80: 12 total, 12 up, 12 in 2024-04-20T11:19:03.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:03.441 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:03 smithi044 ceph-mon[16855]: purged_snaps scrub starts 2024-04-20T11:19:03.441 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:03 smithi044 ceph-mon[16855]: purged_snaps scrub ok 2024-04-20T11:19:03.441 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:03 smithi044 ceph-mon[16855]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-20T11:19:03.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:03 smithi044 ceph-mon[16855]: osd.11 [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] boot 2024-04-20T11:19:03.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:03 smithi044 ceph-mon[16855]: osdmap e80: 12 total, 12 up, 12 in 2024-04-20T11:19:03.442 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 -- 172.21.15.44:0/3761033681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf4103540 msgr2=0x7fbcf4105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/3761033681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf4103540 0x7fbcf4105930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009990 tx=0x7fbce8009ca0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 -- 172.21.15.44:0/3761033681 shutdown_connections 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/3761033681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf41087a0 0x7fbcf410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/3761033681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf4103540 0x7fbcf4105930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/3761033681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 0x7fbcf4108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 -- 172.21.15.44:0/3761033681 >> 172.21.15.44:0/3761033681 conn(0x7fbcf40fcf50 msgr2=0x7fbcf40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:03.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 -- 172.21.15.44:0/3761033681 shutdown_connections 2024-04-20T11:19:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.481+0000 7fbcfbe00700 1 -- 172.21.15.44:0/3761033681 wait complete. 2024-04-20T11:19:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.482+0000 7fbcfbe00700 1 Processor -- start 2024-04-20T11:19:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 -- start start 2024-04-20T11:19:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf4103540 0x7fbcf419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:03.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 0x7fbcf419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:03.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf41087a0 0x7fbcf41a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:03.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf410e830 con 0x7fbcf4103540 2024-04-20T11:19:03.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf410e6b0 con 0x7fbcf4105e70 2024-04-20T11:19:03.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcfbe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf410e9b0 con 0x7fbcf41087a0 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcf9400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf4103540 0x7fbcf419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcf9400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf4103540 0x7fbcf419b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55842/0 (socket says 172.21.15.44:55842) 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcf8a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 0x7fbcf419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcf9400700 1 -- 172.21.15.44:0/14427393 learned_addr learned my addr 172.21.15.44:0/14427393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.483+0000 7fbcf9e00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf41087a0 0x7fbcf41a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf8a00700 1 -- 172.21.15.44:0/14427393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf41087a0 msgr2=0x7fbcf41a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf8a00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf41087a0 0x7fbcf41a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf8a00700 1 -- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf4103540 msgr2=0x7fbcf419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf8a00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf4103540 0x7fbcf419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf8a00700 1 -- 172.21.15.44:0/14427393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf41a2c70 con 0x7fbcf4105e70 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf8a00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 0x7fbcf419b550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9f0 tx=0x7fbce000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:03.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004220 con 0x7fbcf4105e70 2024-04-20T11:19:03.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbce0004d30 con 0x7fbcf4105e70 2024-04-20T11:19:03.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0005060 con 0x7fbcf4105e70 2024-04-20T11:19:03.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.484+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf41a2f60 con 0x7fbcf4105e70 2024-04-20T11:19:03.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.485+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf41a33c0 con 0x7fbcf4105e70 2024-04-20T11:19:03.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.486+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf4069780 con 0x7fbcf4105e70 2024-04-20T11:19:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.488+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbce0004750 con 0x7fbcf4105e70 2024-04-20T11:19:03.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.489+0000 7fbcf2000700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbcdc0a3cc0 0x7fbcdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:03.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.489+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbce00cfea0 con 0x7fbcf4105e70 2024-04-20T11:19:03.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.490+0000 7fbcf9400700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbcdc0a3cc0 0x7fbcdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:03.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.490+0000 7fbcf9400700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbcdc0a3cc0 0x7fbcdc0a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbce8006010 tx=0x7fbce800b910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:03.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.493+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbce009cc80 con 0x7fbcf4105e70 2024-04-20T11:19:03.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.729+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbcf4063700 con 0x7fbcf4105e70 2024-04-20T11:19:03.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.733+0000 7fbcf2000700 1 -- 172.21.15.44:0/14427393 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fbce009c580 con 0x7fbcf4105e70 2024-04-20T11:19:03.733 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:03.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbcdc0a3cc0 msgr2=0x7fbcdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbcdc0a3cc0 0x7fbcdc0a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbce8006010 tx=0x7fbce800b910 comp rx=0 tx=0).stop 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 msgr2=0x7fbcf419b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 0x7fbcf419b550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9f0 tx=0x7fbce000ed00 comp rx=0 tx=0).stop 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 shutdown_connections 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbcdc0a3cc0 0x7fbcdc0a6180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbcf41087a0 0x7fbcf41a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcf4103540 0x7fbcf419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 --2- 172.21.15.44:0/14427393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4105e70 0x7fbcf419b550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 >> 172.21.15.44:0/14427393 conn(0x7fbcf40fcf50 msgr2=0x7fbcf4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:03.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 shutdown_connections 2024-04-20T11:19:03.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:03.736+0000 7fbcfbe00700 1 -- 172.21.15.44:0/14427393 wait complete. 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: Detected new or changed devices on smithi165 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/14427393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-20T11:19:04.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:04.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:04.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T11:19:04.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: Detected new or changed devices on smithi165 2024-04-20T11:19:04.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/14427393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:04.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: Detected new or changed devices on smithi165 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/14427393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:04.712 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713611942,"num_in_osds":12,"osd_in_since":1713611922,"num_remapped_pgs":0} 2024-04-20T11:19:04.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T11:19:05.161 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:19:05.929 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:05 smithi044 ceph-mon[16855]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:06.186 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:05 smithi165 ceph-mon[21975]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:06.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:05 smithi078 ceph-mon[21464]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:07.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 -- 172.21.15.44:0/2734154988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54801092e0 msgr2=0x7f54801096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 --2- 172.21.15.44:0/2734154988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54801092e0 0x7f54801096c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5470009920 tx=0x7f5470009c30 comp rx=0 tx=0).stop 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 -- 172.21.15.44:0/2734154988 shutdown_connections 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 --2- 172.21.15.44:0/2734154988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f548010e3c0 0x7f5480115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 --2- 172.21.15.44:0/2734154988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54801092e0 0x7f54801096c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 --2- 172.21.15.44:0/2734154988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5480109c90 0x7f548010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.754+0000 7f5484a00700 1 -- 172.21.15.44:0/2734154988 >> 172.21.15.44:0/2734154988 conn(0x7f5480076220 msgr2=0x7f5480107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.755+0000 7f5484a00700 1 -- 172.21.15.44:0/2734154988 shutdown_connections 2024-04-20T11:19:07.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.755+0000 7f5484a00700 1 -- 172.21.15.44:0/2734154988 wait complete. 2024-04-20T11:19:07.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 Processor -- start 2024-04-20T11:19:07.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 -- start start 2024-04-20T11:19:07.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54801092e0 0x7f54801a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:07.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5480109c90 0x7f54801a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:07.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 0x7f548019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54801194e0 con 0x7f5480109c90 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5480119360 con 0x7f54801092e0 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.756+0000 7f5484a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5480119660 con 0x7f548010e3c0 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547e800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 0x7f548019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f5477e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5480109c90 0x7f54801a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547e800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 0x7f548019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58520/0 (socket says 172.21.15.44:58520) 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547e800700 1 -- 172.21.15.44:0/2421688459 learned_addr learned my addr 172.21.15.44:0/2421688459 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:07.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547de00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54801092e0 0x7f54801a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547de00700 1 -- 172.21.15.44:0/2421688459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 msgr2=0x7f548019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547de00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 0x7f548019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547de00700 1 -- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5480109c90 msgr2=0x7f54801a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547de00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5480109c90 0x7f54801a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547de00700 1 -- 172.21.15.44:0/2421688459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54801a0020 con 0x7f54801092e0 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f547e800700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 0x7f548019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:07.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.757+0000 7f5477e00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5480109c90 0x7f54801a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.758+0000 7f547de00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54801092e0 0x7f54801a5670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5470009230 tx=0x7f5470010f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.758+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470027070 con 0x7f54801092e0 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.758+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f547000f960 con 0x7f54801092e0 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.758+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470004930 con 0x7f54801092e0 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.758+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54801a02b0 con 0x7f54801092e0 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.758+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54801ac420 con 0x7f54801092e0 2024-04-20T11:19:07.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.759+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548010a7c0 con 0x7f54801092e0 2024-04-20T11:19:07.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.761+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f547002a070 con 0x7f54801092e0 2024-04-20T11:19:07.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.763+0000 7f5476a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54640a3c40 0x7f54640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:07.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.763+0000 7f5477e00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54640a3c40 0x7f54640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:07.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.765+0000 7f5477e00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54640a3c40 0x7f54640a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54801a0f00 tx=0x7f546800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:07.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.766+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5470009480 con 0x7f54801092e0 2024-04-20T11:19:07.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:07.771+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5470013030 con 0x7f54801092e0 2024-04-20T11:19:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:07 smithi044 ceph-mon[16855]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.000+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f548002df90 con 0x7f54801092e0 2024-04-20T11:19:08.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.002+0000 7f5476a00700 1 -- 172.21.15.44:0/2421688459 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18408 (secure 0 0 0) 0x7f547009d830 con 0x7f54801092e0 2024-04-20T11:19:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:08.003 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":81,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:19:03.182461+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:19:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54640a3c40 msgr2=0x7f54640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54640a3c40 0x7f54640a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54801a0f00 tx=0x7f546800b040 comp rx=0 tx=0).stop 2024-04-20T11:19:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54801092e0 msgr2=0x7f54801a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54801092e0 0x7f54801a5670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5470009230 tx=0x7f5470010f70 comp rx=0 tx=0).stop 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 shutdown_connections 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54640a3c40 0x7f54640a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f548010e3c0 0x7f548019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54801092e0 0x7f54801a5670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 --2- 172.21.15.44:0/2421688459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5480109c90 0x7f54801a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.005+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 >> 172.21.15.44:0/2421688459 conn(0x7f5480076220 msgr2=0x7f5480076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.006+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 shutdown_connections 2024-04-20T11:19:08.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:08.006+0000 7f5484a00700 1 -- 172.21.15.44:0/2421688459 wait complete. 2024-04-20T11:19:08.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:07 smithi078 ceph-mon[21464]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:08.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:07 smithi165 ceph-mon[21975]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:08.906 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:19:08.907 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-20T11:19:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:08 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2421688459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:09.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:08 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2421688459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:09.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:08 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2421688459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:09.352 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:19:10.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:09 smithi078 ceph-mon[21464]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:10.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:09 smithi044 ceph-mon[16855]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:09 smithi165 ceph-mon[21975]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:11.146 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:10 smithi044 ceph-mon[16855]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:11.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:10 smithi078 ceph-mon[21464]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:10 smithi165 ceph-mon[21975]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.708+0000 7f3935800700 1 -- 172.21.15.44:0/3312993346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3930103b90 msgr2=0x7f3930103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.708+0000 7f3935800700 1 --2- 172.21.15.44:0/3312993346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3930103b90 0x7f3930103f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39200098f0 tx=0x7f3920009c00 comp rx=0 tx=0).stop 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 -- 172.21.15.44:0/3312993346 shutdown_connections 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 --2- 172.21.15.44:0/3312993346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 0x7f393010f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 --2- 172.21.15.44:0/3312993346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3930103b90 0x7f3930103f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 --2- 172.21.15.44:0/3312993346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39301044b0 0x7f393010c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 -- 172.21.15.44:0/3312993346 >> 172.21.15.44:0/3312993346 conn(0x7f39300fd3e0 msgr2=0x7f39300ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 -- 172.21.15.44:0/3312993346 shutdown_connections 2024-04-20T11:19:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.709+0000 7f3935800700 1 -- 172.21.15.44:0/3312993346 wait complete. 2024-04-20T11:19:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.710+0000 7f3935800700 1 Processor -- start 2024-04-20T11:19:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 -- start start 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 0x7f393019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39301044b0 0x7f393019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 0x7f393019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3930112e90 con 0x7f3930103b90 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3930112d10 con 0x7f393010cfc0 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.711+0000 7f3935800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3930113010 con 0x7f39301044b0 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 0x7f393019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 0x7f393019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48710/0 (socket says 172.21.15.44:48710) 2024-04-20T11:19:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 -- 172.21.15.44:0/680141279 learned_addr learned my addr 172.21.15.44:0/680141279 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392f400700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39301044b0 0x7f393019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f3934e00700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 0x7f393019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 -- 172.21.15.44:0/680141279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39301044b0 msgr2=0x7f393019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39301044b0 0x7f393019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 -- 172.21.15.44:0/680141279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 msgr2=0x7f393019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 0x7f393019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392fe00700 1 -- 172.21.15.44:0/680141279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39301a0560 con 0x7f3930103b90 2024-04-20T11:19:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f392f400700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39301044b0 0x7f393019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.712+0000 7f3934e00700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 0x7f393019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.713+0000 7f392fe00700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 0x7f393019b550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3920010f70 tx=0x7f3920010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.713+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920027070 con 0x7f3930103b90 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.713+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39301a07f0 con 0x7f3930103b90 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.713+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f392000fa60 con 0x7f3930103b90 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.713+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920004990 con 0x7f3930103b90 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.713+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a80d0 con 0x7f3930103b90 2024-04-20T11:19:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.714+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3930069740 con 0x7f3930103b90 2024-04-20T11:19:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.716+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f392002a070 con 0x7f3930103b90 2024-04-20T11:19:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.718+0000 7f392cc00700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f39140a3c40 0x7f39140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.718+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f39200d0a40 con 0x7f3930103b90 2024-04-20T11:19:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.718+0000 7f392f400700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f39140a3c40 0x7f39140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.719+0000 7f392f400700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f39140a3c40 0x7f39140a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f393010e860 tx=0x7f391c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.722+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f392005eff0 con 0x7f3930103b90 2024-04-20T11:19:11.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.953+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39301a82a0 con 0x7f3930103b90 2024-04-20T11:19:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.953+0000 7f392cc00700 1 -- 172.21.15.44:0/680141279 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f3920013030 con 0x7f3930103b90 2024-04-20T11:19:11.953 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:19:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.956+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f39140a3c40 msgr2=0x7f39140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.956+0000 7f3935800700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f39140a3c40 0x7f39140a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f393010e860 tx=0x7f391c009310 comp rx=0 tx=0).stop 2024-04-20T11:19:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.956+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 msgr2=0x7f393019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.956+0000 7f3935800700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 0x7f393019b550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3920010f70 tx=0x7f3920010fa0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 shutdown_connections 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f39140a3c40 0x7f39140a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393010cfc0 0x7f393019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3930103b90 0x7f393019b550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 --2- 172.21.15.44:0/680141279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39301044b0 0x7f393019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 >> 172.21.15.44:0/680141279 conn(0x7f39300fd3e0 msgr2=0x7f39300ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 shutdown_connections 2024-04-20T11:19:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:11.957+0000 7f3935800700 1 -- 172.21.15.44:0/680141279 wait complete. 2024-04-20T11:19:12.349 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:12 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/680141279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:12.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:12 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/680141279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:12.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:12 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/680141279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:12.951 INFO:tasks.cephadm:Adding mds.a on smithi044 2024-04-20T11:19:12.951 INFO:tasks.cephadm:Adding mds.d on smithi044 2024-04-20T11:19:12.951 INFO:tasks.cephadm:Adding mds.g on smithi044 2024-04-20T11:19:12.951 INFO:tasks.cephadm:Adding mds.j on smithi044 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.b on smithi078 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.e on smithi078 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.h on smithi078 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.k on smithi078 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.c on smithi165 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.f on smithi165 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.i on smithi165 2024-04-20T11:19:12.952 INFO:tasks.cephadm:Adding mds.l on smithi165 2024-04-20T11:19:12.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph orch apply mds all '12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l' 2024-04-20T11:19:13.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:13 smithi044 ceph-mon[16855]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:13.205 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:19:13.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:13 smithi165 ceph-mon[21975]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:13.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:13 smithi078 ceph-mon[21464]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.010+0000 7fc4f8f75700 1 -- 172.21.15.165:0/333674212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 msgr2=0x7fc4f4109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.010+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/333674212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f4109e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b0a0 tx=0x7fc4ec00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 -- 172.21.15.165:0/333674212 shutdown_connections 2024-04-20T11:19:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/333674212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f4109e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/333674212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4f41019c0 0x7fc4f4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/333674212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f4102bc0 0x7fc4f4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 -- 172.21.15.165:0/333674212 >> 172.21.15.165:0/333674212 conn(0x7fc4f40fcf80 msgr2=0x7fc4f40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 -- 172.21.15.165:0/333674212 shutdown_connections 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 -- 172.21.15.165:0/333674212 wait complete. 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 Processor -- start 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.011+0000 7fc4f8f75700 1 -- start start 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4f41019c0 0x7fc4f4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f4102bc0 0x7fc4f4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f41a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f410d760 con 0x7fc4f4102bc0 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f410d5e0 con 0x7fc4f41019c0 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f410d8e0 con 0x7fc4f4103560 2024-04-20T11:19:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f41a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f41a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47028/0 (socket says 172.21.15.165:47028) 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 -- 172.21.15.165:0/3577677541 learned_addr learned my addr 172.21.15.165:0/3577677541 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f259c700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4f41019c0 0x7fc4f4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 -- 172.21.15.165:0/3577677541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4f41019c0 msgr2=0x7fc4f4198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4f41019c0 0x7fc4f4198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 -- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f4102bc0 msgr2=0x7fc4f4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f4102bc0 0x7fc4f4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 -- 172.21.15.165:0/3577677541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f41a0a70 con 0x7fc4f4103560 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f2d9d700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f41a03d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b070 tx=0x7fc4ec004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec021030 con 0x7fc4f4103560 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.012+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f41a0d00 con 0x7fc4f4103560 2024-04-20T11:19:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.013+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4ec007bb0 con 0x7fc4f4103560 2024-04-20T11:19:15.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.013+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec004890 con 0x7fc4f4103560 2024-04-20T11:19:15.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.013+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f41a1230 con 0x7fc4f4103560 2024-04-20T11:19:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.014+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f4069740 con 0x7fc4f4103560 2024-04-20T11:19:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.015+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4ec024070 con 0x7fc4f4103560 2024-04-20T11:19:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.015+0000 7fc4e37fe700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4dc0a3c40 0x7fc4dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.015+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4ec00f020 con 0x7fc4f4103560 2024-04-20T11:19:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.016+0000 7fc4f259c700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4dc0a3c40 0x7fc4dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.017+0000 7fc4f259c700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4dc0a3c40 0x7fc4dc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4005fd0 tx=0x7fc4e400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.020+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc4ec004000 con 0x7fc4f4103560 2024-04-20T11:19:15.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:15 smithi044 ceph-mon[16855]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:15.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.231+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}) -- 0x7fc4f40ff1d0 con 0x7fc4dc0a3c40 2024-04-20T11:19:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.249+0000 7fc4e37fe700 1 -- 172.21.15.165:0/3577677541 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc4f40ff1d0 con 0x7fc4dc0a3c40 2024-04-20T11:19:15.249 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mds.all update... 2024-04-20T11:19:15.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4dc0a3c40 msgr2=0x7fc4dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:15.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4dc0a3c40 0x7fc4dc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4005fd0 tx=0x7fc4e400a560 comp rx=0 tx=0).stop 2024-04-20T11:19:15.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 msgr2=0x7fc4f41a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:15.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f41a03d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b070 tx=0x7fc4ec004ef0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 shutdown_connections 2024-04-20T11:19:15.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4dc0a3c40 0x7fc4dc0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f4103560 0x7fc4f41a03d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4f41019c0 0x7fc4f4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 --2- 172.21.15.165:0/3577677541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4f4102bc0 0x7fc4f4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:15.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 >> 172.21.15.165:0/3577677541 conn(0x7fc4f40fcf80 msgr2=0x7fc4f40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:15.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 shutdown_connections 2024-04-20T11:19:15.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T11:19:15.252+0000 7fc4f8f75700 1 -- 172.21.15.165:0/3577677541 wait complete. 2024-04-20T11:19:15.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:15 smithi078 ceph-mon[21464]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:15 smithi165 ceph-mon[21975]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:15.898 DEBUG:teuthology.orchestra.run.smithi044:mds.a> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.a.service 2024-04-20T11:19:15.905 DEBUG:teuthology.orchestra.run.smithi044:mds.d> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.d.service 2024-04-20T11:19:15.953 DEBUG:teuthology.orchestra.run.smithi044:mds.g> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.g.service 2024-04-20T11:19:15.999 DEBUG:teuthology.orchestra.run.smithi044:mds.j> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.j.service 2024-04-20T11:19:16.044 DEBUG:teuthology.orchestra.run.smithi078:mds.b> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.b.service 2024-04-20T11:19:16.046 DEBUG:teuthology.orchestra.run.smithi078:mds.e> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.e.service 2024-04-20T11:19:16.089 DEBUG:teuthology.orchestra.run.smithi078:mds.h> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.h.service 2024-04-20T11:19:16.133 DEBUG:teuthology.orchestra.run.smithi078:mds.k> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.k.service 2024-04-20T11:19:16.177 DEBUG:teuthology.orchestra.run.smithi165:mds.c> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.c.service 2024-04-20T11:19:16.180 DEBUG:teuthology.orchestra.run.smithi165:mds.f> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.f.service 2024-04-20T11:19:16.223 DEBUG:teuthology.orchestra.run.smithi165:mds.i> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.i.service 2024-04-20T11:19:16.266 DEBUG:teuthology.orchestra.run.smithi165:mds.l> sudo journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.l.service 2024-04-20T11:19:16.309 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-20T11:19:16.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:19:16.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:16 smithi078 ceph-mon[21464]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:16.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:16 smithi078 ceph-mon[21464]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-20T11:19:16.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:16.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:16 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:16.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:16 smithi165 ceph-mon[21975]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:16.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:16 smithi165 ceph-mon[21975]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-20T11:19:16.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:16.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:16 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 -- 172.21.15.44:0/209995451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 msgr2=0x7fe7ac110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 --2- 172.21.15.44:0/209995451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 0x7fe7ac110820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009900 tx=0x7fe7a402bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 -- 172.21.15.44:0/209995451 shutdown_connections 2024-04-20T11:19:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 --2- 172.21.15.44:0/209995451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 0x7fe7ac110820 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 --2- 172.21.15.44:0/209995451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1095d0 0x7fe7ac109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 --2- 172.21.15.44:0/209995451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 -- 172.21.15.44:0/209995451 >> 172.21.15.44:0/209995451 conn(0x7fe7ac0773a0 msgr2=0x7fe7ac0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 -- 172.21.15.44:0/209995451 shutdown_connections 2024-04-20T11:19:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.538+0000 7fe7b1400640 1 -- 172.21.15.44:0/209995451 wait complete. 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.539+0000 7fe7b1400640 1 Processor -- start 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 -- start start 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ac1095d0 0x7fe7ac19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 0x7fe7ac1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac114140 con 0x7fe7ac1095d0 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac113fc0 con 0x7fe7ac1083e0 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b1400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1142c0 con 0x7fe7ac109f90 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ac1095d0 0x7fe7ac19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:47054/0 (socket says 172.21.15.44:47054) 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7abe00640 1 -- 172.21.15.44:0/218590484 learned_addr learned my addr 172.21.15.44:0/218590484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.540+0000 7fe7b0a00640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 0x7fe7ac1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7abe00640 1 -- 172.21.15.44:0/218590484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 msgr2=0x7fe7ac1a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7abe00640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 0x7fe7ac1a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7abe00640 1 -- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ac1095d0 msgr2=0x7fe7ac19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7abe00640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ac1095d0 0x7fe7ac19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7abe00640 1 -- 172.21.15.44:0/218590484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac1a6370 con 0x7fe7ac1083e0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7abe00640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac19e750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00ebc0 tx=0x7fe79c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003b70 con 0x7fe7ac1083e0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac1a6660 con 0x7fe7ac1083e0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe79c003d10 con 0x7fe7ac1083e0 2024-04-20T11:19:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c019980 con 0x7fe7ac1083e0 2024-04-20T11:19:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.541+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1a6c00 con 0x7fe7ac1083e0 2024-04-20T11:19:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.543+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac10f700 con 0x7fe7ac1083e0 2024-04-20T11:19:16.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.544+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe79c019b20 con 0x7fe7ac1083e0 2024-04-20T11:19:16.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.546+0000 7fe7a8c00640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7900a3bb0 0x7fe7900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:16.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.546+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe79c0d0890 con 0x7fe7ac1083e0 2024-04-20T11:19:16.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.546+0000 7fe7ab400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7900a3bb0 0x7fe7900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:16.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.547+0000 7fe7ab400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7900a3bb0 0x7fe7900a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac19fc60 tx=0x7fe794005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:16.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.550+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe79c09d670 con 0x7fe7ac1083e0 2024-04-20T11:19:16.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:16 smithi044 ceph-mon[16855]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:19:16.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:16 smithi044 ceph-mon[16855]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-20T11:19:16.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:16.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:16 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:16.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.772+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7ac062b60 con 0x7fe7ac1083e0 2024-04-20T11:19:16.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.774+0000 7fe7a8c00640 1 -- 172.21.15.44:0/218590484 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18408 (secure 0 0 0) 0x7fe79c09cf70 con 0x7fe7ac1083e0 2024-04-20T11:19:16.774 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:16.775 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":81,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:19:03.182461+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7900a3bb0 msgr2=0x7fe7900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7900a3bb0 0x7fe7900a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac19fc60 tx=0x7fe794005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 msgr2=0x7fe7ac19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac19e750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00ebc0 tx=0x7fe79c00c6a0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 shutdown_connections 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7900a3bb0 0x7fe7900a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7ac109f90 0x7fe7ac1a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ac1095d0 0x7fe7ac19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 --2- 172.21.15.44:0/218590484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac1083e0 0x7fe7ac19e750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.776+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 >> 172.21.15.44:0/218590484 conn(0x7fe7ac0773a0 msgr2=0x7fe7ac10c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.777+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 shutdown_connections 2024-04-20T11:19:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:16.777+0000 7fe7b1400640 1 -- 172.21.15.44:0/218590484 wait complete. 2024-04-20T11:19:16.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:19:16.800 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 -- 172.21.15.44:0/2446724939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 msgr2=0x7f5d98109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 --2- 172.21.15.44:0/2446724939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98109eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009ca0 tx=0x7f5d9002c100 comp rx=0 tx=0).stop 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 -- 172.21.15.44:0/2446724939 shutdown_connections 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 --2- 172.21.15.44:0/2446724939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98109eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 --2- 172.21.15.44:0/2446724939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d98105130 0x7f5d98107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 --2- 172.21.15.44:0/2446724939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98102800 0x7f5d98104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 -- 172.21.15.44:0/2446724939 >> 172.21.15.44:0/2446724939 conn(0x7f5d980fc230 msgr2=0x7f5d980fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 -- 172.21.15.44:0/2446724939 shutdown_connections 2024-04-20T11:19:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.018+0000 7f5d97200640 1 -- 172.21.15.44:0/2446724939 wait complete. 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.019+0000 7f5d97200640 1 Processor -- start 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.019+0000 7f5d97200640 1 -- start start 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d97200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98102800 0x7f5d98102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d97200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d98105130 0x7f5d981007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d97200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d97200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9810db50 con 0x7f5d98105130 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d97200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9810d9d0 con 0x7f5d98102800 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d97200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9810dcd0 con 0x7f5d98107a60 2024-04-20T11:19:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d98105130 0x7f5d981007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42702/0 (socket says 172.21.15.44:42702) 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 -- 172.21.15.44:0/280924973 learned_addr learned my addr 172.21.15.44:0/280924973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d95e00640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98102800 0x7f5d98102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 -- 172.21.15.44:0/280924973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98102800 msgr2=0x7f5d98102170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98102800 0x7f5d98102170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 -- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d98105130 msgr2=0x7f5d981007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d98105130 0x7f5d981007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.020+0000 7f5d96800640 1 -- 172.21.15.44:0/280924973 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d98101580 con 0x7f5d98107a60 2024-04-20T11:19:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.021+0000 7f5d96800640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98100cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d9002c640 tx=0x7f5d90034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.021+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90039040 con 0x7f5d98107a60 2024-04-20T11:19:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.021+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5d90044070 con 0x7f5d98107a60 2024-04-20T11:19:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.021+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900043a0 con 0x7f5d98107a60 2024-04-20T11:19:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.021+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d981a2170 con 0x7f5d98107a60 2024-04-20T11:19:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.021+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d981a2670 con 0x7f5d98107a60 2024-04-20T11:19:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.023+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d980fdd20 con 0x7f5d98107a60 2024-04-20T11:19:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.024+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d90004540 con 0x7f5d98107a60 2024-04-20T11:19:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.026+0000 7f5d8ea00640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5d7c0a3bb0 0x7f5d7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.026+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5d900f2510 con 0x7f5d98107a60 2024-04-20T11:19:17.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.026+0000 7f5d95e00640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5d7c0a3bb0 0x7f5d7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.027+0000 7f5d95e00640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5d7c0a3bb0 0x7f5d7c0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009bd0 tx=0x7f5d88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:17.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.030+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5d900bf2f0 con 0x7f5d98107a60 2024-04-20T11:19:17.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.237+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d98062b60 con 0x7f5d98107a60 2024-04-20T11:19:17.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.238+0000 7f5d8ea00640 1 -- 172.21.15.44:0/280924973 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f5d900bebf0 con 0x7f5d98107a60 2024-04-20T11:19:17.239 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5d7c0a3bb0 msgr2=0x7f5d7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5d7c0a3bb0 0x7f5d7c0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009bd0 tx=0x7f5d88005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 msgr2=0x7f5d98100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98100cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d9002c640 tx=0x7f5d90034670 comp rx=0 tx=0).stop 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 shutdown_connections 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5d7c0a3bb0 0x7f5d7c0a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d98107a60 0x7f5d98100cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d98105130 0x7f5d981007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 --2- 172.21.15.44:0/280924973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98102800 0x7f5d98102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 >> 172.21.15.44:0/280924973 conn(0x7f5d980fc230 msgr2=0x7f5d98103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:17.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 shutdown_connections 2024-04-20T11:19:17.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.241+0000 7f5d97200640 1 -- 172.21.15.44:0/280924973 wait complete. 2024-04-20T11:19:17.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 -- 172.21.15.44:0/587713730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 msgr2=0x7f7eec109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 --2- 172.21.15.44:0/587713730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec109a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009900 tx=0x7f7ed402bc50 comp rx=0 tx=0).stop 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 -- 172.21.15.44:0/587713730 shutdown_connections 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 --2- 172.21.15.44:0/587713730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 0x7f7eec110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 --2- 172.21.15.44:0/587713730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec109a50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 --2- 172.21.15.44:0/587713730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 0x7f7eec108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 -- 172.21.15.44:0/587713730 >> 172.21.15.44:0/587713730 conn(0x7f7eec0773a0 msgr2=0x7f7eec0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:17.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 -- 172.21.15.44:0/587713730 shutdown_connections 2024-04-20T11:19:17.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.470+0000 7f7eeb200640 1 -- 172.21.15.44:0/587713730 wait complete. 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.471+0000 7f7eeb200640 1 Processor -- start 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 -- start start 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 0x7f7eec19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 0x7f7eec1a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec114190 con 0x7f7eec1095d0 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec114010 con 0x7f7eec109f90 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eeb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec114310 con 0x7f7eec1083e0 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7ee3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7ee9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 0x7f7eec19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7ee3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48770/0 (socket says 172.21.15.44:48770) 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7ee3e00640 1 -- 172.21.15.44:0/1105195737 learned_addr learned my addr 172.21.15.44:0/1105195737 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.472+0000 7f7eea800640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 0x7f7eec1a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eea800640 1 -- 172.21.15.44:0/1105195737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 msgr2=0x7f7eec19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eea800640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 0x7f7eec19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eea800640 1 -- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 msgr2=0x7f7eec19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eea800640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eea800640 1 -- 172.21.15.44:0/1105195737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec1a6380 con 0x7f7eec109f90 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7ee3e00640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7ee9e00640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 0x7f7eec19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eea800640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 0x7f7eec1a5d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009fd0 tx=0x7f7ee400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400ce20 con 0x7f7eec109f90 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ee4004b20 con 0x7f7eec109f90 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4010790 con 0x7f7eec109f90 2024-04-20T11:19:17.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.473+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec1a6670 con 0x7f7eec109f90 2024-04-20T11:19:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.474+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec1a6b70 con 0x7f7eec109f90 2024-04-20T11:19:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.475+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec10f6a0 con 0x7f7eec109f90 2024-04-20T11:19:17.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.477+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7ee4005020 con 0x7f7eec109f90 2024-04-20T11:19:17.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.478+0000 7f7ee2a00640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7ed00a3c30 0x7f7ed00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:17.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.478+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7ee40d0ae0 con 0x7f7eec109f90 2024-04-20T11:19:17.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.479+0000 7f7ee9e00640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7ed00a3c30 0x7f7ed00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:17.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:17.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:17.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:17.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: Deploying daemon mds.l on smithi165 2024-04-20T11:19:17.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/218590484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:17.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/280924973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.479+0000 7f7ee9e00640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7ed00a3c30 0x7f7ed00a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7eec109430 tx=0x7f7edc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:17.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.480+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7ee409d8c0 con 0x7f7eec109f90 2024-04-20T11:19:17.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: Deploying daemon mds.l on smithi165 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/218590484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:17.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/280924973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: Deploying daemon mds.l on smithi165 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/218590484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:17.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/280924973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:17.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.729+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eec02d5d0 con 0x7f7eec109f90 2024-04-20T11:19:17.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.729+0000 7f7ee2a00640 1 -- 172.21.15.44:0/1105195737 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f7ee409d1c0 con 0x7f7eec109f90 2024-04-20T11:19:17.730 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:17.730 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-20T11:19:17.730 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 1 2024-04-20T11:19:17.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.732+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7ed00a3c30 msgr2=0x7f7ed00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.732+0000 7f7eeb200640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7ed00a3c30 0x7f7ed00a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7eec109430 tx=0x7f7edc005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:17.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.732+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 msgr2=0x7f7eec1a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.732+0000 7f7eeb200640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 0x7f7eec1a5d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009fd0 tx=0x7f7ee400ef10 comp rx=0 tx=0).stop 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 shutdown_connections 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7ed00a3c30 0x7f7ed00a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eec109f90 0x7f7eec1a5d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eec1095d0 0x7f7eec19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 --2- 172.21.15.44:0/1105195737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eec1083e0 0x7f7eec19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 >> 172.21.15.44:0/1105195737 conn(0x7f7eec0773a0 msgr2=0x7f7eec10c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 shutdown_connections 2024-04-20T11:19:17.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:17.733+0000 7f7eeb200640 1 -- 172.21.15.44:0/1105195737 wait complete. 2024-04-20T11:19:18.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:18 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1105195737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:18.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:18 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1105195737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:18.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:18 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1105195737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:19.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:19 smithi165 ceph-mon[21975]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:19.480 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:19:19 smithi165 systemd[1]: Started Ceph mds.l for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:19.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:19 smithi078 ceph-mon[21464]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:19.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:19 smithi044 ceph-mon[16855]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:19.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 -- 172.21.15.44:0/1854421310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c101e00 msgr2=0x7efe6c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 --2- 172.21.15.44:0/1854421310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c101e00 0x7efe6c102220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009900 tx=0x7efe5c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 -- 172.21.15.44:0/1854421310 shutdown_connections 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 --2- 172.21.15.44:0/1854421310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6c1039b0 0x7efe6c10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 --2- 172.21.15.44:0/1854421310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c102ff0 0x7efe6c103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 --2- 172.21.15.44:0/1854421310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c101e00 0x7efe6c102220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 -- 172.21.15.44:0/1854421310 >> 172.21.15.44:0/1854421310 conn(0x7efe6c0fd370 msgr2=0x7efe6c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 -- 172.21.15.44:0/1854421310 shutdown_connections 2024-04-20T11:19:19.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.970+0000 7efe6b200640 1 -- 172.21.15.44:0/1854421310 wait complete. 2024-04-20T11:19:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 Processor -- start 2024-04-20T11:19:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 -- start start 2024-04-20T11:19:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 0x7efe6c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c102ff0 0x7efe6c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6c1039b0 0x7efe6c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10db60 con 0x7efe6c102ff0 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10d9e0 con 0x7efe6c101e00 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.971+0000 7efe6b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10dce0 con 0x7efe6c1039b0 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 0x7efe6c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 0x7efe6c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:52462/0 (socket says 172.21.15.44:52462) 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 -- 172.21.15.44:0/297745047 learned_addr learned my addr 172.21.15.44:0/297745047 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69400640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c102ff0 0x7efe6c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe6a800640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6c1039b0 0x7efe6c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 -- 172.21.15.44:0/297745047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6c1039b0 msgr2=0x7efe6c1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6c1039b0 0x7efe6c1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 -- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c102ff0 msgr2=0x7efe6c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c102ff0 0x7efe6c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 -- 172.21.15.44:0/297745047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1a2000 con 0x7efe6c101e00 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.972+0000 7efe69e00640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 0x7efe6c19a3e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe5c000c00 tx=0x7efe5c004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.973+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c043020 con 0x7efe6c101e00 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.973+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe5c004250 con 0x7efe6c101e00 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.973+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1a2290 con 0x7efe6c101e00 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.973+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003b60 con 0x7efe6c101e00 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.973+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1a2830 con 0x7efe6c101e00 2024-04-20T11:19:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.975+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c1a2a30 con 0x7efe6c101e00 2024-04-20T11:19:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.976+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efe5c046020 con 0x7efe6c101e00 2024-04-20T11:19:19.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.978+0000 7efe62a00640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efe500a3c30 0x7efe500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:19.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.978+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efe5c0f1fe0 con 0x7efe6c101e00 2024-04-20T11:19:19.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.978+0000 7efe69400640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efe500a3c30 0x7efe500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.979+0000 7efe69400640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efe500a3c30 0x7efe500a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe6c19b8f0 tx=0x7efe58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:19.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:19.983+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efe5c04d020 con 0x7efe6c101e00 2024-04-20T11:19:20.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.231+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe6c1a3060 con 0x7efe6c101e00 2024-04-20T11:19:20.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.232+0000 7efe62a00640 1 -- 172.21.15.44:0/297745047 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7efe5c0bee00 con 0x7efe6c101e00 2024-04-20T11:19:20.232 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:20.232 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T11:19:20.233 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-04-20T11:19:20.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.234+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efe500a3c30 msgr2=0x7efe500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:20.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.234+0000 7efe6b200640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efe500a3c30 0x7efe500a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe6c19b8f0 tx=0x7efe58005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:20.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.234+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 msgr2=0x7efe6c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:20.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.234+0000 7efe6b200640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 0x7efe6c19a3e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe5c000c00 tx=0x7efe5c004a30 comp rx=0 tx=0).stop 2024-04-20T11:19:20.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 shutdown_connections 2024-04-20T11:19:20.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efe500a3c30 0x7efe500a60f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:20.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6c1039b0 0x7efe6c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:20.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe6c102ff0 0x7efe6c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:20.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 --2- 172.21.15.44:0/297745047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c101e00 0x7efe6c19a3e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:20.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 >> 172.21.15.44:0/297745047 conn(0x7efe6c0fd370 msgr2=0x7efe6c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:20.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 shutdown_connections 2024-04-20T11:19:20.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:20.235+0000 7efe6b200640 1 -- 172.21.15.44:0/297745047 wait complete. 2024-04-20T11:19:20.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6832/3129181151,v1:172.21.15.165:6833/3129181151] up:boot 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: fsmap 1 up:standby 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: Deploying daemon mds.k on smithi078 2024-04-20T11:19:20.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:20 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/297745047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:20.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6832/3129181151,v1:172.21.15.165:6833/3129181151] up:boot 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: fsmap 1 up:standby 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: Deploying daemon mds.k on smithi078 2024-04-20T11:19:20.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:20 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/297745047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:20.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6832/3129181151,v1:172.21.15.165:6833/3129181151] up:boot 2024-04-20T11:19:20.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: fsmap 1 up:standby 2024-04-20T11:19:20.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T11:19:20.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: Deploying daemon mds.k on smithi078 2024-04-20T11:19:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:20 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/297745047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:21 smithi078 ceph-mon[21464]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:21.938 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:21 smithi078 systemd[1]: Starting Ceph mds.k for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:21 smithi044 ceph-mon[16855]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:21 smithi165 ceph-mon[21975]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:22.228 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:21 smithi078 podman[50444]: 2024-04-20 11:19:21.866644819 +0000 UTC m=+0.017720050 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:22.229 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:21 smithi078 podman[50444]: 2024-04-20 11:19:21.985119962 +0000 UTC m=+0.136195180 container create af72995753b688d14cb070cc27330a3be9be3edccfa7aab5e339782c30922547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-20T11:19:22.229 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:22 smithi078 podman[50444]: 2024-04-20 11:19:22.229724364 +0000 UTC m=+0.380799585 container init af72995753b688d14cb070cc27330a3be9be3edccfa7aab5e339782c30922547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T11:19:22.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:22.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 -- 172.21.15.44:0/264875775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 msgr2=0x7fe43010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:22.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 --2- 172.21.15.44:0/264875775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 0x7fe43010a3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe428009ca0 tx=0x7fe42802c0f0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 -- 172.21.15.44:0/264875775 shutdown_connections 2024-04-20T11:19:22.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 --2- 172.21.15.44:0/264875775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 0x7fe43010a3f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 --2- 172.21.15.44:0/264875775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe430103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 --2- 172.21.15.44:0/264875775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe430101fb0 0x7fe4301023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 -- 172.21.15.44:0/264875775 >> 172.21.15.44:0/264875775 conn(0x7fe4300fd540 msgr2=0x7fe4300ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:22.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 -- 172.21.15.44:0/264875775 shutdown_connections 2024-04-20T11:19:22.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.478+0000 7fe435a00640 1 -- 172.21.15.44:0/264875775 wait complete. 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.479+0000 7fe435a00640 1 Processor -- start 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.479+0000 7fe435a00640 1 -- start start 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe435a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe430101fb0 0x7fe43019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe435a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe43019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe435a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 0x7fe4301a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe435a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010dd60 con 0x7fe430103b60 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe435a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010dbe0 con 0x7fe4301031a0 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe435a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010dee0 con 0x7fe430101fb0 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe43019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe43019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:52486/0 (socket says 172.21.15.44:52486) 2024-04-20T11:19:22.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 0x7fe4301a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 -- 172.21.15.44:0/1143551197 learned_addr learned my addr 172.21.15.44:0/1143551197 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42ea00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe430101fb0 0x7fe43019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 -- 172.21.15.44:0/1143551197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe430101fb0 msgr2=0x7fe43019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe430101fb0 0x7fe43019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 -- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 msgr2=0x7fe4301a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 0x7fe4301a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.480+0000 7fe42e000640 1 -- 172.21.15.44:0/1143551197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4301a2160 con 0x7fe4301031a0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.481+0000 7fe42e000640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe43019aa80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00ea60 tx=0x7fe41c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.481+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c00cde0 con 0x7fe4301031a0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.481+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe41c0035f0 con 0x7fe4301031a0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.481+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4301a2450 con 0x7fe4301031a0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.481+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c010720 con 0x7fe4301031a0 2024-04-20T11:19:22.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.481+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301a29f0 con 0x7fe4301031a0 2024-04-20T11:19:22.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.482+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4301a2cb0 con 0x7fe4301031a0 2024-04-20T11:19:22.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.485+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe41c004020 con 0x7fe4301031a0 2024-04-20T11:19:22.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.486+0000 7fe427400640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe4140a3c30 0x7fe4140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:22.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.486+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe41c0d0320 con 0x7fe4301031a0 2024-04-20T11:19:22.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.486+0000 7fe42ea00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe4140a3c30 0x7fe4140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:22.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.487+0000 7fe42ea00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe4140a3c30 0x7fe4140a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4200095f0 tx=0x7fe420005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:22.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.490+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe41c09d100 con 0x7fe4301031a0 2024-04-20T11:19:22.687 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:22 smithi078 podman[50444]: 2024-04-20 11:19:22.232346806 +0000 UTC m=+0.383422024 container start af72995753b688d14cb070cc27330a3be9be3edccfa7aab5e339782c30922547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-20T11:19:22.688 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:22 smithi078 ceph-mds[50463]: -- 172.21.15.78:0/2971113942 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b662ea9a40 con 0x55b662e7cc00 2024-04-20T11:19:22.688 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k[50459]: starting mds.k at 2024-04-20T11:19:22.688 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:22 smithi078 bash[50444]: af72995753b688d14cb070cc27330a3be9be3edccfa7aab5e339782c30922547 2024-04-20T11:19:22.688 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:19:22 smithi078 systemd[1]: Started Ceph mds.k for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.741+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4301a3380 con 0x7fe4301031a0 2024-04-20T11:19:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.742+0000 7fe427400640 1 -- 172.21.15.44:0/1143551197 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fe41c09ca00 con 0x7fe4301031a0 2024-04-20T11:19:22.742 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:22.742 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T11:19:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-04-20T11:19:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.744+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe4140a3c30 msgr2=0x7fe4140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.744+0000 7fe435a00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe4140a3c30 0x7fe4140a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4200095f0 tx=0x7fe420005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.744+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 msgr2=0x7fe43019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.744+0000 7fe435a00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe43019aa80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00ea60 tx=0x7fe41c00ef20 comp rx=0 tx=0).stop 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 shutdown_connections 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe4140a3c30 0x7fe4140a60f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe430103b60 0x7fe4301a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4301031a0 0x7fe43019aa80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 --2- 172.21.15.44:0/1143551197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe430101fb0 0x7fe43019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 >> 172.21.15.44:0/1143551197 conn(0x7fe4300fd540 msgr2=0x7fe430074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 shutdown_connections 2024-04-20T11:19:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:22.745+0000 7fe435a00640 1 -- 172.21.15.44:0/1143551197 wait complete. 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: Deploying daemon mds.j on smithi044 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6832/715928335,v1:172.21.15.78:6833/715928335] up:boot 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: fsmap 2 up:standby 2024-04-20T11:19:23.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:19:23.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:23 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1143551197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:23.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: Deploying daemon mds.j on smithi044 2024-04-20T11:19:23.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6832/715928335,v1:172.21.15.78:6833/715928335] up:boot 2024-04-20T11:19:23.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: fsmap 2 up:standby 2024-04-20T11:19:23.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:19:23.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:23 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1143551197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: Deploying daemon mds.j on smithi044 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6832/715928335,v1:172.21.15.78:6833/715928335] up:boot 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: fsmap 2 up:standby 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:19:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:23 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1143551197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:24.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:24.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 -- 172.21.15.44:0/1744493508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc105bb0 msgr2=0x7f45cc10c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:24.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 --2- 172.21.15.44:0/1744493508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc105bb0 0x7f45cc10c440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ca0 tx=0x7f45c402c100 comp rx=0 tx=0).stop 2024-04-20T11:19:24.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 -- 172.21.15.44:0/1744493508 shutdown_connections 2024-04-20T11:19:24.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 --2- 172.21.15.44:0/1744493508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc105bb0 0x7f45cc10c440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:24.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 --2- 172.21.15.44:0/1744493508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc1051f0 0x7f45cc105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:24.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 --2- 172.21.15.44:0/1744493508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:24.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 -- 172.21.15.44:0/1744493508 >> 172.21.15.44:0/1744493508 conn(0x7f45cc0ff570 msgr2=0x7f45cc1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:24.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 -- 172.21.15.44:0/1744493508 shutdown_connections 2024-04-20T11:19:24.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.989+0000 7f45d2600640 1 -- 172.21.15.44:0/1744493508 wait complete. 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.990+0000 7f45d2600640 1 Processor -- start 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.990+0000 7f45d2600640 1 -- start start 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1051f0 0x7f45cc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc105bb0 0x7f45cc1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d2600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc10fdb0 con 0x7f45cc1051f0 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d2600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc10fc30 con 0x7f45cc105bb0 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d2600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc10ff30 con 0x7f45cc104000 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58482/0 (socket says 172.21.15.44:58482) 2024-04-20T11:19:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d1200640 1 -- 172.21.15.44:0/1863640501 learned_addr learned my addr 172.21.15.44:0/1863640501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45d1c00640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc105bb0 0x7f45cc1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.991+0000 7f45cbe00640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1051f0 0x7f45cc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45d1200640 1 -- 172.21.15.44:0/1863640501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc105bb0 msgr2=0x7f45cc1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45d1200640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc105bb0 0x7f45cc1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45d1200640 1 -- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1051f0 msgr2=0x7f45cc19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45d1200640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1051f0 0x7f45cc19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45d1200640 1 -- 172.21.15.44:0/1863640501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc1a1f90 con 0x7f45cc104000 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45cbe00640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1051f0 0x7f45cc19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:24.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.992+0000 7f45d1c00640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc105bb0 0x7f45cc1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:24.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.993+0000 7f45d1200640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc19a370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9c0 tx=0x7f45c000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:24.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.993+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c000cd80 con 0x7f45cc104000 2024-04-20T11:19:24.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.993+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f45c00035f0 con 0x7f45cc104000 2024-04-20T11:19:24.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.993+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc1a2280 con 0x7f45cc104000 2024-04-20T11:19:24.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.993+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0010670 con 0x7f45cc104000 2024-04-20T11:19:24.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.993+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1a27d0 con 0x7f45cc104000 2024-04-20T11:19:24.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.994+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc10b370 con 0x7f45cc104000 2024-04-20T11:19:24.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.996+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f45c00037a0 con 0x7f45cc104000 2024-04-20T11:19:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.998+0000 7f45c9600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45b40a3c30 0x7f45b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.998+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f45c00d0290 con 0x7f45cc104000 2024-04-20T11:19:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.998+0000 7f45cbe00640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45b40a3c30 0x7f45b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:24.999+0000 7f45cbe00640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45b40a3c30 0x7f45b40a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19b880 tx=0x7f45b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.000+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f45c009d070 con 0x7f45cc104000 2024-04-20T11:19:25.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.278+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45cc02d5d0 con 0x7f45cc104000 2024-04-20T11:19:25.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.279+0000 7f45c9600640 1 -- 172.21.15.44:0/1863640501 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f45c009c970 con 0x7f45cc104000 2024-04-20T11:19:25.280 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:25.280 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T11:19:25.280 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-04-20T11:19:25.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.281+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45b40a3c30 msgr2=0x7f45b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:25.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.281+0000 7f45d2600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45b40a3c30 0x7f45b40a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19b880 tx=0x7f45b8005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 msgr2=0x7f45cc19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc19a370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9c0 tx=0x7f45c000ee80 comp rx=0 tx=0).stop 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 shutdown_connections 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45b40a3c30 0x7f45b40a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45cc105bb0 0x7f45cc1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45cc1051f0 0x7f45cc19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 --2- 172.21.15.44:0/1863640501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45cc104000 0x7f45cc19a370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 >> 172.21.15.44:0/1863640501 conn(0x7f45cc0ff570 msgr2=0x7f45cc108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 shutdown_connections 2024-04-20T11:19:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:25.282+0000 7f45d2600640 1 -- 172.21.15.44:0/1863640501 wait complete. 2024-04-20T11:19:25.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:25 smithi044 ceph-mon[16855]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:25.605 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:25 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1863640501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:25.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:25 smithi078 ceph-mon[21464]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:25.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:25 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1863640501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:25.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:25 smithi165 ceph-mon[21975]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:25.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:25 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1863640501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:25.945 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:25 smithi044 systemd[1]: Starting Ceph mds.j for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:26.305 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:25 smithi044 podman[56662]: 2024-04-20 11:19:25.882062687 +0000 UTC m=+0.039609916 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:26.305 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 podman[56662]: 2024-04-20 11:19:26.017344258 +0000 UTC m=+0.174891515 container create 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-20T11:19:26.305 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 podman[56662]: 2024-04-20 11:19:26.305228974 +0000 UTC m=+0.462776238 container init 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-20T11:19:26.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 podman[56662]: 2024-04-20 11:19:26.316801759 +0000 UTC m=+0.474349036 container start 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T11:19:26.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 ceph-mds[56681]: -- 172.21.15.44:0/743494947 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a79e69a40 con 0x560a79e3c800 2024-04-20T11:19:26.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j[56677]: starting mds.j at 2024-04-20T11:19:26.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 bash[56662]: 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b 2024-04-20T11:19:26.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:19:26 smithi044 systemd[1]: Started Ceph mds.j for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:27.302 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:27.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.516+0000 7f6a65c00640 1 -- 172.21.15.44:0/1856837494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601020f0 msgr2=0x7f6a60102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.516+0000 7f6a65c00640 1 --2- 172.21.15.44:0/1856837494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601020f0 0x7f6a60102570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009860 tx=0x7f6a4802bb70 comp rx=0 tx=0).stop 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 -- 172.21.15.44:0/1856837494 shutdown_connections 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 --2- 172.21.15.44:0/1856837494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a60102ab0 0x7f6a60109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 --2- 172.21.15.44:0/1856837494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601020f0 0x7f6a60102570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 --2- 172.21.15.44:0/1856837494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a601013e0 0x7f6a60101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 -- 172.21.15.44:0/1856837494 >> 172.21.15.44:0/1856837494 conn(0x7f6a600fbbd0 msgr2=0x7f6a600fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 -- 172.21.15.44:0/1856837494 shutdown_connections 2024-04-20T11:19:27.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.517+0000 7f6a65c00640 1 -- 172.21.15.44:0/1856837494 wait complete. 2024-04-20T11:19:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 Processor -- start 2024-04-20T11:19:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 -- start start 2024-04-20T11:19:27.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 0x7f6a6019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 0x7f6a6019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60102ab0 0x7f6a601a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010ddb0 con 0x7f6a601020f0 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010dc30 con 0x7f6a60102ab0 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.518+0000 7f6a65c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010df30 con 0x7f6a601013e0 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 0x7f6a6019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a5ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 0x7f6a6019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 0x7f6a6019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57068/0 (socket says 172.21.15.44:57068) 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a5ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 0x7f6a6019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58514/0 (socket says 172.21.15.44:58514) 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 -- 172.21.15.44:0/3448922169 learned_addr learned my addr 172.21.15.44:0/3448922169 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a5f400640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60102ab0 0x7f6a601a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 -- 172.21.15.44:0/3448922169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 msgr2=0x7f6a6019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 0x7f6a6019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 -- 172.21.15.44:0/3448922169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60102ab0 msgr2=0x7f6a601a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60102ab0 0x7f6a601a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a56000640 1 -- 172.21.15.44:0/3448922169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a601a2270 con 0x7f6a601020f0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.519+0000 7f6a5ea00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 0x7f6a6019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.520+0000 7f6a56000640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 0x7f6a6019ab90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a48037040 tx=0x7f6a48035f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.520+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48041020 con 0x7f6a601020f0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.520+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a601a2500 con 0x7f6a601020f0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.520+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a4802ca10 con 0x7f6a601020f0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.520+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48039040 con 0x7f6a601020f0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.520+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a2a40 con 0x7f6a601020f0 2024-04-20T11:19:27.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.521+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a601a2ca0 con 0x7f6a601020f0 2024-04-20T11:19:27.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.525+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a48044020 con 0x7f6a601020f0 2024-04-20T11:19:27.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.527+0000 7f6a57e00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6a400a3c30 0x7f6a400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:27.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.527+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6a480f17e0 con 0x7f6a601020f0 2024-04-20T11:19:27.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.527+0000 7f6a5ea00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6a400a3c30 0x7f6a400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:27.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.528+0000 7f6a5ea00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6a400a3c30 0x7f6a400a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a6006a130 tx=0x7f6a50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:27.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.532+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6a4804b020 con 0x7f6a601020f0 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6834/858649103,v1:172.21.15.44:6835/858649103] up:boot 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: fsmap 3 up:standby 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:27.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:27 smithi044 ceph-mon[16855]: Deploying daemon mds.d on smithi044 2024-04-20T11:19:27.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6834/858649103,v1:172.21.15.44:6835/858649103] up:boot 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: fsmap 3 up:standby 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:27 smithi078 ceph-mon[21464]: Deploying daemon mds.d on smithi044 2024-04-20T11:19:27.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:27.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6834/858649103,v1:172.21.15.44:6835/858649103] up:boot 2024-04-20T11:19:27.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: fsmap 3 up:standby 2024-04-20T11:19:27.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:27 smithi165 ceph-mon[21975]: Deploying daemon mds.d on smithi044 2024-04-20T11:19:27.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.790+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a601a2e70 con 0x7f6a601020f0 2024-04-20T11:19:27.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.791+0000 7f6a57e00640 1 -- 172.21.15.44:0/3448922169 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f6a480be5c0 con 0x7f6a601020f0 2024-04-20T11:19:27.792 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:27.792 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-20T11:19:27.793 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 4 2024-04-20T11:19:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6a400a3c30 msgr2=0x7f6a400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6a400a3c30 0x7f6a400a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a6006a130 tx=0x7f6a50005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 msgr2=0x7f6a6019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 0x7f6a6019ab90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a48037040 tx=0x7f6a48035f10 comp rx=0 tx=0).stop 2024-04-20T11:19:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 shutdown_connections 2024-04-20T11:19:27.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6a400a3c30 0x7f6a400a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60102ab0 0x7f6a601a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a601020f0 0x7f6a6019ab90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 --2- 172.21.15.44:0/3448922169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a601013e0 0x7f6a6019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:27.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 >> 172.21.15.44:0/3448922169 conn(0x7f6a600fbbd0 msgr2=0x7f6a60109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:27.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 shutdown_connections 2024-04-20T11:19:27.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:27.796+0000 7f6a65c00640 1 -- 172.21.15.44:0/3448922169 wait complete. 2024-04-20T11:19:28.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:28 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3448922169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:28.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:28 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3448922169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:28.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:28 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3448922169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:29.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:29 smithi165 ceph-mon[21975]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:29.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:29.865 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:29 smithi044 ceph-mon[16855]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:29 smithi078 ceph-mon[21464]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:30.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 -- 172.21.15.44:0/2351601954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038102f60 msgr2=0x7fe0381033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 --2- 172.21.15.44:0/2351601954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038102f60 0x7fe0381033e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe02402bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 -- 172.21.15.44:0/2351601954 shutdown_connections 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 --2- 172.21.15.44:0/2351601954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038103920 0x7fe03810a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 --2- 172.21.15.44:0/2351601954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038102f60 0x7fe0381033e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 --2- 172.21.15.44:0/2351601954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038101e10 0x7fe038102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 -- 172.21.15.44:0/2351601954 >> 172.21.15.44:0/2351601954 conn(0x7fe0380fd3a0 msgr2=0x7fe0380ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 -- 172.21.15.44:0/2351601954 shutdown_connections 2024-04-20T11:19:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.035+0000 7fe03f800640 1 -- 172.21.15.44:0/2351601954 wait complete. 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.036+0000 7fe03f800640 1 Processor -- start 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.036+0000 7fe03f800640 1 -- start start 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038101e10 0x7fe03819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03f800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 0x7fe03819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03f800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 0x7fe0381a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810db90 con 0x7fe038101e10 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03f800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810da10 con 0x7fe038102f60 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03f800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810dd10 con 0x7fe038103920 2024-04-20T11:19:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe037e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 0x7fe03819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 0x7fe0381a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe037e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 0x7fe03819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55178/0 (socket says 172.21.15.44:55178) 2024-04-20T11:19:30.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 0x7fe0381a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41852/0 (socket says 172.21.15.44:41852) 2024-04-20T11:19:30.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe037e00640 1 -- 172.21.15.44:0/3857456989 learned_addr learned my addr 172.21.15.44:0/3857456989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:30.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.037+0000 7fe03ce00640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038101e10 0x7fe03819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe037e00640 1 -- 172.21.15.44:0/3857456989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 msgr2=0x7fe0381a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe037e00640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 0x7fe0381a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe037e00640 1 -- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038101e10 msgr2=0x7fe03819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe037e00640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038101e10 0x7fe03819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe037e00640 1 -- 172.21.15.44:0/3857456989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0381a2030 con 0x7fe038102f60 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe03d800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 0x7fe0381a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.038+0000 7fe037e00640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 0x7fe03819a950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe024000c00 tx=0x7fe024004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.039+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024043020 con 0x7fe038102f60 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.039+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0381a22c0 con 0x7fe038102f60 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.039+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe024004250 con 0x7fe038102f60 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.039+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024003b20 con 0x7fe038102f60 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.039+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0381a2860 con 0x7fe038102f60 2024-04-20T11:19:30.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.040+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0381a2b80 con 0x7fe038102f60 2024-04-20T11:19:30.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.043+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe024046020 con 0x7fe038102f60 2024-04-20T11:19:30.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.045+0000 7fe035600640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0200a3bb0 0x7fe0200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:30.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.045+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe0240f26c0 con 0x7fe038102f60 2024-04-20T11:19:30.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.045+0000 7fe03ce00640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0200a3bb0 0x7fe0200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:30.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.046+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe02404d020 con 0x7fe038102f60 2024-04-20T11:19:30.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.046+0000 7fe03ce00640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0200a3bb0 0x7fe0200a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009bd0 tx=0x7fe02c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.306+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0381a2d50 con 0x7fe038102f60 2024-04-20T11:19:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.307+0000 7fe035600640 1 -- 172.21.15.44:0/3857456989 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fe0240bf4a0 con 0x7fe038102f60 2024-04-20T11:19:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:30.308 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-20T11:19:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 4 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0200a3bb0 msgr2=0x7fe0200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0200a3bb0 0x7fe0200a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009bd0 tx=0x7fe02c005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 msgr2=0x7fe03819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 0x7fe03819a950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe024000c00 tx=0x7fe024004a30 comp rx=0 tx=0).stop 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 shutdown_connections 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0200a3bb0 0x7fe0200a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038103920 0x7fe0381a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038102f60 0x7fe03819a950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 --2- 172.21.15.44:0/3857456989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe038101e10 0x7fe03819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 >> 172.21.15.44:0/3857456989 conn(0x7fe0380fd3a0 msgr2=0x7fe038074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 shutdown_connections 2024-04-20T11:19:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:30.310+0000 7fe03f800640 1 -- 172.21.15.44:0/3857456989 wait complete. 2024-04-20T11:19:30.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:30 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3857456989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:30.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:30 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3857456989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:30.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:30 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3857456989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:31.195 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:19:30 smithi044 systemd[1]: Started Ceph mds.d for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:31.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:31.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:31.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:31.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:31 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:31.782 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:31 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:31.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:31.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:31.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:31.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:31.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:31 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:32.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:32.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.558+0000 7fa6ffe00640 1 -- 172.21.15.44:0/4173388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 msgr2=0x7fa700105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.558+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/4173388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa700105650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009900 tx=0x7fa6e802bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 -- 172.21.15.44:0/4173388693 shutdown_connections 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/4173388693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 0x7fa70010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/4173388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa700105650 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/4173388693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 0x7fa700104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 -- 172.21.15.44:0/4173388693 >> 172.21.15.44:0/4173388693 conn(0x7fa7000ff570 msgr2=0x7fa700101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 -- 172.21.15.44:0/4173388693 shutdown_connections 2024-04-20T11:19:32.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.559+0000 7fa6ffe00640 1 -- 172.21.15.44:0/4173388693 wait complete. 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 Processor -- start 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 -- start start 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 0x7fa70019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa70019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 0x7fa7001a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa70010fd40 con 0x7fa700103fe0 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa70010fbc0 con 0x7fa700105b90 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.560+0000 7fa6ffe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa70010fec0 con 0x7fa7001051d0 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa70019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:32.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa70019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41872/0 (socket says 172.21.15.44:41872) 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 -- 172.21.15.44:0/1342789411 learned_addr learned my addr 172.21.15.44:0/1342789411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6ff400640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 0x7fa7001a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6fea00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 0x7fa70019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 -- 172.21.15.44:0/1342789411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 msgr2=0x7fa7001a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 0x7fa7001a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 -- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 msgr2=0x7fa70019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 0x7fa70019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 -- 172.21.15.44:0/1342789411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7001a2010 con 0x7fa7001051d0 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6fea00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 0x7fa70019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6ff400640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 0x7fa7001a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.561+0000 7fa6f6000640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa70019a930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009230 tx=0x7fa6e8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.562+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8043020 con 0x7fa7001051d0 2024-04-20T11:19:32.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.562+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7001a22a0 con 0x7fa7001051d0 2024-04-20T11:19:32.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.562+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6e8004250 con 0x7fa7001051d0 2024-04-20T11:19:32.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.562+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003b20 con 0x7fa7001051d0 2024-04-20T11:19:32.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.562+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7001a2840 con 0x7fa7001051d0 2024-04-20T11:19:32.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.563+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70010b2a0 con 0x7fa7001051d0 2024-04-20T11:19:32.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.565+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6e8046020 con 0x7fa7001051d0 2024-04-20T11:19:32.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.566+0000 7fa6f7e00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6e40a3c30 0x7fa6e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:32.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.567+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa6e80f2110 con 0x7fa7001051d0 2024-04-20T11:19:32.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.567+0000 7fa6fea00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6e40a3c30 0x7fa6e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:32.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.567+0000 7fa6fea00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6e40a3c30 0x7fa6e40a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa700105030 tx=0x7fa6f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:32.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.571+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6e804d020 con 0x7fa7001051d0 2024-04-20T11:19:32.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:32 smithi044 ceph-mon[16855]: Deploying daemon mds.e on smithi078 2024-04-20T11:19:32.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:32 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] up:boot 2024-04-20T11:19:32.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:32 smithi044 ceph-mon[16855]: fsmap 4 up:standby 2024-04-20T11:19:32.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:19:32.731 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:32 smithi078 ceph-mon[21464]: Deploying daemon mds.e on smithi078 2024-04-20T11:19:32.731 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:32 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] up:boot 2024-04-20T11:19:32.731 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:32 smithi078 ceph-mon[21464]: fsmap 4 up:standby 2024-04-20T11:19:32.731 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:19:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.821+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa700062b60 con 0x7fa7001051d0 2024-04-20T11:19:32.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.822+0000 7fa6f7e00640 1 -- 172.21.15.44:0/1342789411 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fa6e80beef0 con 0x7fa7001051d0 2024-04-20T11:19:32.823 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:32.823 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-20T11:19:32.823 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 5 2024-04-20T11:19:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.824+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6e40a3c30 msgr2=0x7fa6e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.824+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6e40a3c30 0x7fa6e40a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa700105030 tx=0x7fa6f0005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.824+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 msgr2=0x7fa70019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.824+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa70019a930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009230 tx=0x7fa6e8004ab0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 shutdown_connections 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6e40a3c30 0x7fa6e40a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa700105b90 0x7fa7001a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7001051d0 0x7fa70019a930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 --2- 172.21.15.44:0/1342789411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa700103fe0 0x7fa70019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 >> 172.21.15.44:0/1342789411 conn(0x7fa7000ff570 msgr2=0x7fa700108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 shutdown_connections 2024-04-20T11:19:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:32.825+0000 7fa6ffe00640 1 -- 172.21.15.44:0/1342789411 wait complete. 2024-04-20T11:19:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:32 smithi165 ceph-mon[21975]: Deploying daemon mds.e on smithi078 2024-04-20T11:19:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:32 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] up:boot 2024-04-20T11:19:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:32 smithi165 ceph-mon[21975]: fsmap 4 up:standby 2024-04-20T11:19:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:19:33.426 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 systemd[1]: Starting Ceph mds.e for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:33.684 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:33 smithi078 ceph-mon[21464]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:33.684 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:33 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1342789411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:33.684 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 podman[50769]: 2024-04-20 11:19:33.329331953 +0000 UTC m=+0.017756400 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:33.684 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 podman[50769]: 2024-04-20 11:19:33.440362729 +0000 UTC m=+0.128787162 container create 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T11:19:33.880 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:33 smithi165 ceph-mon[21975]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:33.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:33 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1342789411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:33.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 podman[50769]: 2024-04-20 11:19:33.684997057 +0000 UTC m=+0.373421491 container init 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T11:19:33.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 podman[50769]: 2024-04-20 11:19:33.687472758 +0000 UTC m=+0.375897202 container start 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:19:33.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 ceph-mds[50788]: -- 172.21.15.78:0/569758645 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56478fd95a40 con 0x56478fd68800 2024-04-20T11:19:33.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e[50784]: starting mds.e at 2024-04-20T11:19:33.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 bash[50769]: 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c 2024-04-20T11:19:33.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:19:33 smithi078 systemd[1]: Started Ceph mds.e for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:33.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:33 smithi044 ceph-mon[16855]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:33.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:33 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1342789411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:34.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:34.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: Deploying daemon mds.f on smithi165 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] up:boot 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: fsmap 5 up:standby 2024-04-20T11:19:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:34 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 -- 172.21.15.44:0/4240193194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70075f40 msgr2=0x7f0c70074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/4240193194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70075f40 0x7f0c70074390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009ca0 tx=0x7f0c6002c190 comp rx=0 tx=0).stop 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 -- 172.21.15.44:0/4240193194 shutdown_connections 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/4240193194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70109c90 0x7f0c7010c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/4240193194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c70074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/4240193194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70075f40 0x7f0c70074390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 -- 172.21.15.44:0/4240193194 >> 172.21.15.44:0/4240193194 conn(0x7f0c700fd370 msgr2=0x7f0c700ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 -- 172.21.15.44:0/4240193194 shutdown_connections 2024-04-20T11:19:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.071+0000 7f0c6fe00640 1 -- 172.21.15.44:0/4240193194 wait complete. 2024-04-20T11:19:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 Processor -- start 2024-04-20T11:19:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 -- start start 2024-04-20T11:19:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c7019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70075f40 0x7f0c7019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70109c90 0x7f0c701a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7010fdb0 con 0x7f0c70074960 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7010fc30 con 0x7f0c70075f40 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.072+0000 7f0c6fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7010ff30 con 0x7f0c70109c90 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c7019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70109c90 0x7f0c701a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c7019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51208/0 (socket says 172.21.15.44:51208) 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 -- 172.21.15.44:0/3074921452 learned_addr learned my addr 172.21.15.44:0/3074921452 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c66000640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70075f40 0x7f0c7019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 -- 172.21.15.44:0/3074921452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70109c90 msgr2=0x7f0c701a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70109c90 0x7f0c701a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 -- 172.21.15.44:0/3074921452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70075f40 msgr2=0x7f0c7019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70075f40 0x7f0c7019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 -- 172.21.15.44:0/3074921452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c701a1fc0 con 0x7f0c70074960 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c66000640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70075f40 0x7f0c7019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.073+0000 7f0c6ea00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c7019a3a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0c60032780 tx=0x7f0c60034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.074+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60039040 con 0x7f0c70074960 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.074+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c701a2250 con 0x7f0c70074960 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.074+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0c60044070 con 0x7f0c70074960 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.074+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600044e0 con 0x7f0c70074960 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.074+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c701a2760 con 0x7f0c70074960 2024-04-20T11:19:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.075+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c70108a50 con 0x7f0c70074960 2024-04-20T11:19:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.077+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c60004680 con 0x7f0c70074960 2024-04-20T11:19:35.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.079+0000 7f0c67e00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c540a3bb0 0x7f0c540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:35.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.079+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0c600f2ff0 con 0x7f0c70074960 2024-04-20T11:19:35.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.079+0000 7f0c66000640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c540a3bb0 0x7f0c540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.080+0000 7f0c66000640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c540a3bb0 0x7f0c540a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c7019b8b0 tx=0x7f0c58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:35.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.082+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c600bfec0 con 0x7f0c70074960 2024-04-20T11:19:35.147 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: Deploying daemon mds.f on smithi165 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] up:boot 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: fsmap 5 up:standby 2024-04-20T11:19:35.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:34 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:19:35.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:35.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:35.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:35.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: Deploying daemon mds.f on smithi165 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] up:boot 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: fsmap 5 up:standby 2024-04-20T11:19:35.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:34 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:19:35.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.330+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c7002d5d0 con 0x7f0c70074960 2024-04-20T11:19:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.331+0000 7f0c67e00640 1 -- 172.21.15.44:0/3074921452 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f0c600bf7c0 con 0x7f0c70074960 2024-04-20T11:19:35.332 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:35.332 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-20T11:19:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 6 2024-04-20T11:19:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.334+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c540a3bb0 msgr2=0x7f0c540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.334+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c540a3bb0 0x7f0c540a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c7019b8b0 tx=0x7f0c58005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.334+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 msgr2=0x7f0c7019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.334+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c7019a3a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0c60032780 tx=0x7f0c60034760 comp rx=0 tx=0).stop 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 shutdown_connections 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c540a3bb0 0x7f0c540a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c70109c90 0x7f0c701a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c70075f40 0x7f0c7019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 --2- 172.21.15.44:0/3074921452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c70074960 0x7f0c7019a3a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 >> 172.21.15.44:0/3074921452 conn(0x7f0c700fd370 msgr2=0x7f0c70105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 shutdown_connections 2024-04-20T11:19:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:35.335+0000 7f0c6fe00640 1 -- 172.21.15.44:0/3074921452 wait complete. 2024-04-20T11:19:36.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:35 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3074921452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:36.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:35 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3074921452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:36.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:35 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3074921452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:36.230 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 systemd[1]: Starting Ceph mds.f for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:36.609 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 podman[52370]: 2024-04-20 11:19:36.339238695 +0000 UTC m=+0.114326468 container create 76a599a47916ad8810d008eb1c18be252f30a621fea2ba22b3790d5acc69c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T11:19:36.609 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 podman[52370]: 2024-04-20 11:19:36.243206096 +0000 UTC m=+0.018293847 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:36.609 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 podman[52370]: 2024-04-20 11:19:36.583442179 +0000 UTC m=+0.358529906 container init 76a599a47916ad8810d008eb1c18be252f30a621fea2ba22b3790d5acc69c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-20T11:19:36.609 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 podman[52370]: 2024-04-20 11:19:36.586178436 +0000 UTC m=+0.361266172 container start 76a599a47916ad8810d008eb1c18be252f30a621fea2ba22b3790d5acc69c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T11:19:36.610 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 ceph-mds[52389]: -- 172.21.15.165:0/2827477808 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561aa0f37a40 con 0x561aa0f0b000 2024-04-20T11:19:36.979 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f[52385]: starting mds.f at 2024-04-20T11:19:36.979 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 bash[52370]: 76a599a47916ad8810d008eb1c18be252f30a621fea2ba22b3790d5acc69c9ab 2024-04-20T11:19:36.980 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:19:36 smithi165 systemd[1]: Started Ceph mds.f for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:37.357 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 -- 172.21.15.44:0/1408165913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 msgr2=0x7f6408105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 --2- 172.21.15.44:0/1408165913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 0x7f6408105830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009860 tx=0x7f63f002bb70 comp rx=0 tx=0).stop 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 -- 172.21.15.44:0/1408165913 shutdown_connections 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 --2- 172.21.15.44:0/1408165913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6408105d70 0x7f640810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 --2- 172.21.15.44:0/1408165913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 0x7f6408105830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 --2- 172.21.15.44:0/1408165913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f64081045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 -- 172.21.15.44:0/1408165913 >> 172.21.15.44:0/1408165913 conn(0x7f64080ff750 msgr2=0x7f6408101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 -- 172.21.15.44:0/1408165913 shutdown_connections 2024-04-20T11:19:37.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.581+0000 7f640ea00640 1 -- 172.21.15.44:0/1408165913 wait complete. 2024-04-20T11:19:37.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.582+0000 7f640ea00640 1 Processor -- start 2024-04-20T11:19:37.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.582+0000 7f640ea00640 1 -- start start 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f640819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 0x7f640819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6408105d70 0x7f64081a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f640810ff70 con 0x7f64081041c0 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ea00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f640810fdf0 con 0x7f64081053b0 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64081100f0 con 0x7f6408105d70 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f640819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f640819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51228/0 (socket says 172.21.15.44:51228) 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 -- 172.21.15.44:0/194114504 learned_addr learned my addr 172.21.15.44:0/194114504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f640ca00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6408105d70 0x7f64081a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f63ff400640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 0x7f640819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 -- 172.21.15.44:0/194114504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6408105d70 msgr2=0x7f64081a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6408105d70 0x7f64081a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 -- 172.21.15.44:0/194114504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 msgr2=0x7f640819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 0x7f640819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.583+0000 7f6407e00640 1 -- 172.21.15.44:0/194114504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64081a2180 con 0x7f64081041c0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.584+0000 7f6407e00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f640819a560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9a0 tx=0x7f63f800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.584+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800cd80 con 0x7f64081041c0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.584+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64081a2470 con 0x7f64081041c0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.584+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f63f8010930 con 0x7f64081041c0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.584+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64081a29e0 con 0x7f64081041c0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.584+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8018a40 con 0x7f64081041c0 2024-04-20T11:19:37.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.585+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640810b480 con 0x7f64081041c0 2024-04-20T11:19:37.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.587+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f63f8010ae0 con 0x7f64081041c0 2024-04-20T11:19:37.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.589+0000 7f6405600640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:37.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.589+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f63f80d02f0 con 0x7f64081041c0 2024-04-20T11:19:37.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.589+0000 7f63ff400640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:37.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.590+0000 7f63ff400640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009830 tx=0x7f63f0034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:37.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.593+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f63f8053ff0 con 0x7f64081041c0 2024-04-20T11:19:37.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.834+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f640802d5d0 con 0x7f64081041c0 2024-04-20T11:19:37.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.835+0000 7f6405600640 1 -- 172.21.15.44:0/194114504 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f63f809d0d0 con 0x7f64081041c0 2024-04-20T11:19:37.836 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:37.836 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-20T11:19:37.836 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2024-04-20T11:19:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.837+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f63ec0a3bb0 msgr2=0x7f63ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.837+0000 7f640ea00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009830 tx=0x7f63f0034b50 comp rx=0 tx=0).stop 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.837+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 msgr2=0x7f640819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.837+0000 7f640ea00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f640819a560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9a0 tx=0x7f63f800ee60 comp rx=0 tx=0).stop 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 shutdown_connections 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009830 tx=0x7f63f0034b50 comp rx=0 tx=0).stop 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6408105d70 0x7f64081a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64081053b0 0x7f640819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 --2- 172.21.15.44:0/194114504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64081041c0 0x7f640819a560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 >> 172.21.15.44:0/194114504 conn(0x7f64080ff750 msgr2=0x7f6408108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 shutdown_connections 2024-04-20T11:19:37.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:37.838+0000 7f640ea00640 1 -- 172.21.15.44:0/194114504 wait complete. 2024-04-20T11:19:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6834/1721181859,v1:172.21.15.165:6835/1721181859] up:boot 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: fsmap 6 up:standby 2024-04-20T11:19:37.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:37 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:38.008 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6834/1721181859,v1:172.21.15.165:6835/1721181859] up:boot 2024-04-20T11:19:38.009 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: fsmap 6 up:standby 2024-04-20T11:19:38.009 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:37 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:19:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:38.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:38.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6834/1721181859,v1:172.21.15.165:6835/1721181859] up:boot 2024-04-20T11:19:38.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: fsmap 6 up:standby 2024-04-20T11:19:38.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:37 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:19:39.051 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:38 smithi165 ceph-mon[21975]: Deploying daemon mds.c on smithi165 2024-04-20T11:19:39.052 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:38 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/194114504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:39.181 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:38 smithi078 ceph-mon[21464]: Deploying daemon mds.c on smithi165 2024-04-20T11:19:39.181 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:38 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/194114504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:39.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:38 smithi044 ceph-mon[16855]: Deploying daemon mds.c on smithi165 2024-04-20T11:19:39.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:38 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/194114504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:39.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:39.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:39 smithi165 ceph-mon[21975]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:39.991 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:19:39 smithi165 systemd[1]: Started Ceph mds.c for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 -- 172.21.15.44:0/308664514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520107ac0 msgr2=0x7f7520109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 --2- 172.21.15.44:0/308664514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520107ac0 0x7f7520109f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7518009ca0 tx=0x7f751802c100 comp rx=0 tx=0).stop 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 -- 172.21.15.44:0/308664514 shutdown_connections 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 --2- 172.21.15.44:0/308664514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520107ac0 0x7f7520109f10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 --2- 172.21.15.44:0/308664514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520105190 0x7f7520107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 --2- 172.21.15.44:0/308664514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 0x7f7520104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 -- 172.21.15.44:0/308664514 >> 172.21.15.44:0/308664514 conn(0x7f75200fc230 msgr2=0x7f75200fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 -- 172.21.15.44:0/308664514 shutdown_connections 2024-04-20T11:19:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.081+0000 7f7525000640 1 -- 172.21.15.44:0/308664514 wait complete. 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 Processor -- start 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 -- start start 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 0x7f752019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 0x7f752019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520107ac0 0x7f75201a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f752010dbb0 con 0x7f7520107ac0 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f752010da30 con 0x7f7520105190 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.082+0000 7f7525000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f752010dd30 con 0x7f7520102860 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f751fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520107ac0 0x7f75201a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 0x7f752019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:40.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f751f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 0x7f752019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:40.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 0x7f752019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:38198/0 (socket says 172.21.15.44:38198) 2024-04-20T11:19:40.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 -- 172.21.15.44:0/460305411 learned_addr learned my addr 172.21.15.44:0/460305411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:40.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 -- 172.21.15.44:0/460305411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 msgr2=0x7f752019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:40.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 0x7f752019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 -- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520107ac0 msgr2=0x7f75201a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520107ac0 0x7f75201a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 -- 172.21.15.44:0/460305411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75201a1fe0 con 0x7f7520105190 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f751fe00640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520107ac0 0x7f75201a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f751f400640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 0x7f752019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.083+0000 7f7516a00640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 0x7f752019a900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f750800e9c0 tx=0x7f750800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.084+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750800cd80 con 0x7f7520105190 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.084+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75201a22d0 con 0x7f7520105190 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.084+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7508005520 con 0x7f7520105190 2024-04-20T11:19:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.084+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508010690 con 0x7f7520105190 2024-04-20T11:19:40.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.084+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75201a2870 con 0x7f7520105190 2024-04-20T11:19:40.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.085+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75201095b0 con 0x7f7520105190 2024-04-20T11:19:40.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.087+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f75080046c0 con 0x7f7520105190 2024-04-20T11:19:40.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.089+0000 7f751cc00640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75040a3bb0 0x7f75040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:40.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.089+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f75080d0370 con 0x7f7520105190 2024-04-20T11:19:40.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.089+0000 7f751f400640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75040a3bb0 0x7f75040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:40.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.090+0000 7f751f400640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75040a3bb0 0x7f75040a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7510009fc0 tx=0x7f7510005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:40.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.096+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f750809d150 con 0x7f7520105190 2024-04-20T11:19:40.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:39 smithi078 ceph-mon[21464]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:40.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:39 smithi044 ceph-mon[16855]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:40.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.344+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f752002d5d0 con 0x7f7520105190 2024-04-20T11:19:40.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.345+0000 7f751cc00640 1 -- 172.21.15.44:0/460305411 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f750809ca50 con 0x7f7520105190 2024-04-20T11:19:40.345 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:40.346 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T11:19:40.346 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-04-20T11:19:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.347+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75040a3bb0 msgr2=0x7f75040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.347+0000 7f7525000640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75040a3bb0 0x7f75040a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7510009fc0 tx=0x7f7510005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.347+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 msgr2=0x7f752019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:40.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.347+0000 7f7525000640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 0x7f752019a900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f750800e9c0 tx=0x7f750800ee80 comp rx=0 tx=0).stop 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.347+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 shutdown_connections 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75040a3bb0 0x7f75040a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7520107ac0 0x7f75201a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7520105190 0x7f752019a900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 --2- 172.21.15.44:0/460305411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7520102860 0x7f752019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 >> 172.21.15.44:0/460305411 conn(0x7f75200fc230 msgr2=0x7f7520101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 shutdown_connections 2024-04-20T11:19:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:40.348+0000 7f7525000640 1 -- 172.21.15.44:0/460305411 wait complete. 2024-04-20T11:19:41.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6836/2414137592,v1:172.21.15.165:6837/2414137592] up:boot 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: fsmap 7 up:standby 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:41.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/460305411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6836/2414137592,v1:172.21.15.165:6837/2414137592] up:boot 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: fsmap 7 up:standby 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:41.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/460305411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6836/2414137592,v1:172.21.15.165:6837/2414137592] up:boot 2024-04-20T11:19:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: fsmap 7 up:standby 2024-04-20T11:19:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:41.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/460305411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:41.850 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:41 smithi078 ceph-mon[21464]: Deploying daemon mds.h on smithi078 2024-04-20T11:19:41.850 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:41 smithi078 ceph-mon[21464]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:42.187 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 systemd[1]: Starting Ceph mds.h for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:42.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:41 smithi044 ceph-mon[16855]: Deploying daemon mds.h on smithi078 2024-04-20T11:19:42.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:41 smithi044 ceph-mon[16855]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:42.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:41 smithi165 ceph-mon[21975]: Deploying daemon mds.h on smithi078 2024-04-20T11:19:42.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:41 smithi165 ceph-mon[21975]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:42.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:42.540 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 podman[51092]: 2024-04-20 11:19:42.155435165 +0000 UTC m=+0.018008936 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:42.540 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 podman[51092]: 2024-04-20 11:19:42.271811885 +0000 UTC m=+0.134385688 container create 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-20T11:19:42.540 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 podman[51092]: 2024-04-20 11:19:42.541098811 +0000 UTC m=+0.403672572 container init 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:19:42.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 -- 172.21.15.44:0/3286596804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 msgr2=0x7f71ac10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 --2- 172.21.15.44:0/3286596804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac10c600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009be0 tx=0x7f71a402bef0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 -- 172.21.15.44:0/3286596804 shutdown_connections 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 --2- 172.21.15.44:0/3286596804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac10c600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 --2- 172.21.15.44:0/3286596804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1053b0 0x7f71ac105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 --2- 172.21.15.44:0/3286596804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ac1041c0 0x7f71ac1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 -- 172.21.15.44:0/3286596804 >> 172.21.15.44:0/3286596804 conn(0x7f71ac0ff750 msgr2=0x7f71ac101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 -- 172.21.15.44:0/3286596804 shutdown_connections 2024-04-20T11:19:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.594+0000 7f71b1a00640 1 -- 172.21.15.44:0/3286596804 wait complete. 2024-04-20T11:19:42.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 Processor -- start 2024-04-20T11:19:42.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 -- start start 2024-04-20T11:19:42.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1041c0 0x7f71ac19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:42.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ac1053b0 0x7f71ac19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:42.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:42.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10ffc0 con 0x7f71ac1041c0 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10fe40 con 0x7f71ac105d70 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.595+0000 7f71b1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac110140 con 0x7f71ac1053b0 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1041c0 0x7f71ac19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1041c0 0x7f71ac19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39318/0 (socket says 172.21.15.44:39318) 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71aaa00640 1 -- 172.21.15.44:0/1503083776 learned_addr learned my addr 172.21.15.44:0/1503083776 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71a2000640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ac1053b0 0x7f71ac19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 -- 172.21.15.44:0/1503083776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ac1053b0 msgr2=0x7f71ac19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ac1053b0 0x7f71ac19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 -- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1041c0 msgr2=0x7f71ac19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1041c0 0x7f71ac19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 -- 172.21.15.44:0/1503083776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac1a2110 con 0x7f71ac105d70 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.596+0000 7f71ab400640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac1a1aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f71a4036040 tx=0x7f71a402cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.597+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4043020 con 0x7f71ac105d70 2024-04-20T11:19:42.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.597+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71a4004040 con 0x7f71ac105d70 2024-04-20T11:19:42.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.597+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac1a23a0 con 0x7f71ac105d70 2024-04-20T11:19:42.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.597+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a403e7d0 con 0x7f71ac105d70 2024-04-20T11:19:42.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.597+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1a2940 con 0x7f71ac105d70 2024-04-20T11:19:42.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.598+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac10b410 con 0x7f71ac105d70 2024-04-20T11:19:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.600+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f71a4046020 con 0x7f71ac105d70 2024-04-20T11:19:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.601+0000 7f71a3e00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f71900a3c30 0x7f71900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.601+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f71a40f2570 con 0x7f71ac105d70 2024-04-20T11:19:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.602+0000 7f71aaa00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f71900a3c30 0x7f71900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.602+0000 7f71aaa00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f71900a3c30 0x7f71900a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f719c0096c0 tx=0x7f719c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.606+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f71a404d020 con 0x7f71ac105d70 2024-04-20T11:19:42.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.863+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71ac062b60 con 0x7f71ac105d70 2024-04-20T11:19:42.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.864+0000 7f71a3e00640 1 -- 172.21.15.44:0/1503083776 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f71a40bf350 con 0x7f71ac105d70 2024-04-20T11:19:42.864 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:42.865 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-20T11:19:42.865 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-04-20T11:19:42.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f71900a3c30 msgr2=0x7f71900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:42.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f71900a3c30 0x7f71900a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f719c0096c0 tx=0x7f719c009260 comp rx=0 tx=0).stop 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 msgr2=0x7f71ac1a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac1a1aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f71a4036040 tx=0x7f71a402cbc0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 shutdown_connections 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f71900a3c30 0x7f71900a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac105d70 0x7f71ac1a1aa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ac1053b0 0x7f71ac19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 --2- 172.21.15.44:0/1503083776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ac1041c0 0x7f71ac19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 >> 172.21.15.44:0/1503083776 conn(0x7f71ac0ff750 msgr2=0x7f71ac1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 shutdown_connections 2024-04-20T11:19:42.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:42.867+0000 7f71b1a00640 1 -- 172.21.15.44:0/1503083776 wait complete. 2024-04-20T11:19:42.937 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 podman[51092]: 2024-04-20 11:19:42.543682566 +0000 UTC m=+0.406256337 container start 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-20T11:19:42.937 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 ceph-mds[51111]: -- 172.21.15.78:0/2838762992 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561471ed1a40 con 0x561471ea5000 2024-04-20T11:19:42.938 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h[51107]: starting mds.h at 2024-04-20T11:19:42.938 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 bash[51092]: 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 2024-04-20T11:19:42.938 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:19:42 smithi078 systemd[1]: Started Ceph mds.h for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6836/2547716992,v1:172.21.15.78:6837/2547716992] up:boot 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: fsmap 8 up:standby 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:19:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1503083776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6836/2547716992,v1:172.21.15.78:6837/2547716992] up:boot 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: fsmap 8 up:standby 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:19:44.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1503083776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:44.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:44.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6836/2547716992,v1:172.21.15.78:6837/2547716992] up:boot 2024-04-20T11:19:44.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: fsmap 8 up:standby 2024-04-20T11:19:44.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:19:44.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1503083776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:44.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:44 smithi165 ceph-mon[21975]: Deploying daemon mds.i on smithi165 2024-04-20T11:19:44.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:44.973 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:44 smithi044 ceph-mon[16855]: Deploying daemon mds.i on smithi165 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 -- 172.21.15.44:0/4056529636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 msgr2=0x7f562c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 --2- 172.21.15.44:0/4056529636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c104c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f561c009900 tx=0x7f561c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 -- 172.21.15.44:0/4056529636 shutdown_connections 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 --2- 172.21.15.44:0/4056529636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c107ac0 0x7f562c109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 --2- 172.21.15.44:0/4056529636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c105190 0x7f562c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 --2- 172.21.15.44:0/4056529636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c104c50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 -- 172.21.15.44:0/4056529636 >> 172.21.15.44:0/4056529636 conn(0x7f562c0fc230 msgr2=0x7f562c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 -- 172.21.15.44:0/4056529636 shutdown_connections 2024-04-20T11:19:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.115+0000 7f5631000640 1 -- 172.21.15.44:0/4056529636 wait complete. 2024-04-20T11:19:45.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.116+0000 7f5631000640 1 Processor -- start 2024-04-20T11:19:45.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.116+0000 7f5631000640 1 -- start start 2024-04-20T11:19:45.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5631000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5631000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 0x7f562c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5631000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c107ac0 0x7f562c1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5631000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f562c10db60 con 0x7f562c105190 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5631000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f562c10d9e0 con 0x7f562c102860 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5631000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f562c10dce0 con 0x7f562c107ac0 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5623e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 0x7f562c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c107ac0 0x7f562c1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5623e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 0x7f562c19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39342/0 (socket says 172.21.15.44:39342) 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5623e00640 1 -- 172.21.15.44:0/2726017957 learned_addr learned my addr 172.21.15.44:0/2726017957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562b400640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562be00640 1 -- 172.21.15.44:0/2726017957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 msgr2=0x7f562c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562be00640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562be00640 1 -- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 msgr2=0x7f562c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562be00640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 0x7f562c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562be00640 1 -- 172.21.15.44:0/2726017957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c1a2060 con 0x7f562c107ac0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f5623e00640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 0x7f562c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.117+0000 7f562b400640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.118+0000 7f562be00640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c107ac0 0x7f562c1a19f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5624009fd0 tx=0x7f562400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.118+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562400ce20 con 0x7f562c107ac0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.118+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c1a2350 con 0x7f562c107ac0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.118+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5624004b20 con 0x7f562c107ac0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.118+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624010790 con 0x7f562c107ac0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.118+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1a28a0 con 0x7f562c107ac0 2024-04-20T11:19:45.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.119+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c109610 con 0x7f562c107ac0 2024-04-20T11:19:45.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.121+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5624005020 con 0x7f562c107ac0 2024-04-20T11:19:45.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.123+0000 7f5628c00640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f56100a3c30 0x7f56100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:45.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.123+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f56240d0b50 con 0x7f562c107ac0 2024-04-20T11:19:45.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.123+0000 7f562b400640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f56100a3c30 0x7f56100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:45.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.124+0000 7f562b400640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f56100a3c30 0x7f56100a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f561c000c00 tx=0x7f561c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.129+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f562409d930 con 0x7f562c107ac0 2024-04-20T11:19:45.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:44 smithi078 ceph-mon[21464]: Deploying daemon mds.i on smithi165 2024-04-20T11:19:45.220 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 systemd[1]: Starting Ceph mds.i for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.379+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f562c02d5d0 con 0x7f562c107ac0 2024-04-20T11:19:45.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.381+0000 7f5628c00640 1 -- 172.21.15.44:0/2726017957 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f562409d230 con 0x7f562c107ac0 2024-04-20T11:19:45.387 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:45.387 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-20T11:19:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-04-20T11:19:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.383+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f56100a3c30 msgr2=0x7f56100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.383+0000 7f5631000640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f56100a3c30 0x7f56100a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f561c000c00 tx=0x7f561c005fd0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.383+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c107ac0 msgr2=0x7f562c1a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.383+0000 7f5631000640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c107ac0 0x7f562c1a19f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5624009fd0 tx=0x7f562400ef10 comp rx=0 tx=0).stop 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 shutdown_connections 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f56100a3c30 0x7f56100a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f562c107ac0 0x7f562c1a19f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c105190 0x7f562c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 --2- 172.21.15.44:0/2726017957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f562c102860 0x7f562c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 >> 172.21.15.44:0/2726017957 conn(0x7f562c0fc230 msgr2=0x7f562c1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 shutdown_connections 2024-04-20T11:19:45.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:45.384+0000 7f5631000640 1 -- 172.21.15.44:0/2726017957 wait complete. 2024-04-20T11:19:45.473 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 podman[52984]: 2024-04-20 11:19:45.234939732 +0000 UTC m=+0.019153608 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:45.474 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 podman[52984]: 2024-04-20 11:19:45.356749588 +0000 UTC m=+0.140963453 container create 1e7b0f13131aabd1ecea72742487511b21e02f068d690c44348ca330831466fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T11:19:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:45 smithi165 ceph-mon[21975]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:45 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2726017957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:45.730 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 podman[52984]: 2024-04-20 11:19:45.592632715 +0000 UTC m=+0.376846579 container init 1e7b0f13131aabd1ecea72742487511b21e02f068d690c44348ca330831466fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T11:19:45.730 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 podman[52984]: 2024-04-20 11:19:45.595381115 +0000 UTC m=+0.379594978 container start 1e7b0f13131aabd1ecea72742487511b21e02f068d690c44348ca330831466fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:19:45.730 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 ceph-mds[53003]: -- 172.21.15.165:0/1247555311 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55795e8a3a40 con 0x55795e877000 2024-04-20T11:19:45.730 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i[52999]: starting mds.i at 2024-04-20T11:19:45.730 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 bash[52984]: 1e7b0f13131aabd1ecea72742487511b21e02f068d690c44348ca330831466fb 2024-04-20T11:19:45.730 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:19:45 smithi165 systemd[1]: Started Ceph mds.i for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:46.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:45 smithi078 ceph-mon[21464]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:46.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:45 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2726017957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:46.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:45 smithi044 ceph-mon[16855]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:46.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:45 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2726017957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:46.973 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] up:boot 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: fsmap 9 up:standby 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:46.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:46 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] up:boot 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: fsmap 9 up:standby 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:46.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:46.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:46 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:47.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] up:boot 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: fsmap 9 up:standby 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:47.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:46 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:47.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:47.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 -- 172.21.15.44:0/3946590940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810e340 msgr2=0x7f997810e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 --2- 172.21.15.44:0/3946590940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810e340 0x7f997810e7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9970009ca0 tx=0x7f997002c100 comp rx=0 tx=0).stop 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 -- 172.21.15.44:0/3946590940 shutdown_connections 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 --2- 172.21.15.44:0/3946590940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810e340 0x7f997810e7c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 --2- 172.21.15.44:0/3946590940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f997810b990 0x7f997810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 --2- 172.21.15.44:0/3946590940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9978109020 0x7f997810b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 -- 172.21.15.44:0/3946590940 >> 172.21.15.44:0/3946590940 conn(0x7f997806fe40 msgr2=0x7f9978072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 -- 172.21.15.44:0/3946590940 shutdown_connections 2024-04-20T11:19:47.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.628+0000 7f997d200640 1 -- 172.21.15.44:0/3946590940 wait complete. 2024-04-20T11:19:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 Processor -- start 2024-04-20T11:19:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 -- start start 2024-04-20T11:19:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 0x7f99781a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810b990 0x7f99781a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f997810e340 0x7f99781ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f997811cb40 con 0x7f997810b990 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f997811c9c0 con 0x7f997810e340 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.629+0000 7f997d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f997811ccc0 con 0x7f9978109020 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f996fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810b990 0x7f99781a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 0x7f99781a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 0x7f99781a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:32820/0 (socket says 172.21.15.44:32820) 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 -- 172.21.15.44:0/3090779763 learned_addr learned my addr 172.21.15.44:0/3090779763 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f996fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810b990 0x7f99781a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39360/0 (socket says 172.21.15.44:39360) 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9976800640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f997810e340 0x7f99781ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 -- 172.21.15.44:0/3090779763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f997810e340 msgr2=0x7f99781ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f997810e340 0x7f99781ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 -- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810b990 msgr2=0x7f99781a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810b990 0x7f99781a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 -- 172.21.15.44:0/3090779763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99781aed10 con 0x7f9978109020 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.630+0000 7f9975e00640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 0x7f99781a70f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f996800e9f0 tx=0x7f996800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.631+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996800cd80 con 0x7f9978109020 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.631+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99781aefa0 con 0x7f9978109020 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.631+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9968010930 con 0x7f9978109020 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.631+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968018b00 con 0x7f9978109020 2024-04-20T11:19:47.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.631+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99781af4b0 con 0x7f9978109020 2024-04-20T11:19:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.632+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9978071930 con 0x7f9978109020 2024-04-20T11:19:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.634+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9968018ca0 con 0x7f9978109020 2024-04-20T11:19:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.636+0000 7f996ea00640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f995c0a3bb0 0x7f995c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.636+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f99680d0460 con 0x7f9978109020 2024-04-20T11:19:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.636+0000 7f996fe00640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f995c0a3bb0 0x7f995c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.637+0000 7f996fe00640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f995c0a3bb0 0x7f995c0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99781a8600 tx=0x7f9960005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:47.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.638+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f996809cbf0 con 0x7f9978109020 2024-04-20T11:19:47.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.896+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997802d5d0 con 0x7f9978109020 2024-04-20T11:19:47.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.897+0000 7f996ea00640 1 -- 172.21.15.44:0/3090779763 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f99680a0ef0 con 0x7f9978109020 2024-04-20T11:19:47.897 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:47.897 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T11:19:47.897 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 10 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f995c0a3bb0 msgr2=0x7f995c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f995c0a3bb0 0x7f995c0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99781a8600 tx=0x7f9960005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 msgr2=0x7f99781a70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 0x7f99781a70f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f996800e9f0 tx=0x7f996800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 shutdown_connections 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f995c0a3bb0 0x7f995c0a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f997810e340 0x7f99781ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f997810b990 0x7f99781a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 --2- 172.21.15.44:0/3090779763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9978109020 0x7f99781a70f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:47.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.900+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 >> 172.21.15.44:0/3090779763 conn(0x7f997806fe40 msgr2=0x7f99781099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:47.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.901+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 shutdown_connections 2024-04-20T11:19:47.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:47.901+0000 7f997d200640 1 -- 172.21.15.44:0/3090779763 wait complete. 2024-04-20T11:19:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:47 smithi165 ceph-mon[21975]: Deploying daemon mds.g on smithi044 2024-04-20T11:19:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:47 smithi165 ceph-mon[21975]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:48.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:47 smithi078 ceph-mon[21464]: Deploying daemon mds.g on smithi044 2024-04-20T11:19:48.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:47 smithi078 ceph-mon[21464]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:48.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:47 smithi044 ceph-mon[16855]: Deploying daemon mds.g on smithi044 2024-04-20T11:19:48.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:47 smithi044 ceph-mon[16855]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:49.136 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:48 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3090779763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:49.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:48 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3090779763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:49.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:48 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3090779763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:49.445 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:49 smithi044 systemd[1]: Starting Ceph mds.g for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:49.759 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:49 smithi044 ceph-mon[16855]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:49.759 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:49 smithi044 podman[57642]: 2024-04-20 11:19:49.432949648 +0000 UTC m=+0.035030321 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:49.760 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:49 smithi044 podman[57642]: 2024-04-20 11:19:49.560833869 +0000 UTC m=+0.162914604 container create 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T11:19:49.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 -- 172.21.15.44:0/3402440805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 msgr2=0x7f4c3c104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 --2- 172.21.15.44:0/3402440805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c104ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009900 tx=0x7f4c2c02bc50 comp rx=0 tx=0).stop 2024-04-20T11:19:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 -- 172.21.15.44:0/3402440805 shutdown_connections 2024-04-20T11:19:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 --2- 172.21.15.44:0/3402440805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c3c107d40 0x7f4c3c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 --2- 172.21.15.44:0/3402440805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c3c105410 0x7f4c3c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 --2- 172.21.15.44:0/3402440805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c104ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 -- 172.21.15.44:0/3402440805 >> 172.21.15.44:0/3402440805 conn(0x7f4c3c0fc450 msgr2=0x7f4c3c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 -- 172.21.15.44:0/3402440805 shutdown_connections 2024-04-20T11:19:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.137+0000 7f4c43000640 1 -- 172.21.15.44:0/3402440805 wait complete. 2024-04-20T11:19:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.138+0000 7f4c43000640 1 Processor -- start 2024-04-20T11:19:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.138+0000 7f4c43000640 1 -- start start 2024-04-20T11:19:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c43000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c43000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c3c105410 0x7f4c3c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c43000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c3c107d40 0x7f4c3c1a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c43000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c10dd90 con 0x7f4c3c102ae0 2024-04-20T11:19:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c43000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c10dc10 con 0x7f4c3c107d40 2024-04-20T11:19:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c43000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c10df10 con 0x7f4c3c105410 2024-04-20T11:19:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45850/0 (socket says 172.21.15.44:45850) 2024-04-20T11:19:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c3be00640 1 -- 172.21.15.44:0/1335451515 learned_addr learned my addr 172.21.15.44:0/1335451515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c41000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c3c107d40 0x7f4c3c1a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.139+0000 7f4c3b400640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c3c105410 0x7f4c3c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c3be00640 1 -- 172.21.15.44:0/1335451515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c3c105410 msgr2=0x7f4c3c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c3be00640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c3c105410 0x7f4c3c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c3be00640 1 -- 172.21.15.44:0/1335451515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c3c107d40 msgr2=0x7f4c3c1a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c3be00640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c3c107d40 0x7f4c3c1a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c3be00640 1 -- 172.21.15.44:0/1335451515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c1a2200 con 0x7f4c3c102ae0 2024-04-20T11:19:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c3be00640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c19a5e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005ab0 tx=0x7f4c2c004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c043020 con 0x7f4c3c102ae0 2024-04-20T11:19:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c1a2490 con 0x7f4c3c102ae0 2024-04-20T11:19:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4c2c004250 con 0x7f4c3c102ae0 2024-04-20T11:19:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c003b30 con 0x7f4c3c102ae0 2024-04-20T11:19:50.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.140+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c1a2a30 con 0x7f4c3c102ae0 2024-04-20T11:19:50.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.142+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3c109830 con 0x7f4c3c102ae0 2024-04-20T11:19:50.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.144+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4c2c046020 con 0x7f4c3c102ae0 2024-04-20T11:19:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.145+0000 7f4c38c00640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4c200a3bb0 0x7f4c200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:50.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.145+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4c2c0f21e0 con 0x7f4c3c102ae0 2024-04-20T11:19:50.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.145+0000 7f4c3b400640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4c200a3bb0 0x7f4c200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:50.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.146+0000 7f4c3b400640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4c200a3bb0 0x7f4c200a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c280095f0 tx=0x7f4c28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:50.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.150+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4c2c04d020 con 0x7f4c3c102ae0 2024-04-20T11:19:50.155 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:50 smithi044 podman[57642]: 2024-04-20 11:19:50.006188892 +0000 UTC m=+0.608269596 container init 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T11:19:50.156 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:50 smithi044 podman[57642]: 2024-04-20 11:19:50.015956905 +0000 UTC m=+0.618037573 container start 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True) 2024-04-20T11:19:50.156 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:50 smithi044 ceph-mds[57688]: -- 172.21.15.44:0/1385448384 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561cfb39a40 con 0x5561cfb0cc00 2024-04-20T11:19:50.156 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g[57660]: starting mds.g at 2024-04-20T11:19:50.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:49 smithi078 ceph-mon[21464]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:50.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:49 smithi165 ceph-mon[21975]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.410+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c3c02d5d0 con 0x7f4c3c102ae0 2024-04-20T11:19:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.411+0000 7f4c38c00640 1 -- 172.21.15.44:0/1335451515 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f4c2c0bf0e0 con 0x7f4c3c102ae0 2024-04-20T11:19:50.411 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:50.411 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T11:19:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 10 2024-04-20T11:19:50.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4c200a3bb0 msgr2=0x7f4c200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4c200a3bb0 0x7f4c200a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c280095f0 tx=0x7f4c28005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 msgr2=0x7f4c3c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c19a5e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005ab0 tx=0x7f4c2c004a30 comp rx=0 tx=0).stop 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 shutdown_connections 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4c200a3bb0 0x7f4c200a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c3c107d40 0x7f4c3c1a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c3c105410 0x7f4c3c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 --2- 172.21.15.44:0/1335451515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c3c102ae0 0x7f4c3c19a5e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 >> 172.21.15.44:0/1335451515 conn(0x7f4c3c0fc450 msgr2=0x7f4c3c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 shutdown_connections 2024-04-20T11:19:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:50.414+0000 7f4c43000640 1 -- 172.21.15.44:0/1335451515 wait complete. 2024-04-20T11:19:50.431 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:50 smithi044 bash[57642]: 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 2024-04-20T11:19:50.431 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:19:50 smithi044 systemd[1]: Started Ceph mds.g for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:51.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: Deploying daemon mds.a on smithi044 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1335451515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] up:boot 2024-04-20T11:19:51.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: fsmap 10 up:standby 2024-04-20T11:19:51.538 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:19:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: Deploying daemon mds.a on smithi044 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1335451515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] up:boot 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: fsmap 10 up:standby 2024-04-20T11:19:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:19:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: Deploying daemon mds.a on smithi044 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1335451515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] up:boot 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: fsmap 10 up:standby 2024-04-20T11:19:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:19:52.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 -- 172.21.15.44:0/54454054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c102860 msgr2=0x7f4b3c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 --2- 172.21.15.44:0/54454054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c102860 0x7f4b3c104c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 -- 172.21.15.44:0/54454054 shutdown_connections 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 --2- 172.21.15.44:0/54454054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c107ac0 0x7f4b3c109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 --2- 172.21.15.44:0/54454054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c105190 0x7f4b3c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 --2- 172.21.15.44:0/54454054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c102860 0x7f4b3c104c50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 -- 172.21.15.44:0/54454054 >> 172.21.15.44:0/54454054 conn(0x7f4b3c0fc230 msgr2=0x7f4b3c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 -- 172.21.15.44:0/54454054 shutdown_connections 2024-04-20T11:19:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.663+0000 7f4b3b200640 1 -- 172.21.15.44:0/54454054 wait complete. 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.664+0000 7f4b3b200640 1 Processor -- start 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.664+0000 7f4b3b200640 1 -- start start 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 0x7f4b3c1021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c105190 0x7f4b3c100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c107ac0 0x7f4b3c100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c10dbb0 con 0x7f4b3c102860 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c10da30 con 0x7f4b3c107ac0 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c10dd30 con 0x7f4b3c105190 2024-04-20T11:19:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b39e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 0x7f4b3c1021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b39e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 0x7f4b3c1021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45860/0 (socket says 172.21.15.44:45860) 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b39e00640 1 -- 172.21.15.44:0/3186573262 learned_addr learned my addr 172.21.15.44:0/3186573262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b3a800640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c107ac0 0x7f4b3c100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.665+0000 7f4b39400640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c105190 0x7f4b3c100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b39e00640 1 -- 172.21.15.44:0/3186573262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c105190 msgr2=0x7f4b3c100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b39e00640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c105190 0x7f4b3c100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b39e00640 1 -- 172.21.15.44:0/3186573262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c107ac0 msgr2=0x7f4b3c100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b39e00640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c107ac0 0x7f4b3c100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b39e00640 1 -- 172.21.15.44:0/3186573262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c1015e0 con 0x7f4b3c102860 2024-04-20T11:19:52.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b3a800640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c107ac0 0x7f4b3c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b39e00640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 0x7f4b3c1021d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005ab0 tx=0x7f4b2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c043020 con 0x7f4b3c102860 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c1a2320 con 0x7f4b3c102860 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4b2c004250 con 0x7f4b3c102860 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003b90 con 0x7f4b3c102860 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.666+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a2870 con 0x7f4b3c102860 2024-04-20T11:19:52.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.668+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c0fdd20 con 0x7f4b3c102860 2024-04-20T11:19:52.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.670+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4b2c046020 con 0x7f4b3c102860 2024-04-20T11:19:52.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.671+0000 7f4b32a00640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b200a3bb0 0x7f4b200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.671+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4b2c0f2150 con 0x7f4b3c102860 2024-04-20T11:19:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.672+0000 7f4b39400640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b200a3bb0 0x7f4b200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:52.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.672+0000 7f4b39400640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b200a3bb0 0x7f4b200a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c101f40 tx=0x7f4b28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:52.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.676+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4b2c04d020 con 0x7f4b3c102860 2024-04-20T11:19:52.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.948+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b3c02d7f0 con 0x7f4b3c102860 2024-04-20T11:19:52.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.955+0000 7f4b32a00640 1 -- 172.21.15.44:0/3186573262 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f4b2c0bef30 con 0x7f4b3c102860 2024-04-20T11:19:52.955 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:52.955 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-20T11:19:52.955 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 11 2024-04-20T11:19:52.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.958+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b200a3bb0 msgr2=0x7f4b200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:52.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.958+0000 7f4b3b200640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b200a3bb0 0x7f4b200a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c101f40 tx=0x7f4b28005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:52.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.958+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 msgr2=0x7f4b3c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:52.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.958+0000 7f4b3b200640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 0x7f4b3c1021d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005ab0 tx=0x7f4b2c004ab0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.959+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 shutdown_connections 2024-04-20T11:19:52.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.959+0000 7f4b3b200640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b200a3bb0 0x7f4b200a6070 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c101f40 tx=0x7f4b28005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:52.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.959+0000 7f4b3b200640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b3c107ac0 0x7f4b3c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.959+0000 7f4b3b200640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b3c105190 0x7f4b3c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.959+0000 7f4b3b200640 1 --2- 172.21.15.44:0/3186573262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b3c102860 0x7f4b3c1021d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:52.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.959+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 >> 172.21.15.44:0/3186573262 conn(0x7f4b3c0fc230 msgr2=0x7f4b3c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:52.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.961+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 shutdown_connections 2024-04-20T11:19:52.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:52.961+0000 7f4b3b200640 1 -- 172.21.15.44:0/3186573262 wait complete. 2024-04-20T11:19:53.568 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:53 smithi044 ceph-mon[16855]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:53.570 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:53 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3186573262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:53.570 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:53 smithi044 systemd[1]: Starting Ceph mds.a for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:53 smithi078 ceph-mon[21464]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:53.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:53 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3186573262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:53 smithi165 ceph-mon[21975]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:53.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:53 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3186573262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:54.190 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:53 smithi044 podman[58041]: 2024-04-20 11:19:53.84120824 +0000 UTC m=+0.036787746 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:54.191 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:53 smithi044 podman[58041]: 2024-04-20 11:19:53.968634695 +0000 UTC m=+0.164214242 container create c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:19:54.695 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:54 smithi044 podman[58041]: 2024-04-20 11:19:54.291372133 +0000 UTC m=+0.486951671 container init c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T11:19:54.695 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:54 smithi044 podman[58041]: 2024-04-20 11:19:54.302318929 +0000 UTC m=+0.497898442 container start c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:19:54.695 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:54 smithi044 ceph-mds[58059]: -- 172.21.15.44:0/2128683960 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1fb79ba40 con 0x55f1fb76f000 2024-04-20T11:19:54.696 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a[58055]: starting mds.a at 2024-04-20T11:19:54.696 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:54 smithi044 bash[58041]: c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 2024-04-20T11:19:54.696 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:19:54 smithi044 systemd[1]: Started Ceph mds.a for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:54.988 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:55.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.215+0000 7fc501a00640 1 -- 172.21.15.44:0/50809588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 msgr2=0x7fc4fc10c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:55.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.215+0000 7fc501a00640 1 --2- 172.21.15.44:0/50809588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc10c420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009ca0 tx=0x7fc4f402c0f0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 -- 172.21.15.44:0/50809588 shutdown_connections 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 --2- 172.21.15.44:0/50809588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc10c420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 --2- 172.21.15.44:0/50809588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4fc1051d0 0x7fc4fc105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 --2- 172.21.15.44:0/50809588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc103fe0 0x7fc4fc104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 -- 172.21.15.44:0/50809588 >> 172.21.15.44:0/50809588 conn(0x7fc4fc0ff570 msgr2=0x7fc4fc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 -- 172.21.15.44:0/50809588 shutdown_connections 2024-04-20T11:19:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.216+0000 7fc501a00640 1 -- 172.21.15.44:0/50809588 wait complete. 2024-04-20T11:19:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 Processor -- start 2024-04-20T11:19:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 -- start start 2024-04-20T11:19:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4fc103fe0 0x7fc4fc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc1051d0 0x7fc4fc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10fd90 con 0x7fc4fc103fe0 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10fc10 con 0x7fc4fc105b90 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.217+0000 7fc501a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10ff10 con 0x7fc4fc1051d0 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc4fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4fc103fe0 0x7fc4fc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc1a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:44436/0 (socket says 172.21.15.44:44436) 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 -- 172.21.15.44:0/763927736 learned_addr learned my addr 172.21.15.44:0/763927736 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc4f3e00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc1051d0 0x7fc4fc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 -- 172.21.15.44:0/763927736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc1051d0 msgr2=0x7fc4fc19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc1051d0 0x7fc4fc19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 -- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4fc103fe0 msgr2=0x7fc4fc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4fc103fe0 0x7fc4fc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 -- 172.21.15.44:0/763927736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc1a1f50 con 0x7fc4fc105b90 2024-04-20T11:19:55.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.218+0000 7fc501000640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc1a18e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4f402c630 tx=0x7fc4f4034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.219+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4039040 con 0x7fc4fc105b90 2024-04-20T11:19:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.219+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4f4044070 con 0x7fc4fc105b90 2024-04-20T11:19:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.219+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc1a21e0 con 0x7fc4fc105b90 2024-04-20T11:19:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.219+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4004430 con 0x7fc4fc105b90 2024-04-20T11:19:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.219+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc1a2720 con 0x7fc4fc105b90 2024-04-20T11:19:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.220+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc10b2a0 con 0x7fc4fc105b90 2024-04-20T11:19:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.222+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4f40045d0 con 0x7fc4fc105b90 2024-04-20T11:19:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.223+0000 7fc4f9600640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4e00a3c30 0x7fc4e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.224+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4f40f24d0 con 0x7fc4fc105b90 2024-04-20T11:19:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.224+0000 7fc4fbe00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4e00a3c30 0x7fc4e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.224+0000 7fc4fbe00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4e00a3c30 0x7fc4e00a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc105030 tx=0x7fc4ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.227+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc4f40bf2b0 con 0x7fc4fc105b90 2024-04-20T11:19:55.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.478+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4fc062b60 con 0x7fc4fc105b90 2024-04-20T11:19:55.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.479+0000 7fc4f9600640 1 -- 172.21.15.44:0/763927736 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7fc4f40098b0 con 0x7fc4fc105b90 2024-04-20T11:19:55.479 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:55.479 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-20T11:19:55.479 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 12 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4e00a3c30 msgr2=0x7fc4e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4e00a3c30 0x7fc4e00a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc105030 tx=0x7fc4ec005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 msgr2=0x7fc4fc1a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc1a18e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4f402c630 tx=0x7fc4f4034670 comp rx=0 tx=0).stop 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 shutdown_connections 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc4e00a3c30 0x7fc4e00a60f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc105b90 0x7fc4fc1a18e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc1051d0 0x7fc4fc19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 --2- 172.21.15.44:0/763927736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4fc103fe0 0x7fc4fc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:55.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 >> 172.21.15.44:0/763927736 conn(0x7fc4fc0ff570 msgr2=0x7fc4fc108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:55.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.482+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 shutdown_connections 2024-04-20T11:19:55.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:55.483+0000 7fc501a00640 1 -- 172.21.15.44:0/763927736 wait complete. 2024-04-20T11:19:55.637 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:55.637 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.637 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.637 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.637 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: Deploying daemon mds.b on smithi078 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6840/503506102,v1:172.21.15.44:6841/503506102] up:boot 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: fsmap 11 up:standby 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T11:19:55.638 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:55 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/763927736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:55.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:55.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: Deploying daemon mds.b on smithi078 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6840/503506102,v1:172.21.15.44:6841/503506102] up:boot 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: fsmap 11 up:standby 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T11:19:55.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:55 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/763927736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: Deploying daemon mds.b on smithi078 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6840/503506102,v1:172.21.15.44:6841/503506102] up:boot 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: fsmap 11 up:standby 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T11:19:55.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:55 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/763927736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:57.112 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:56 smithi078 systemd[1]: Starting Ceph mds.b for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:19:57.112 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 podman[51402]: 2024-04-20 11:19:57.015664735 +0000 UTC m=+0.017999421 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:19:57.380 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 podman[51402]: 2024-04-20 11:19:57.136777476 +0000 UTC m=+0.139112150 container create 731415ecc45a0242693f71ed4240297f802f7ade015cfd756dc864c90233b4a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T11:19:57.380 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 podman[51402]: 2024-04-20 11:19:57.380849646 +0000 UTC m=+0.383184324 container init 731415ecc45a0242693f71ed4240297f802f7ade015cfd756dc864c90233b4a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-20T11:19:57.501 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:19:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:57 smithi078 ceph-mon[21464]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:57 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:57 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 podman[51402]: 2024-04-20 11:19:57.383546151 +0000 UTC m=+0.385880835 container start 731415ecc45a0242693f71ed4240297f802f7ade015cfd756dc864c90233b4a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 ceph-mds[51420]: -- 172.21.15.78:0/3588664895 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bcaba7a40 con 0x555bcab7b000 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: starting mds.b at 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 bash[51402]: 731415ecc45a0242693f71ed4240297f802f7ade015cfd756dc864c90233b4a7 2024-04-20T11:19:57.688 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:19:57 smithi078 systemd[1]: Started Ceph mds.b for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:19:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.741+0000 7fc9eb200640 1 -- 172.21.15.44:0/4196464772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec100e10 msgr2=0x7fc9ec101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.741+0000 7fc9eb200640 1 --2- 172.21.15.44:0/4196464772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec100e10 0x7fc9ec101290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009900 tx=0x7fc9d402bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.741+0000 7fc9eb200640 1 -- 172.21.15.44:0/4196464772 shutdown_connections 2024-04-20T11:19:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.741+0000 7fc9eb200640 1 --2- 172.21.15.44:0/4196464772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.741+0000 7fc9eb200640 1 --2- 172.21.15.44:0/4196464772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec100e10 0x7fc9ec101290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.741+0000 7fc9eb200640 1 --2- 172.21.15.44:0/4196464772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 0x7fc9ec1004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.742+0000 7fc9eb200640 1 -- 172.21.15.44:0/4196464772 >> 172.21.15.44:0/4196464772 conn(0x7fc9ec0fb9e0 msgr2=0x7fc9ec0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.742+0000 7fc9eb200640 1 -- 172.21.15.44:0/4196464772 shutdown_connections 2024-04-20T11:19:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.742+0000 7fc9eb200640 1 -- 172.21.15.44:0/4196464772 wait complete. 2024-04-20T11:19:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 Processor -- start 2024-04-20T11:19:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 -- start start 2024-04-20T11:19:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 0x7fc9ec19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:57.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec100e10 0x7fc9ec19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec10fda0 con 0x7fc9ec100e10 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec10fc20 con 0x7fc9ec1017d0 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.743+0000 7fc9eb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec10ff20 con 0x7fc9ec1000b0 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec100e10 0x7fc9ec19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9e9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 0x7fc9ec19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec1a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:44452/0 (socket says 172.21.15.44:44452) 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 -- 172.21.15.44:0/2324273259 learned_addr learned my addr 172.21.15.44:0/2324273259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9e9e00640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 0x7fc9ec19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51238/0 (socket says 172.21.15.44:51238) 2024-04-20T11:19:57.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 -- 172.21.15.44:0/2324273259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 msgr2=0x7fc9ec19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 0x7fc9ec19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 -- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec100e10 msgr2=0x7fc9ec19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec100e10 0x7fc9ec19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 -- 172.21.15.44:0/2324273259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec1a2050 con 0x7fc9ec1017d0 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9e3e00640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec100e10 0x7fc9ec19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.744+0000 7fc9ea800640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec1a19e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400eb10 tx=0x7fc9e400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.745+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4004b20 con 0x7fc9ec1017d0 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.745+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9e4004cc0 con 0x7fc9ec1017d0 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.745+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40107f0 con 0x7fc9ec1017d0 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.745+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec1a2340 con 0x7fc9ec1017d0 2024-04-20T11:19:57.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.745+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec1a2810 con 0x7fc9ec1017d0 2024-04-20T11:19:57.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.746+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec10bbc0 con 0x7fc9ec1017d0 2024-04-20T11:19:57.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.748+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc9e4005020 con 0x7fc9ec1017d0 2024-04-20T11:19:57.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.749+0000 7fc9e2a00640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc9d00a3c30 0x7fc9d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:57.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.749+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc9e40d0540 con 0x7fc9ec1017d0 2024-04-20T11:19:57.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.749+0000 7fc9e9e00640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc9d00a3c30 0x7fc9d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:57.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.750+0000 7fc9e9e00640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc9d00a3c30 0x7fc9d00a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0095f0 tx=0x7fc9dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:57.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:57.753+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc9e409d320 con 0x7fc9ec1017d0 2024-04-20T11:19:57.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:57 smithi044 ceph-mon[16855]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:57.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:57 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:57.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:57 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:57 smithi165 ceph-mon[21975]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:57 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:57 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.038+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9ec02d5d0 con 0x7fc9ec1017d0 2024-04-20T11:19:58.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.039+0000 7fc9e2a00640 1 -- 172.21.15.44:0/2324273259 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7fc9e4087090 con 0x7fc9ec1017d0 2024-04-20T11:19:58.039 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:58.039 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-20T11:19:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 13 2024-04-20T11:19:58.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.041+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc9d00a3c30 msgr2=0x7fc9d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.041+0000 7fc9eb200640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc9d00a3c30 0x7fc9d00a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0095f0 tx=0x7fc9dc005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:58.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.041+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 msgr2=0x7fc9ec1a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.041+0000 7fc9eb200640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec1a19e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400eb10 tx=0x7fc9e400c2d0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 shutdown_connections 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc9d00a3c30 0x7fc9d00a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ec1017d0 0x7fc9ec1a19e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9ec100e10 0x7fc9ec19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 --2- 172.21.15.44:0/2324273259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9ec1000b0 0x7fc9ec19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 >> 172.21.15.44:0/2324273259 conn(0x7fc9ec0fb9e0 msgr2=0x7fc9ec0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 shutdown_connections 2024-04-20T11:19:58.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.042+0000 7fc9eb200640 1 -- 172.21.15.44:0/2324273259 wait complete. 2024-04-20T11:19:58.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 -- 172.21.15.44:0/2089005312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0075f20 msgr2=0x7f64c0074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 --2- 172.21.15.44:0/2089005312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0075f20 0x7f64c0074370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009ca0 tx=0x7f64b002c190 comp rx=0 tx=0).stop 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 -- 172.21.15.44:0/2089005312 shutdown_connections 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 --2- 172.21.15.44:0/2089005312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c0109cb0 0x7f64c010c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 --2- 172.21.15.44:0/2089005312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64c0074940 0x7f64c0074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 --2- 172.21.15.44:0/2089005312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0075f20 0x7f64c0074370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 -- 172.21.15.44:0/2089005312 >> 172.21.15.44:0/2089005312 conn(0x7f64c00fd370 msgr2=0x7f64c00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:58.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 -- 172.21.15.44:0/2089005312 shutdown_connections 2024-04-20T11:19:58.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.253+0000 7f64bf200640 1 -- 172.21.15.44:0/2089005312 wait complete. 2024-04-20T11:19:58.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.255+0000 7f64bf200640 1 Processor -- start 2024-04-20T11:19:58.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.255+0000 7f64bf200640 1 -- start start 2024-04-20T11:19:58.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 0x7f64c019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64c0075f20 0x7f64c019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c0109cb0 0x7f64c01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64c010fdb0 con 0x7f64c0075f20 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bf200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64c010fc30 con 0x7f64c0109cb0 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bf200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64c010ff30 con 0x7f64c0074940 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 0x7f64c019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 0x7f64c019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51254/0 (socket says 172.21.15.44:51254) 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 -- 172.21.15.44:0/68567010 learned_addr learned my addr 172.21.15.44:0/68567010 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64be800640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c0109cb0 0x7f64c01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bd400640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64c0075f20 0x7f64c019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 -- 172.21.15.44:0/68567010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c0109cb0 msgr2=0x7f64c01a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c0109cb0 0x7f64c01a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 -- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64c0075f20 msgr2=0x7f64c019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64c0075f20 0x7f64c019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.256+0000 7f64bde00640 1 -- 172.21.15.44:0/68567010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c01a1fe0 con 0x7f64c0074940 2024-04-20T11:19:58.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.257+0000 7f64bde00640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 0x7f64c019a3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009dc0 tx=0x7f64b00346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:58.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.257+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0039040 con 0x7f64c0074940 2024-04-20T11:19:58.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.257+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f64b0044070 con 0x7f64c0074940 2024-04-20T11:19:58.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.257+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0004510 con 0x7f64c0074940 2024-04-20T11:19:58.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.257+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c01a2270 con 0x7f64c0074940 2024-04-20T11:19:58.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.257+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c01a2710 con 0x7f64c0074940 2024-04-20T11:19:58.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.258+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c0108a70 con 0x7f64c0074940 2024-04-20T11:19:58.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.260+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f64b00046b0 con 0x7f64c0074940 2024-04-20T11:19:58.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.262+0000 7f64b6a00640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f64a40a3bb0 0x7f64a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.262+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f64b00f2850 con 0x7f64c0074940 2024-04-20T11:19:58.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.262+0000 7f64bd400640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f64a40a3bb0 0x7f64a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.263+0000 7f64bd400640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f64a40a3bb0 0x7f64a40a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64c019b8d0 tx=0x7f64a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:58.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.264+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f64b00bf630 con 0x7f64c0074940 2024-04-20T11:19:58.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.472+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64c002d7a0 con 0x7f64c0074940 2024-04-20T11:19:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.474+0000 7f64b6a00640 1 -- 172.21.15.44:0/68567010 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18408 (secure 0 0 0) 0x7f64b00bef30 con 0x7f64c0074940 2024-04-20T11:19:58.474 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:58.474 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":81,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:19:03.182461+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f64a40a3bb0 msgr2=0x7f64a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f64a40a3bb0 0x7f64a40a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64c019b8d0 tx=0x7f64a800b410 comp rx=0 tx=0).stop 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 msgr2=0x7f64c019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 0x7f64c019a3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009dc0 tx=0x7f64b00346f0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 shutdown_connections 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f64a40a3bb0 0x7f64a40a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c0109cb0 0x7f64c01a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64c0075f20 0x7f64c019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 --2- 172.21.15.44:0/68567010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c0074940 0x7f64c019a3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.476+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 >> 172.21.15.44:0/68567010 conn(0x7f64c00fd370 msgr2=0x7f64c0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.477+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 shutdown_connections 2024-04-20T11:19:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.477+0000 7f64bf200640 1 -- 172.21.15.44:0/68567010 wait complete. 2024-04-20T11:19:58.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:19:58.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:19:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 -- 172.21.15.44:0/3223809071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb8105390 msgr2=0x7f9bb8105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3223809071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb8105390 0x7f9bb8105810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009900 tx=0x7f9ba402bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 -- 172.21.15.44:0/3223809071 shutdown_connections 2024-04-20T11:19:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3223809071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105d50 0x7f9bb810c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3223809071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb8105390 0x7f9bb8105810 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3223809071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb81041a0 0x7f9bb81045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 -- 172.21.15.44:0/3223809071 >> 172.21.15.44:0/3223809071 conn(0x7f9bb80ff750 msgr2=0x7f9bb8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:58.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 -- 172.21.15.44:0/3223809071 shutdown_connections 2024-04-20T11:19:58.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.700+0000 7f9bbee00640 1 -- 172.21.15.44:0/3223809071 wait complete. 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.701+0000 7f9bbee00640 1 Processor -- start 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 -- start start 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 0x7f9bb819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 0x7f9bb819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb8105d50 0x7f9bb81a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb810ff00 con 0x7f9bb8105d50 2024-04-20T11:19:58.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb810fd80 con 0x7f9bb81041a0 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8110080 con 0x7f9bb8105390 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 0x7f9bb819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bb7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 0x7f9bb819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 0x7f9bb819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45232/0 (socket says 172.21.15.44:45232) 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bb7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 0x7f9bb819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33746/0 (socket says 172.21.15.44:33746) 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bb7e00640 1 -- 172.21.15.44:0/3170903936 learned_addr learned my addr 172.21.15.44:0/3170903936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bbce00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb8105d50 0x7f9bb81a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.702+0000 7f9bb7400640 1 -- 172.21.15.44:0/3170903936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 msgr2=0x7f9bb819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb7400640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 0x7f9bb819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb7400640 1 -- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb8105d50 msgr2=0x7f9bb81a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb7400640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb8105d50 0x7f9bb81a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb7400640 1 -- 172.21.15.44:0/3170903936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb81a2140 con 0x7f9bb8105390 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb7e00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 0x7f9bb819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb7400640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 0x7f9bb819aa60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009230 tx=0x7f9ba4004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4043020 con 0x7f9bb8105390 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb81a23d0 con 0x7f9bb8105390 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ba4004250 con 0x7f9bb8105390 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4003a90 con 0x7f9bb8105390 2024-04-20T11:19:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.703+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb81a2970 con 0x7f9bb8105390 2024-04-20T11:19:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.704+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb810b460 con 0x7f9bb8105390 2024-04-20T11:19:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.706+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9ba4046020 con 0x7f9bb8105390 2024-04-20T11:19:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.708+0000 7f9bb4c00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9b9c0a3bb0 0x7f9b9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.708+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9ba40f1fe0 con 0x7f9bb8105390 2024-04-20T11:19:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.708+0000 7f9bb7e00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9b9c0a3bb0 0x7f9b9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.709+0000 7f9bb7e00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9b9c0a3bb0 0x7f9b9c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80058c0 tx=0x7f9ba8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.712+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9ba404d020 con 0x7f9bb8105390 2024-04-20T11:19:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] up:boot 2024-04-20T11:19:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: fsmap 12 up:standby 2024-04-20T11:19:58.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:19:58.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2324273259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:58 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/68567010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.883 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.883 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] up:boot 2024-04-20T11:19:58.883 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: fsmap 12 up:standby 2024-04-20T11:19:58.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:19:58.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2324273259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:58 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/68567010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.925+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bb8062b60 con 0x7f9bb8105390 2024-04-20T11:19:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.926+0000 7f9bb4c00640 1 -- 172.21.15.44:0/3170903936 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f9ba40bee20 con 0x7f9bb8105390 2024-04-20T11:19:58.925 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9b9c0a3bb0 msgr2=0x7f9b9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9b9c0a3bb0 0x7f9b9c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80058c0 tx=0x7f9ba8005f90 comp rx=0 tx=0).stop 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 msgr2=0x7f9bb819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 0x7f9bb819aa60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009230 tx=0x7f9ba4004a30 comp rx=0 tx=0).stop 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 shutdown_connections 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9b9c0a3bb0 0x7f9b9c0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bb8105d50 0x7f9bb81a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bb8105390 0x7f9bb819aa60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 --2- 172.21.15.44:0/3170903936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb81041a0 0x7f9bb819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 >> 172.21.15.44:0/3170903936 conn(0x7f9bb80ff750 msgr2=0x7f9bb8108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 shutdown_connections 2024-04-20T11:19:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:58.928+0000 7f9bbee00640 1 -- 172.21.15.44:0/3170903936 wait complete. 2024-04-20T11:19:58.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] up:boot 2024-04-20T11:19:58.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: fsmap 12 up:standby 2024-04-20T11:19:58.943 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:19:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2324273259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/68567010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:19:58.946 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 -- 172.21.15.44:0/1761659986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554102fc0 msgr2=0x7ff554103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 --2- 172.21.15.44:0/1761659986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554102fc0 0x7ff554103440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff540009900 tx=0x7ff54002bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 -- 172.21.15.44:0/1761659986 shutdown_connections 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 --2- 172.21.15.44:0/1761659986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff554103980 0x7ff55410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 --2- 172.21.15.44:0/1761659986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554102fc0 0x7ff554103440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 --2- 172.21.15.44:0/1761659986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff5541021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 -- 172.21.15.44:0/1761659986 >> 172.21.15.44:0/1761659986 conn(0x7ff5540fd340 msgr2=0x7ff5540ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 -- 172.21.15.44:0/1761659986 shutdown_connections 2024-04-20T11:19:59.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.135+0000 7ff55a000640 1 -- 172.21.15.44:0/1761659986 wait complete. 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 Processor -- start 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 -- start start 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff55419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff554102fc0 0x7ff55419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554103980 0x7ff5541a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff55410db20 con 0x7ff554103980 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff55410d9a0 con 0x7ff554101dd0 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.136+0000 7ff55a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff55410dca0 con 0x7ff554102fc0 2024-04-20T11:19:59.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff55419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff553e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff554102fc0 0x7ff55419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff55419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45240/0 (socket says 172.21.15.44:45240) 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 -- 172.21.15.44:0/1591614421 learned_addr learned my addr 172.21.15.44:0/1591614421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff559600640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554103980 0x7ff5541a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 -- 172.21.15.44:0/1591614421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff554102fc0 msgr2=0x7ff55419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff554102fc0 0x7ff55419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 -- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554103980 msgr2=0x7ff5541a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554103980 0x7ff5541a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 -- 172.21.15.44:0/1591614421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5541a2000 con 0x7ff554101dd0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff558c00640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff55419a3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e9c0 tx=0x7ff54800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.137+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800cd80 con 0x7ff554101dd0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.138+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff5480035f0 con 0x7ff554101dd0 2024-04-20T11:19:59.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.138+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5541a22f0 con 0x7ff554101dd0 2024-04-20T11:19:59.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.138+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548010690 con 0x7ff554101dd0 2024-04-20T11:19:59.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.138+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5541a2810 con 0x7ff554101dd0 2024-04-20T11:19:59.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.139+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5541a2ad0 con 0x7ff554101dd0 2024-04-20T11:19:59.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.142+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5480037a0 con 0x7ff554101dd0 2024-04-20T11:19:59.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.144+0000 7ff551600640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff53c0a3bb0 0x7ff53c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.144+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5480d05e0 con 0x7ff554101dd0 2024-04-20T11:19:59.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.144+0000 7ff553e00640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff53c0a3bb0 0x7ff53c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.144+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff54809d3c0 con 0x7ff554101dd0 2024-04-20T11:19:59.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.145+0000 7ff553e00640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff53c0a3bb0 0x7ff53c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff540000c00 tx=0x7ff540005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:59.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.360+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff5541a2ca0 con 0x7ff554101dd0 2024-04-20T11:19:59.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.361+0000 7ff551600640 1 -- 172.21.15.44:0/1591614421 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7ff54809ccc0 con 0x7ff554101dd0 2024-04-20T11:19:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:19:59.361 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-04-20T11:19:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:19:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-04-20T11:19:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-04-20T11:19:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:19:59.362 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-04-20T11:19:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.363+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff53c0a3bb0 msgr2=0x7ff53c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.363+0000 7ff55a000640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff53c0a3bb0 0x7ff53c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff540000c00 tx=0x7ff540005fd0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.363+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 msgr2=0x7ff55419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.363+0000 7ff55a000640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff55419a3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e9c0 tx=0x7ff54800ee80 comp rx=0 tx=0).stop 2024-04-20T11:19:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 shutdown_connections 2024-04-20T11:19:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff53c0a3bb0 0x7ff53c0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff554103980 0x7ff5541a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff554102fc0 0x7ff55419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 --2- 172.21.15.44:0/1591614421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff554101dd0 0x7ff55419a3e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 >> 172.21.15.44:0/1591614421 conn(0x7ff5540fd340 msgr2=0x7ff554074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:59.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 shutdown_connections 2024-04-20T11:19:59.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.364+0000 7ff55a000640 1 -- 172.21.15.44:0/1591614421 wait complete. 2024-04-20T11:19:59.381 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T11:19:59.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T11:19:59.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 -- 172.21.15.44:0/2134306793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 msgr2=0x7f919c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 --2- 172.21.15.44:0/2134306793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 0x7f919c104e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f918c009900 tx=0x7f918c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 -- 172.21.15.44:0/2134306793 shutdown_connections 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 --2- 172.21.15.44:0/2134306793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 --2- 172.21.15.44:0/2134306793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 0x7f919c107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 --2- 172.21.15.44:0/2134306793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 0x7f919c104e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.585+0000 7f91a1c00640 1 -- 172.21.15.44:0/2134306793 >> 172.21.15.44:0/2134306793 conn(0x7f919c0fc430 msgr2=0x7f919c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.586+0000 7f91a1c00640 1 -- 172.21.15.44:0/2134306793 shutdown_connections 2024-04-20T11:19:59.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.586+0000 7f91a1c00640 1 -- 172.21.15.44:0/2134306793 wait complete. 2024-04-20T11:19:59.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 Processor -- start 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 -- start start 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 0x7f919c1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 0x7f919c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f919c10dd40 con 0x7f919c107ca0 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f919c10dbc0 con 0x7f919c105370 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f91a1c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f919c10dec0 con 0x7f919c102a40 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f919b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f9193e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 0x7f919c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f919b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42104/0 (socket says 172.21.15.44:42104) 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f919b400640 1 -- 172.21.15.44:0/3695813182 learned_addr learned my addr 172.21.15.44:0/3695813182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:19:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.587+0000 7f919aa00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 0x7f919c1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f919b400640 1 -- 172.21.15.44:0/3695813182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 msgr2=0x7f919c1023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f919b400640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 0x7f919c1023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f919b400640 1 -- 172.21.15.44:0/3695813182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 msgr2=0x7f919c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f919b400640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 0x7f919c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f919b400640 1 -- 172.21.15.44:0/3695813182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c1017c0 con 0x7f919c107ca0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f9193e00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 0x7f919c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f919b400640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c100f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f919400eb10 tx=0x7f919400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194004b20 con 0x7f919c107ca0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c1a2530 con 0x7f919c107ca0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9194004cc0 con 0x7f919c107ca0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.588+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91940107f0 con 0x7f919c107ca0 2024-04-20T11:19:59.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.589+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c1a2a80 con 0x7f919c107ca0 2024-04-20T11:19:59.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.590+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c0fdf20 con 0x7f919c107ca0 2024-04-20T11:19:59.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.592+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9194005020 con 0x7f919c107ca0 2024-04-20T11:19:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.593+0000 7f9193400640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f91800a3c30 0x7f91800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:19:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.593+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f91940d0570 con 0x7f919c107ca0 2024-04-20T11:19:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.594+0000 7f919aa00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f91800a3c30 0x7f91800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:19:59.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.594+0000 7f919aa00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f91800a3c30 0x7f91800a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f918c000c00 tx=0x7f918c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:19:59.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.598+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f919409d350 con 0x7f919c107ca0 2024-04-20T11:19:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:19:59.818+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f919c062b60 con 0x7f919c107ca0 2024-04-20T11:19:59.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:59 smithi078 ceph-mon[21464]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:59 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3170903936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:59 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1591614421' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T11:19:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:19:59 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:59.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:59 smithi044 ceph-mon[16855]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:59.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:59 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3170903936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:59.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:59 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1591614421' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T11:19:59.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:59.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:19:59 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:59 smithi165 ceph-mon[21975]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:19:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:59 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3170903936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:19:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:59 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1591614421' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T11:19:59.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:19:59.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:19:59 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.631+0000 7f9193400640 1 -- 172.21.15.44:0/3695813182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f919409cc50 con 0x7f919c107ca0 2024-04-20T11:20:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-04-20T11:20:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.634+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f91800a3c30 msgr2=0x7f91800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f91800a3c30 0x7f91800a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f918c000c00 tx=0x7f918c005fd0 comp rx=0 tx=0).stop 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 msgr2=0x7f919c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c100f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f919400eb10 tx=0x7f919400c2d0 comp rx=0 tx=0).stop 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 shutdown_connections 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f91800a3c30 0x7f91800a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f919c107ca0 0x7f919c100f30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c105370 0x7f919c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 --2- 172.21.15.44:0/3695813182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f919c102a40 0x7f919c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 >> 172.21.15.44:0/3695813182 conn(0x7f919c0fc430 msgr2=0x7f919c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 shutdown_connections 2024-04-20T11:20:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.635+0000 7f91a1c00640 1 -- 172.21.15.44:0/3695813182 wait complete. 2024-04-20T11:20:00.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T11:20:00.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 -- 172.21.15.44:0/1998147970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb041041e0 msgr2=0x7fbb04104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 --2- 172.21.15.44:0/1998147970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb041041e0 0x7fbb04104600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009900 tx=0x7fbaf402bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 -- 172.21.15.44:0/1998147970 shutdown_connections 2024-04-20T11:20:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 --2- 172.21.15.44:0/1998147970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04105d90 0x7fbb0410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 --2- 172.21.15.44:0/1998147970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041053d0 0x7fbb04105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 --2- 172.21.15.44:0/1998147970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb041041e0 0x7fbb04104600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 -- 172.21.15.44:0/1998147970 >> 172.21.15.44:0/1998147970 conn(0x7fbb040ff750 msgr2=0x7fbb04101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 -- 172.21.15.44:0/1998147970 shutdown_connections 2024-04-20T11:20:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.858+0000 7fbb09c00640 1 -- 172.21.15.44:0/1998147970 wait complete. 2024-04-20T11:20:01.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:00 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3695813182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T11:20:01.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:00 smithi165 ceph-mon[21975]: overall HEALTH_OK 2024-04-20T11:20:01.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:00 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.280 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.280 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.280 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:00 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3695813182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T11:20:01.280 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:00 smithi078 ceph-mon[21464]: overall HEALTH_OK 2024-04-20T11:20:01.280 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:00 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.859+0000 7fbb09c00640 1 Processor -- start 2024-04-20T11:20:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.859+0000 7fbb09c00640 1 -- start start 2024-04-20T11:20:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.859+0000 7fbb09c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 0x7fbb0419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.859+0000 7fbb09c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041053d0 0x7fbb0419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb09c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb04105d90 0x7fbb041a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb09c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410ff40 con 0x7fbb041053d0 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb09c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410fdc0 con 0x7fbb04105d90 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb09c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041100c0 con 0x7fbb041041e0 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 0x7fbb0419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041053d0 0x7fbb0419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb03400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb04105d90 0x7fbb041a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 0x7fbb0419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33814/0 (socket says 172.21.15.44:33814) 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 -- 172.21.15.44:0/3003010259 learned_addr learned my addr 172.21.15.44:0/3003010259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 -- 172.21.15.44:0/3003010259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb04105d90 msgr2=0x7fbb041a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb04105d90 0x7fbb041a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 -- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041053d0 msgr2=0x7fbb0419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041053d0 0x7fbb0419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb02a00640 1 -- 172.21.15.44:0/3003010259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041a21e0 con 0x7fbb041041e0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.860+0000 7fbb03400640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb04105d90 0x7fbb041a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.861+0000 7fbb02a00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 0x7fbb0419a5c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4000c00 tx=0x7fbaf402c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.861+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4043020 con 0x7fbb041041e0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.861+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbaf40042d0 con 0x7fbb041041e0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.861+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4003cc0 con 0x7fbb041041e0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.861+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a2470 con 0x7fbb041041e0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.861+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a2a10 con 0x7fbb041041e0 2024-04-20T11:20:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.862+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0410b550 con 0x7fbb041041e0 2024-04-20T11:20:01.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.864+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbaf4046020 con 0x7fbb041041e0 2024-04-20T11:20:01.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.867+0000 7fbafb400640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbae80a3c30 0x7fbae80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.867+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fbaf40f21d0 con 0x7fbb041041e0 2024-04-20T11:20:01.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.867+0000 7fbb02000640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbae80a3c30 0x7fbae80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.868+0000 7fbb02000640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbae80a3c30 0x7fbae80a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb0419bad0 tx=0x7fbaf000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:01.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:00.870+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbaf404d020 con 0x7fbb041041e0 2024-04-20T11:20:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:00 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3695813182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T11:20:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:00 smithi044 ceph-mon[16855]: overall HEALTH_OK 2024-04-20T11:20:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:00 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.097+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbb0402d5d0 con 0x7fbb041041e0 2024-04-20T11:20:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.642+0000 7fbafb400640 1 -- 172.21.15.44:0/3003010259 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fbaf40beed0 con 0x7fbb041041e0 2024-04-20T11:20:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-04-20T11:20:01.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbae80a3c30 msgr2=0x7fbae80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbae80a3c30 0x7fbae80a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb0419bad0 tx=0x7fbaf000b410 comp rx=0 tx=0).stop 2024-04-20T11:20:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 msgr2=0x7fbb0419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 0x7fbb0419a5c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4000c00 tx=0x7fbaf402c710 comp rx=0 tx=0).stop 2024-04-20T11:20:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 shutdown_connections 2024-04-20T11:20:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbae80a3c30 0x7fbae80a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb04105d90 0x7fbb041a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041053d0 0x7fbb0419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 --2- 172.21.15.44:0/3003010259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041041e0 0x7fbb0419a5c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.650+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 >> 172.21.15.44:0/3003010259 conn(0x7fbb040ff750 msgr2=0x7fbb04108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.651+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 shutdown_connections 2024-04-20T11:20:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.651+0000 7fbb09c00640 1 -- 172.21.15.44:0/3003010259 wait complete. 2024-04-20T11:20:01.692 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T11:20:01.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 -- 172.21.15.44:0/3534389771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968102a20 msgr2=0x7f4968104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 --2- 172.21.15.44:0/3534389771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968102a20 0x7f4968104e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4958009900 tx=0x7f495802bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:01.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 -- 172.21.15.44:0/3534389771 shutdown_connections 2024-04-20T11:20:01.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 --2- 172.21.15.44:0/3534389771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968107c80 0x7f496810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 --2- 172.21.15.44:0/3534389771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968105350 0x7f4968107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 --2- 172.21.15.44:0/3534389771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968102a20 0x7f4968104e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 -- 172.21.15.44:0/3534389771 >> 172.21.15.44:0/3534389771 conn(0x7f49680fc410 msgr2=0x7f49680fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:01.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 -- 172.21.15.44:0/3534389771 shutdown_connections 2024-04-20T11:20:01.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.878+0000 7f496e200640 1 -- 172.21.15.44:0/3534389771 wait complete. 2024-04-20T11:20:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.879+0000 7f496e200640 1 Processor -- start 2024-04-20T11:20:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.879+0000 7f496e200640 1 -- start start 2024-04-20T11:20:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f496e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 0x7f496819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f496e200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968105350 0x7f496819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f496e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968107c80 0x7f49681a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f496e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f496810dd00 con 0x7f4968102a20 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f496e200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f496810db80 con 0x7f4968105350 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f496e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f496810de80 con 0x7f4968107c80 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f4967400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 0x7f496819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f4967400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 0x7f496819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42150/0 (socket says 172.21.15.44:42150) 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f4967400640 1 -- 172.21.15.44:0/360936307 learned_addr learned my addr 172.21.15.44:0/360936307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f4966a00640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968105350 0x7f496819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.880+0000 7f4967e00640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968107c80 0x7f49681a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4967400640 1 -- 172.21.15.44:0/360936307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968107c80 msgr2=0x7f49681a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4967400640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968107c80 0x7f49681a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4967400640 1 -- 172.21.15.44:0/360936307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968105350 msgr2=0x7f496819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4967400640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968105350 0x7f496819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4967400640 1 -- 172.21.15.44:0/360936307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a21f0 con 0x7f4968102a20 2024-04-20T11:20:01.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4966a00640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968105350 0x7f496819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f4967400640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 0x7f496819a5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4958009230 tx=0x7f4958004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958043020 con 0x7f4968102a20 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a2480 con 0x7f4968102a20 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.881+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4958004250 con 0x7f4968102a20 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.882+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958003b50 con 0x7f4968102a20 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.882+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a29d0 con 0x7f4968102a20 2024-04-20T11:20:01.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.883+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49681097d0 con 0x7f4968102a20 2024-04-20T11:20:01.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.885+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4958046020 con 0x7f4968102a20 2024-04-20T11:20:01.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.886+0000 7f495fe00640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f494c0a3bb0 0x7f494c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:01.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.887+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f49580f2320 con 0x7f4968102a20 2024-04-20T11:20:01.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.887+0000 7f4966a00640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f494c0a3bb0 0x7f494c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:01.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.888+0000 7f4966a00640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f494c0a3bb0 0x7f494c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49540095f0 tx=0x7f4954005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:01.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:01.892+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f495804d020 con 0x7f4968102a20 2024-04-20T11:20:01.913 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3695813182' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.914 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:01.915 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3003010259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T11:20:01.915 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:01 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3695813182' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:01.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3003010259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T11:20:01.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:01 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3695813182' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:20:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3003010259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T11:20:01.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:01 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T11:20:02.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.148+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f496802d5d0 con 0x7f4968102a20 2024-04-20T11:20:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.690+0000 7f495fe00640 1 -- 172.21.15.44:0/360936307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f49580beeb0 con 0x7f4968102a20 2024-04-20T11:20:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T11:20:02.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.699+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f494c0a3bb0 msgr2=0x7f494c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.699+0000 7f496e200640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f494c0a3bb0 0x7f494c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49540095f0 tx=0x7f4954005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:02.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.699+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 msgr2=0x7f496819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.699+0000 7f496e200640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 0x7f496819a5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4958009230 tx=0x7f4958004ab0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 shutdown_connections 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f494c0a3bb0 0x7f494c0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968107c80 0x7f49681a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4968105350 0x7f496819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 --2- 172.21.15.44:0/360936307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4968102a20 0x7f496819a5d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 >> 172.21.15.44:0/360936307 conn(0x7f49680fc410 msgr2=0x7f4968102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 shutdown_connections 2024-04-20T11:20:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.700+0000 7f496e200640 1 -- 172.21.15.44:0/360936307 wait complete. 2024-04-20T11:20:02.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 -- 172.21.15.44:0/2395297497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 msgr2=0x7f3614108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 --2- 172.21.15.44:0/2395297497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f3614108800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f360402bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 -- 172.21.15.44:0/2395297497 shutdown_connections 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 --2- 172.21.15.44:0/2395297497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3614109f90 0x7f3614110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 --2- 172.21.15.44:0/2395297497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36141095d0 0x7f3614109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 --2- 172.21.15.44:0/2395297497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f3614108800 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 -- 172.21.15.44:0/2395297497 >> 172.21.15.44:0/2395297497 conn(0x7f36140773a0 msgr2=0x7f36140797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 -- 172.21.15.44:0/2395297497 shutdown_connections 2024-04-20T11:20:02.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.911+0000 7f3618c00640 1 -- 172.21.15.44:0/2395297497 wait complete. 2024-04-20T11:20:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.912+0000 7f3618c00640 1 Processor -- start 2024-04-20T11:20:02.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.912+0000 7f3618c00640 1 -- start start 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.912+0000 7f3618c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f361419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.912+0000 7f3618c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36141095d0 0x7f361419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.912+0000 7f3618c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3614109f90 0x7f36141a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3618c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3614114140 con 0x7f3614109f90 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3618c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3614113fc0 con 0x7f36141083e0 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3618c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36141142c0 con 0x7f36141095d0 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f361419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3614109f90 0x7f36141a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f361419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45332/0 (socket says 172.21.15.44:45332) 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 -- 172.21.15.44:0/3520932162 learned_addr learned my addr 172.21.15.44:0/3520932162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3612a00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36141095d0 0x7f361419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 -- 172.21.15.44:0/3520932162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36141095d0 msgr2=0x7f361419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36141095d0 0x7f361419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 -- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3614109f90 msgr2=0x7f36141a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3614109f90 0x7f36141a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 -- 172.21.15.44:0/3520932162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36141a6410 con 0x7f36141083e0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.913+0000 7f3613400640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f361419e7f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f3604004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.914+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604043020 con 0x7f36141083e0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.914+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3604004250 con 0x7f36141083e0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.914+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36141a66a0 con 0x7f36141083e0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.914+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604003c00 con 0x7f36141083e0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.914+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36141a6c40 con 0x7f36141083e0 2024-04-20T11:20:02.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.915+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361410f700 con 0x7f36141083e0 2024-04-20T11:20:02.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.917+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3604046020 con 0x7f36141083e0 2024-04-20T11:20:02.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.918+0000 7f360be00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f35f80a3c30 0x7f35f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.918+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f36040f2350 con 0x7f36141083e0 2024-04-20T11:20:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.918+0000 7f3612a00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f35f80a3c30 0x7f35f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:02.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.919+0000 7f3612a00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f35f80a3c30 0x7f35f80a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f361419fd00 tx=0x7f3600005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:02.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:02.922+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f360404d020 con 0x7f36141083e0 2024-04-20T11:20:02.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:02 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T11:20:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:02 smithi078 ceph-mon[21464]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T11:20:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:02 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/360936307' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T11:20:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:02 smithi078 ceph-mon[21464]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T11:20:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:02 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T11:20:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:02 smithi044 ceph-mon[16855]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T11:20:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:02 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/360936307' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T11:20:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:02 smithi044 ceph-mon[16855]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T11:20:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:02 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T11:20:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:02 smithi165 ceph-mon[21975]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T11:20:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:02 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/360936307' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T11:20:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:02 smithi165 ceph-mon[21975]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T11:20:03.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.145+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3614062b60 con 0x7f36141083e0 2024-04-20T11:20:03.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.147+0000 7f360be00640 1 -- 172.21.15.44:0/3520932162 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22184 (secure 0 0 0) 0x7f36040bede0 con 0x7f36141083e0 2024-04-20T11:20:03.147 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:03.147 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":84,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:02.149459+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:20:03.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f35f80a3c30 msgr2=0x7f35f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f35f80a3c30 0x7f35f80a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f361419fd00 tx=0x7f3600005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 msgr2=0x7f361419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f361419e7f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f3604004a30 comp rx=0 tx=0).stop 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 shutdown_connections 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f35f80a3c30 0x7f35f80a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3614109f90 0x7f36141a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36141095d0 0x7f361419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 --2- 172.21.15.44:0/3520932162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36141083e0 0x7f361419e7f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 >> 172.21.15.44:0/3520932162 conn(0x7f36140773a0 msgr2=0x7f361410c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 shutdown_connections 2024-04-20T11:20:03.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.150+0000 7f3618c00640 1 -- 172.21.15.44:0/3520932162 wait complete. 2024-04-20T11:20:03.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:20:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 -- 172.21.15.44:0/2882065610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 msgr2=0x7f46001045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 --2- 172.21.15.44:0/2882065610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f46001045e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009ca0 tx=0x7f45f002c100 comp rx=0 tx=0).stop 2024-04-20T11:20:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 -- 172.21.15.44:0/2882065610 shutdown_connections 2024-04-20T11:20:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 --2- 172.21.15.44:0/2882065610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4600105d70 0x7f460010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 --2- 172.21.15.44:0/2882065610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46001053b0 0x7f4600105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 --2- 172.21.15.44:0/2882065610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f46001045e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 -- 172.21.15.44:0/2882065610 >> 172.21.15.44:0/2882065610 conn(0x7f46000ff750 msgr2=0x7f4600101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.365+0000 7f4608600640 1 -- 172.21.15.44:0/2882065610 shutdown_connections 2024-04-20T11:20:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.366+0000 7f4608600640 1 -- 172.21.15.44:0/2882065610 wait complete. 2024-04-20T11:20:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.366+0000 7f4608600640 1 Processor -- start 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 -- start start 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f460019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46001053b0 0x7f460019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600105d70 0x7f46001a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f460010ff70 con 0x7f46001041c0 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f460010fdf0 con 0x7f4600105d70 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4608600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46001100f0 con 0x7f46001053b0 2024-04-20T11:20:03.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4605c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f460019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4605c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f460019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42172/0 (socket says 172.21.15.44:42172) 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4605c00640 1 -- 172.21.15.44:0/1164017156 learned_addr learned my addr 172.21.15.44:0/1164017156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4606600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600105d70 0x7f46001a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.367+0000 7f4605200640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46001053b0 0x7f460019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4605c00640 1 -- 172.21.15.44:0/1164017156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46001053b0 msgr2=0x7f460019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4605c00640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46001053b0 0x7f460019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4605c00640 1 -- 172.21.15.44:0/1164017156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600105d70 msgr2=0x7f46001a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4605c00640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600105d70 0x7f46001a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4605c00640 1 -- 172.21.15.44:0/1164017156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001a2220 con 0x7f46001041c0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4605c00640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f460019a600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45f0037040 tx=0x7f45f00346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0039040 con 0x7f46001041c0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f45f0032340 con 0x7f46001041c0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001a24b0 con 0x7f46001041c0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.368+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0004460 con 0x7f46001041c0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.369+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a29a0 con 0x7f46001041c0 2024-04-20T11:20:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.370+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460010b4e0 con 0x7f46001041c0 2024-04-20T11:20:03.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.372+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f45f0046030 con 0x7f46001041c0 2024-04-20T11:20:03.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.374+0000 7f45fe000640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45e40a3bb0 0x7f45e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.374+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f45f00f2440 con 0x7f46001041c0 2024-04-20T11:20:03.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.374+0000 7f4605200640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45e40a3bb0 0x7f45e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.375+0000 7f4605200640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45e40a3bb0 0x7f45e40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f460019bb10 tx=0x7f45ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.378+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f45f004d060 con 0x7f46001041c0 2024-04-20T11:20:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.592+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f460002d7a0 con 0x7f46001041c0 2024-04-20T11:20:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.595+0000 7f45fe000640 1 -- 172.21.15.44:0/1164017156 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22184 (secure 0 0 0) 0x7f45f00bef60 con 0x7f46001041c0 2024-04-20T11:20:03.595 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:03.595 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":84,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:02.149459+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:20:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45e40a3bb0 msgr2=0x7f45e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45e40a3bb0 0x7f45e40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f460019bb10 tx=0x7f45ec005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 msgr2=0x7f460019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f460019a600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45f0037040 tx=0x7f45f00346f0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 shutdown_connections 2024-04-20T11:20:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f45e40a3bb0 0x7f45e40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4600105d70 0x7f46001a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46001053b0 0x7f460019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.598+0000 7f4608600640 1 --2- 172.21.15.44:0/1164017156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46001041c0 0x7f460019a600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.599+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 >> 172.21.15.44:0/1164017156 conn(0x7f46000ff750 msgr2=0x7f46001086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.599+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 shutdown_connections 2024-04-20T11:20:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.599+0000 7f4608600640 1 -- 172.21.15.44:0/1164017156 wait complete. 2024-04-20T11:20:03.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T11:20:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.809+0000 7fe255800640 1 -- 172.21.15.44:0/1873798632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250102a40 msgr2=0x7fe250104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.809+0000 7fe255800640 1 --2- 172.21.15.44:0/1873798632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250102a40 0x7fe250104e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe240009860 tx=0x7fe24002bb90 comp rx=0 tx=0).stop 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 -- 172.21.15.44:0/1873798632 shutdown_connections 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 --2- 172.21.15.44:0/1873798632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250107ca0 0x7fe25010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 --2- 172.21.15.44:0/1873798632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe250105370 0x7fe250107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 --2- 172.21.15.44:0/1873798632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250102a40 0x7fe250104e30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 -- 172.21.15.44:0/1873798632 >> 172.21.15.44:0/1873798632 conn(0x7fe2500fc430 msgr2=0x7fe2500fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 -- 172.21.15.44:0/1873798632 shutdown_connections 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.810+0000 7fe255800640 1 -- 172.21.15.44:0/1873798632 wait complete. 2024-04-20T11:20:03.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.811+0000 7fe255800640 1 Processor -- start 2024-04-20T11:20:03.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 -- start start 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 0x7fe25019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe250105370 0x7fe25019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250107ca0 0x7fe2501a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe25010dd70 con 0x7fe250105370 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe25010dbf0 con 0x7fe250102a40 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe255800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe25010def0 con 0x7fe250107ca0 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe246a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 0x7fe25019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe246a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 0x7fe25019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45350/0 (socket says 172.21.15.44:45350) 2024-04-20T11:20:03.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe246a00640 1 -- 172.21.15.44:0/1132455255 learned_addr learned my addr 172.21.15.44:0/1132455255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe24ea00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe250105370 0x7fe25019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.812+0000 7fe24f400640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250107ca0 0x7fe2501a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe246a00640 1 -- 172.21.15.44:0/1132455255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250107ca0 msgr2=0x7fe2501a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe246a00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250107ca0 0x7fe2501a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe246a00640 1 -- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe250105370 msgr2=0x7fe25019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe246a00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe250105370 0x7fe25019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe246a00640 1 -- 172.21.15.44:0/1132455255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2501a2240 con 0x7fe250102a40 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe246a00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 0x7fe25019a620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe240034a50 tx=0x7fe240032460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240041030 con 0x7fe250102a40 2024-04-20T11:20:03.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe240005020 con 0x7fe250102a40 2024-04-20T11:20:03.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2501a24d0 con 0x7fe250102a40 2024-04-20T11:20:03.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240039040 con 0x7fe250102a40 2024-04-20T11:20:03.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.813+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501a29c0 con 0x7fe250102a40 2024-04-20T11:20:03.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.815+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2501097f0 con 0x7fe250102a40 2024-04-20T11:20:03.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.816+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe240044070 con 0x7fe250102a40 2024-04-20T11:20:03.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.818+0000 7fe247e00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe2340a3bb0 0x7fe2340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:03.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.818+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe2400f18b0 con 0x7fe250102a40 2024-04-20T11:20:03.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.818+0000 7fe24ea00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe2340a3bb0 0x7fe2340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:03.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.819+0000 7fe24ea00640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe2340a3bb0 0x7fe2340a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe25019bb30 tx=0x7fe238009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:03.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:03.822+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe2400333a0 con 0x7fe250102a40 2024-04-20T11:20:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: pgmap v319: 97 pgs: 14 creating+peering, 3 active+clean, 80 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:20:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/360936307' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] up:active 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3520932162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:03 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1164017156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: pgmap v319: 97 pgs: 14 creating+peering, 3 active+clean, 80 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:20:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:20:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/360936307' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] up:active 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3520932162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:03 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1164017156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:04.096+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe250062b60 con 0x7fe250102a40 2024-04-20T11:20:04.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: pgmap v319: 97 pgs: 14 creating+peering, 3 active+clean, 80 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/360936307' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] up:active 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3520932162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:03 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1164017156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:04.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1132455255' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T11:20:04.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:04 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T11:20:05.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.034+0000 7fe247e00640 1 -- 172.21.15.44:0/1132455255 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe2400be340 con 0x7fe250102a40 2024-04-20T11:20:05.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.038+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe2340a3bb0 msgr2=0x7fe2340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.038+0000 7fe255800640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe2340a3bb0 0x7fe2340a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe25019bb30 tx=0x7fe238009260 comp rx=0 tx=0).stop 2024-04-20T11:20:05.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.038+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 msgr2=0x7fe25019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.038+0000 7fe255800640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 0x7fe25019a620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe240034a50 tx=0x7fe240032460 comp rx=0 tx=0).stop 2024-04-20T11:20:05.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 shutdown_connections 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe2340a3bb0 0x7fe2340a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe250107ca0 0x7fe2501a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe250105370 0x7fe25019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 --2- 172.21.15.44:0/1132455255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe250102a40 0x7fe25019a620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 >> 172.21.15.44:0/1132455255 conn(0x7fe2500fc430 msgr2=0x7fe250102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 shutdown_connections 2024-04-20T11:20:05.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.039+0000 7fe255800640 1 -- 172.21.15.44:0/1132455255 wait complete. 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1132455255' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T11:20:05.057 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:04 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T11:20:05.059 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-20T11:20:05.060 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-20T11:20:05.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1132455255' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T11:20:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:04 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 -- 172.21.15.44:0/61918502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c1084a0 msgr2=0x7ffa0c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 --2- 172.21.15.44:0/61918502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c1084a0 0x7ffa0c1088c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009900 tx=0x7ff9fc02bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 -- 172.21.15.44:0/61918502 shutdown_connections 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 --2- 172.21.15.44:0/61918502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 --2- 172.21.15.44:0/61918502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 --2- 172.21.15.44:0/61918502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c1084a0 0x7ffa0c1088c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 -- 172.21.15.44:0/61918502 >> 172.21.15.44:0/61918502 conn(0x7ffa0c077430 msgr2=0x7ffa0c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:05.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 -- 172.21.15.44:0/61918502 shutdown_connections 2024-04-20T11:20:05.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.278+0000 7ffa10e00640 1 -- 172.21.15.44:0/61918502 wait complete. 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.279+0000 7ffa10e00640 1 Processor -- start 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.279+0000 7ffa10e00640 1 -- start start 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa10e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa0c1084a0 0x7ffa0c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa10e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa10e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa10e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1141f0 con 0x7ffa0c10a050 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa10e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c114070 con 0x7ffa0c109690 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa10e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c114370 con 0x7ffa0c1084a0 2024-04-20T11:20:05.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa0aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42212/0 (socket says 172.21.15.44:42212) 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa0be00640 1 -- 172.21.15.44:0/3979493169 learned_addr learned my addr 172.21.15.44:0/3979493169 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa0aa00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45378/0 (socket says 172.21.15.44:45378) 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.280+0000 7ffa0b400640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa0c1084a0 0x7ffa0c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0aa00640 1 -- 172.21.15.44:0/3979493169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa0c1084a0 msgr2=0x7ffa0c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0aa00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa0c1084a0 0x7ffa0c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0aa00640 1 -- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 msgr2=0x7ffa0c1a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0aa00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0aa00640 1 -- 172.21.15.44:0/3979493169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c1a6490 con 0x7ffa0c109690 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0be00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa0aa00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c19ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e9f0 tx=0x7ff9f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400ccc0 con 0x7ffa0c109690 2024-04-20T11:20:05.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff9f4010930 con 0x7ffa0c109690 2024-04-20T11:20:05.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c1a6780 con 0x7ffa0c109690 2024-04-20T11:20:05.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40189f0 con 0x7ffa0c109690 2024-04-20T11:20:05.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.281+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c1a6d20 con 0x7ffa0c109690 2024-04-20T11:20:05.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.282+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c10f7c0 con 0x7ffa0c109690 2024-04-20T11:20:05.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.284+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff9f4010ae0 con 0x7ffa0c109690 2024-04-20T11:20:05.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.286+0000 7ffa03e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff9f00a3c30 0x7ff9f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:05.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.286+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff9f40d0560 con 0x7ffa0c109690 2024-04-20T11:20:05.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.286+0000 7ffa0b400640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff9f00a3c30 0x7ff9f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:05.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.287+0000 7ffa0b400640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff9f00a3c30 0x7ff9f00a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:05.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.290+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff9f409c9a0 con 0x7ffa0c109690 2024-04-20T11:20:05.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:05.542+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ffa0c062b60 con 0x7ffa0c109690 2024-04-20T11:20:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: pgmap v322: 97 pgs: 29 creating+peering, 46 active+clean, 22 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-04-20T11:20:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-20T11:20:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: Cluster is now healthy 2024-04-20T11:20:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T11:20:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3979493169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-20T11:20:05.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:05 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-20T11:20:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.045+0000 7ffa03e00640 1 -- 172.21.15.44:0/3979493169 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7ff9f40a0ca0 con 0x7ffa0c109690 2024-04-20T11:20:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff9f00a3c30 msgr2=0x7ff9f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff9f00a3c30 0x7ff9f00a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc005fd0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 msgr2=0x7ffa0c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c19ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e9f0 tx=0x7ff9f400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 shutdown_connections 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ff9f00a3c30 0x7ff9f00a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0c10a050 0x7ffa0c1a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c109690 0x7ffa0c19ee00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 --2- 172.21.15.44:0/3979493169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa0c1084a0 0x7ffa0c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 >> 172.21.15.44:0/3979493169 conn(0x7ffa0c077430 msgr2=0x7ffa0c10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 shutdown_connections 2024-04-20T11:20:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.050+0000 7ffa10e00640 1 -- 172.21.15.44:0/3979493169 wait complete. 2024-04-20T11:20:06.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: pgmap v322: 97 pgs: 29 creating+peering, 46 active+clean, 22 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-04-20T11:20:06.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-20T11:20:06.073 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: Cluster is now healthy 2024-04-20T11:20:06.074 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T11:20:06.074 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:06.074 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3979493169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-20T11:20:06.074 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:05 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-20T11:20:06.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-20T11:20:06.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: pgmap v322: 97 pgs: 29 creating+peering, 46 active+clean, 22 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-04-20T11:20:06.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-20T11:20:06.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: Cluster is now healthy 2024-04-20T11:20:06.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T11:20:06.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:06.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3979493169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-20T11:20:06.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:05 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-20T11:20:06.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 -- 172.21.15.44:0/4067423143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 msgr2=0x7fcdc81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:06.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 --2- 172.21.15.44:0/4067423143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 0x7fcdc81023c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009900 tx=0x7fcdbc02bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:06.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 -- 172.21.15.44:0/4067423143 shutdown_connections 2024-04-20T11:20:06.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 --2- 172.21.15.44:0/4067423143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103b50 0x7fcdc810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 --2- 172.21.15.44:0/4067423143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdc8103190 0x7fcdc8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 --2- 172.21.15.44:0/4067423143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 0x7fcdc81023c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 -- 172.21.15.44:0/4067423143 >> 172.21.15.44:0/4067423143 conn(0x7fcdc80fd510 msgr2=0x7fcdc80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:06.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 -- 172.21.15.44:0/4067423143 shutdown_connections 2024-04-20T11:20:06.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.263+0000 7fcdd0800640 1 -- 172.21.15.44:0/4067423143 wait complete. 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.264+0000 7fcdd0800640 1 Processor -- start 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 -- start start 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 0x7fcdc8075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 0x7fcdc8074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdc8103b50 0x7fcdc8074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc810dd00 con 0x7fcdc8101fa0 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc810db80 con 0x7fcdc8103190 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdd0800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc810de80 con 0x7fcdc8103b50 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdcde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 0x7fcdc8075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdcd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 0x7fcdc8074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdcd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 0x7fcdc8074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45402/0 (socket says 172.21.15.44:45402) 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdcd400640 1 -- 172.21.15.44:0/3060609018 learned_addr learned my addr 172.21.15.44:0/3060609018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.265+0000 7fcdce800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdc8103b50 0x7fcdc8074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdcd400640 1 -- 172.21.15.44:0/3060609018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdc8103b50 msgr2=0x7fcdc8074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdcd400640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdc8103b50 0x7fcdc8074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdcd400640 1 -- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 msgr2=0x7fcdc8075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdcd400640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 0x7fcdc8075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdcd400640 1 -- 172.21.15.44:0/3060609018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc8075120 con 0x7fcdc8103190 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdcd400640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 0x7fcdc8074350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800e9f0 tx=0x7fcdb800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800cd80 con 0x7fcdc8103190 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc81a24f0 con 0x7fcdc8103190 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdb8005520 con 0x7fcdc8103190 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8010690 con 0x7fcdc8103190 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.266+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81a2a40 con 0x7fcdc8103190 2024-04-20T11:20:06.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.268+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc8109390 con 0x7fcdc8103190 2024-04-20T11:20:06.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.270+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcdb80046c0 con 0x7fcdc8103190 2024-04-20T11:20:06.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.272+0000 7fcdc2a00640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcdb00a3c30 0x7fcdb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:06.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.272+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcdb80d0720 con 0x7fcdc8103190 2024-04-20T11:20:06.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.272+0000 7fcdcde00640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcdb00a3c30 0x7fcdb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:06.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.273+0000 7fcdcde00640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcdb00a3c30 0x7fcdb00a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc02c900 tx=0x7fcdbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:06.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.276+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcdb809d1b0 con 0x7fcdc8103190 2024-04-20T11:20:06.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:06.525+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fcdc802d5d0 con 0x7fcdc8103190 2024-04-20T11:20:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:20:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-20T11:20:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T11:20:06.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:06.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-20T11:20:06.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-20T11:20:06.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3060609018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T11:20:06.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:06 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T11:20:07.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.058+0000 7fcdc2a00640 1 -- 172.21.15.44:0/3060609018 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fcdb809cab0 con 0x7fcdc8103190 2024-04-20T11:20:07.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.061+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcdb00a3c30 msgr2=0x7fcdb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.061+0000 7fcdd0800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcdb00a3c30 0x7fcdb00a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc02c900 tx=0x7fcdbc005fd0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 msgr2=0x7fcdc8074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 0x7fcdc8074350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800e9f0 tx=0x7fcdb800edf0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 shutdown_connections 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcdb00a3c30 0x7fcdb00a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdc8103b50 0x7fcdc8074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdc8103190 0x7fcdc8074350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 --2- 172.21.15.44:0/3060609018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdc8101fa0 0x7fcdc8075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 >> 172.21.15.44:0/3060609018 conn(0x7fcdc80fd510 msgr2=0x7fcdc8106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 shutdown_connections 2024-04-20T11:20:07.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.062+0000 7fcdd0800640 1 -- 172.21.15.44:0/3060609018 wait complete. 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3060609018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T11:20:07.083 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:06 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T11:20:07.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3060609018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T11:20:07.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:06 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 -- 172.21.15.44:0/871376098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8107bd0 msgr2=0x7efcc8109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 --2- 172.21.15.44:0/871376098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8107bd0 0x7efcc8109fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009900 tx=0x7efcc002bc40 comp rx=0 tx=0).stop 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 -- 172.21.15.44:0/871376098 shutdown_connections 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 --2- 172.21.15.44:0/871376098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc8107bd0 0x7efcc8109fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 --2- 172.21.15.44:0/871376098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc81052a0 0x7efcc8107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 --2- 172.21.15.44:0/871376098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8102970 0x7efcc8104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 -- 172.21.15.44:0/871376098 >> 172.21.15.44:0/871376098 conn(0x7efcc80fc300 msgr2=0x7efcc80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 -- 172.21.15.44:0/871376098 shutdown_connections 2024-04-20T11:20:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.265+0000 7efccea00640 1 -- 172.21.15.44:0/871376098 wait complete. 2024-04-20T11:20:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 Processor -- start 2024-04-20T11:20:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 -- start start 2024-04-20T11:20:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8102970 0x7efcc81022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc81052a0 0x7efcc8100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8107bd0 0x7efcc8100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcc810dc40 con 0x7efcc8107bd0 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc810dac0 con 0x7efcc81052a0 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.266+0000 7efccea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcc810ddc0 con 0x7efcc8102970 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efccd600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8102970 0x7efcc81022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8107bd0 0x7efcc8100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc81052a0 0x7efcc8100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efccd600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8102970 0x7efcc81022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33948/0 (socket says 172.21.15.44:33948) 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efccd600640 1 -- 172.21.15.44:0/4183858683 learned_addr learned my addr 172.21.15.44:0/4183858683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 -- 172.21.15.44:0/4183858683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8102970 msgr2=0x7efcc81022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8102970 0x7efcc81022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 -- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8107bd0 msgr2=0x7efcc8100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8107bd0 0x7efcc8100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 -- 172.21.15.44:0/4183858683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc81016f0 con 0x7efcc81052a0 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcce000640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8107bd0 0x7efcc8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcccc00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc81052a0 0x7efcc8100920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efcc8069bd0 tx=0x7efcb400dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.267+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4017070 con 0x7efcc81052a0 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.268+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efcb40099d0 con 0x7efcc81052a0 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.268+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4005650 con 0x7efcc81052a0 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.268+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc81a22f0 con 0x7efcc81052a0 2024-04-20T11:20:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.268+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc81a2840 con 0x7efcc81052a0 2024-04-20T11:20:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.269+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc80fddf0 con 0x7efcc81052a0 2024-04-20T11:20:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.270+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efcb4009b80 con 0x7efcc81052a0 2024-04-20T11:20:07.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.272+0000 7efcc6000640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efcb00a3c30 0x7efcb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:07.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.272+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7efcb40cff50 con 0x7efcc81052a0 2024-04-20T11:20:07.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.272+0000 7efccd600640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efcb00a3c30 0x7efcb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:07.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.273+0000 7efccd600640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efcb00a3c30 0x7efcb00a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009fd0 tx=0x7efcbc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.276+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efcb4020020 con 0x7efcc81052a0 2024-04-20T11:20:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.529+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7efcc802d5d0 con 0x7efcc81052a0 2024-04-20T11:20:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.801+0000 7efcc6000640 1 -- 172.21.15.44:0/4183858683 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7efcb409c9e0 con 0x7efcc81052a0 2024-04-20T11:20:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efcb00a3c30 msgr2=0x7efcb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efcb00a3c30 0x7efcb00a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009fd0 tx=0x7efcbc009500 comp rx=0 tx=0).stop 2024-04-20T11:20:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc81052a0 msgr2=0x7efcc8100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc81052a0 0x7efcc8100920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efcc8069bd0 tx=0x7efcb400dbb0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 shutdown_connections 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efcb00a3c30 0x7efcb00a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcc8107bd0 0x7efcc8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc81052a0 0x7efcc8100920 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 --2- 172.21.15.44:0/4183858683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcc8102970 0x7efcc81022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.805+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 >> 172.21.15.44:0/4183858683 conn(0x7efcc80fc300 msgr2=0x7efcc81032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.806+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 shutdown_connections 2024-04-20T11:20:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:07.806+0000 7efccea00640 1 -- 172.21.15.44:0/4183858683 wait complete. 2024-04-20T11:20:07.837 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-20T11:20:07.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-20T11:20:07.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: pgmap v323: 97 pgs: 26 creating+peering, 67 active+clean, 4 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] up:active 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4183858683' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:20:07.571+0000 7ff8f2550700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T11:20:07.938 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:20:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:20:07.571+0000 7ff8f2550700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-20T11:20:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: pgmap v323: 97 pgs: 26 creating+peering, 67 active+clean, 4 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-20T11:20:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T11:20:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] up:active 2024-04-20T11:20:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-20T11:20:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-20T11:20:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-20T11:20:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-20T11:20:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4183858683' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T11:20:07.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:07 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 -- 172.21.15.44:0/1513902874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe788105220 msgr2=0x7fe788107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 --2- 172.21.15.44:0/1513902874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe788105220 0x7fe788107610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe774009900 tx=0x7fe77402bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 -- 172.21.15.44:0/1513902874 shutdown_connections 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 --2- 172.21.15.44:0/1513902874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 0x7fe788109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 --2- 172.21.15.44:0/1513902874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe788105220 0x7fe788107610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 --2- 172.21.15.44:0/1513902874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7881028f0 0x7fe788104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 -- 172.21.15.44:0/1513902874 >> 172.21.15.44:0/1513902874 conn(0x7fe7880fc300 msgr2=0x7fe7880fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 -- 172.21.15.44:0/1513902874 shutdown_connections 2024-04-20T11:20:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.040+0000 7fe78ee00640 1 -- 172.21.15.44:0/1513902874 wait complete. 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.041+0000 7fe78ee00640 1 Processor -- start 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 -- start start 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 0x7fe78819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe788105220 0x7fe78819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 0x7fe7881a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe78810dbe0 con 0x7fe788107b50 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe78810da60 con 0x7fe7881028f0 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe78810dd60 con 0x7fe788105220 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 0x7fe78819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 0x7fe78819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:45432/0 (socket says 172.21.15.44:45432) 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78da00640 1 -- 172.21.15.44:0/649146260 learned_addr learned my addr 172.21.15.44:0/649146260 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78e400640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 0x7fe7881a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78d000640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe788105220 0x7fe78819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.042+0000 7fe78da00640 1 -- 172.21.15.44:0/649146260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe788105220 msgr2=0x7fe78819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78da00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe788105220 0x7fe78819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78da00640 1 -- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 msgr2=0x7fe7881a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78da00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 0x7fe7881a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78da00640 1 -- 172.21.15.44:0/649146260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7881a20b0 con 0x7fe7881028f0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78e400640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 0x7fe7881a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78da00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 0x7fe78819a490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00ea80 tx=0x7fe77c00ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00ce40 con 0x7fe7881028f0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe77c004b20 con 0x7fe7881028f0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011830 con 0x7fe7881028f0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7881a23a0 con 0x7fe7881028f0 2024-04-20T11:20:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.043+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7881a2870 con 0x7fe7881028f0 2024-04-20T11:20:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.044+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7881096a0 con 0x7fe7881028f0 2024-04-20T11:20:08.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.046+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe77c0119d0 con 0x7fe7881028f0 2024-04-20T11:20:08.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.048+0000 7fe786000640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7700a3c30 0x7fe7700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:08.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.048+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe77c0d09b0 con 0x7fe7881028f0 2024-04-20T11:20:08.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.048+0000 7fe78d000640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7700a3c30 0x7fe7700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:08.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.049+0000 7fe78d000640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7700a3c30 0x7fe7700a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe77402c900 tx=0x7fe774005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.052+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe77c016030 con 0x7fe7881028f0 2024-04-20T11:20:08.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: pgmap v323: 97 pgs: 26 creating+peering, 67 active+clean, 4 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-20T11:20:08.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T11:20:08.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] up:active 2024-04-20T11:20:08.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-20T11:20:08.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-20T11:20:08.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-20T11:20:08.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-20T11:20:08.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4183858683' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T11:20:08.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:07 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T11:20:08.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.277+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe7880623a0 con 0x7fe7700a3c30 2024-04-20T11:20:08.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:20:08.292+0000 7fa532a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:20:08.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:20:08.292+0000 7fa532a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:20:08.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:20:08.292+0000 7fa532a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:20:08.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:20:08.292+0000 7fa532a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:20:08.696 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:20:08.292+0000 7fa532a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:20:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.854+0000 7fe786000640 1 -- 172.21.15.44:0/649146260 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7880623a0 con 0x7fe7700a3c30 2024-04-20T11:20:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.856+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7700a3c30 msgr2=0x7fe7700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7700a3c30 0x7fe7700a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe77402c900 tx=0x7fe774005fd0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 msgr2=0x7fe78819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 0x7fe78819a490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00ea80 tx=0x7fe77c00ef40 comp rx=0 tx=0).stop 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 shutdown_connections 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe7700a3c30 0x7fe7700a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe788107b50 0x7fe7881a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe788105220 0x7fe78819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 --2- 172.21.15.44:0/649146260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7881028f0 0x7fe78819a490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 >> 172.21.15.44:0/649146260 conn(0x7fe7880fc300 msgr2=0x7fe788101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 shutdown_connections 2024-04-20T11:20:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:08.857+0000 7fe78ee00640 1 -- 172.21.15.44:0/649146260 wait complete. 2024-04-20T11:20:08.876 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 -- 172.21.15.44:0/319871993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c105290 msgr2=0x7fb35c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 --2- 172.21.15.44:0/319871993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c105290 0x7fb35c105710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb348009900 tx=0x7fb34802bc30 comp rx=0 tx=0).stop 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 -- 172.21.15.44:0/319871993 shutdown_connections 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 --2- 172.21.15.44:0/319871993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 0x7fb35c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 --2- 172.21.15.44:0/319871993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c105290 0x7fb35c105710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 --2- 172.21.15.44:0/319871993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb35c1040a0 0x7fb35c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 -- 172.21.15.44:0/319871993 >> 172.21.15.44:0/319871993 conn(0x7fb35c0ff610 msgr2=0x7fb35c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 -- 172.21.15.44:0/319871993 shutdown_connections 2024-04-20T11:20:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.066+0000 7fb361800640 1 -- 172.21.15.44:0/319871993 wait complete. 2024-04-20T11:20:09.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 Processor -- start 2024-04-20T11:20:09.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 -- start start 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 0x7fb35c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb35c105290 0x7fb35c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 0x7fb35c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10fe00 con 0x7fb35c105c50 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10fc80 con 0x7fb35c1040a0 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.067+0000 7fb361800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10ff80 con 0x7fb35c105290 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb35be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 0x7fb35c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 0x7fb35c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb35be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 0x7fb35c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55130/0 (socket says 172.21.15.44:55130) 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb35be00640 1 -- 172.21.15.44:0/3288837964 learned_addr learned my addr 172.21.15.44:0/3288837964 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb35b400640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb35c105290 0x7fb35c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 -- 172.21.15.44:0/3288837964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb35c105290 msgr2=0x7fb35c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb35c105290 0x7fb35c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 -- 172.21.15.44:0/3288837964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 msgr2=0x7fb35c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 0x7fb35c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 -- 172.21.15.44:0/3288837964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c1a2070 con 0x7fb35c105c50 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb35be00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 0x7fb35c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.068+0000 7fb360e00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 0x7fb35c1a1a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e970 tx=0x7fb35400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.069+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35400ccc0 con 0x7fb35c105c50 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.069+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c1a2360 con 0x7fb35c105c50 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.069+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb354005020 con 0x7fb35c105c50 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.069+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354004a20 con 0x7fb35c105c50 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.069+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c1a2830 con 0x7fb35c105c50 2024-04-20T11:20:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.070+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c10b3c0 con 0x7fb35c105c50 2024-04-20T11:20:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.072+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb354004c20 con 0x7fb35c105c50 2024-04-20T11:20:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.074+0000 7fb358c00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3400a3bb0 0x7fb3400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.074+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb3540d0370 con 0x7fb35c105c50 2024-04-20T11:20:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.074+0000 7fb35be00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3400a3bb0 0x7fb3400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.075+0000 7fb35be00640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3400a3bb0 0x7fb3400a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb34c011750 tx=0x7fb34c0096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.078+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb35409ce00 con 0x7fb35c105c50 2024-04-20T11:20:09.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T11:20:09.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] up:active 2024-04-20T11:20:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-20T11:20:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T11:20:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-04-20T11:20:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-20T11:20:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 799 B/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-20T11:20:09.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:08 smithi078 ceph-mon[21464]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T11:20:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] up:active 2024-04-20T11:20:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-20T11:20:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T11:20:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-04-20T11:20:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-20T11:20:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 799 B/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-20T11:20:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:08 smithi044 ceph-mon[16855]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:09.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T11:20:09.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] up:active 2024-04-20T11:20:09.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-20T11:20:09.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T11:20:09.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-04-20T11:20:09.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-20T11:20:09.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 799 B/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-20T11:20:09.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:08 smithi165 ceph-mon[21975]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.309+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb35c0623a0 con 0x7fb3400a3bb0 2024-04-20T11:20:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.345+0000 7fb358c00640 1 -- 172.21.15.44:0/3288837964 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb35c0623a0 con 0x7fb3400a3bb0 2024-04-20T11:20:09.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.347+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3400a3bb0 msgr2=0x7fb3400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.347+0000 7fb361800640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3400a3bb0 0x7fb3400a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb34c011750 tx=0x7fb34c0096d0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 msgr2=0x7fb35c1a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 0x7fb35c1a1a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e970 tx=0x7fb35400ee30 comp rx=0 tx=0).stop 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 shutdown_connections 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3400a3bb0 0x7fb3400a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb35c105c50 0x7fb35c1a1a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb35c105290 0x7fb35c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 --2- 172.21.15.44:0/3288837964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb35c1040a0 0x7fb35c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 >> 172.21.15.44:0/3288837964 conn(0x7fb35c0ff610 msgr2=0x7fb35c108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 shutdown_connections 2024-04-20T11:20:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.348+0000 7fb361800640 1 -- 172.21.15.44:0/3288837964 wait complete. 2024-04-20T11:20:09.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-20T11:20:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 -- 172.21.15.44:0/1098340919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105290 msgr2=0x7f4b34105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 --2- 172.21.15.44:0/1098340919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105290 0x7f4b34105710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b2002bc40 comp rx=0 tx=0).stop 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 -- 172.21.15.44:0/1098340919 shutdown_connections 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 --2- 172.21.15.44:0/1098340919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b34105c50 0x7f4b3410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 --2- 172.21.15.44:0/1098340919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105290 0x7f4b34105710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 --2- 172.21.15.44:0/1098340919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b341040a0 0x7f4b341044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 -- 172.21.15.44:0/1098340919 >> 172.21.15.44:0/1098340919 conn(0x7f4b340ff610 msgr2=0x7f4b34101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 -- 172.21.15.44:0/1098340919 shutdown_connections 2024-04-20T11:20:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.578+0000 7f4b33200640 1 -- 172.21.15.44:0/1098340919 wait complete. 2024-04-20T11:20:09.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.579+0000 7f4b33200640 1 Processor -- start 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.579+0000 7f4b33200640 1 -- start start 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.579+0000 7f4b33200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 0x7f4b3419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b33200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 0x7f4b3419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b33200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105c50 0x7f4b341a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b33200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3410fe50 con 0x7f4b34105290 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b33200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3410fcd0 con 0x7f4b34105c50 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b33200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3410ffd0 con 0x7f4b341040a0 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 0x7f4b3419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 0x7f4b3419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 0x7f4b3419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40674/0 (socket says 172.21.15.44:40674) 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31e00640 1 -- 172.21.15.44:0/912531676 learned_addr learned my addr 172.21.15.44:0/912531676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 0x7f4b3419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50474/0 (socket says 172.21.15.44:50474) 2024-04-20T11:20:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b32800640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105c50 0x7f4b341a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 -- 172.21.15.44:0/912531676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 msgr2=0x7f4b3419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 0x7f4b3419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 -- 172.21.15.44:0/912531676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105c50 msgr2=0x7f4b341a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105c50 0x7f4b341a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31400640 1 -- 172.21.15.44:0/912531676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b341a20a0 con 0x7f4b34105290 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b31e00640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 0x7f4b3419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.580+0000 7f4b32800640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105c50 0x7f4b341a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.581+0000 7f4b31400640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 0x7f4b3419a9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4b20036040 tx=0x7f4b2002c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.581+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20043020 con 0x7f4b34105290 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.581+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4b200042d0 con 0x7f4b34105290 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.581+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b341a2330 con 0x7f4b34105290 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.581+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003bb0 con 0x7f4b34105290 2024-04-20T11:20:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.581+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b341a2850 con 0x7f4b34105290 2024-04-20T11:20:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.582+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b340690f0 con 0x7f4b34105290 2024-04-20T11:20:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.584+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4b20046020 con 0x7f4b34105290 2024-04-20T11:20:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.587+0000 7f4b2aa00640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b180a3bb0 0x7f4b180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.587+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4b200f2950 con 0x7f4b34105290 2024-04-20T11:20:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.587+0000 7f4b31e00640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b180a3bb0 0x7f4b180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.588+0000 7f4b31e00640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b180a3bb0 0x7f4b180a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b240058c0 tx=0x7f4b24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.590+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4b2004d020 con 0x7f4b34105290 2024-04-20T11:20:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.825+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4b340623a0 con 0x7f4b180a3bb0 2024-04-20T11:20:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.841+0000 7f4b2aa00640 1 -- 172.21.15.44:0/912531676 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4b340623a0 con 0x7f4b180a3bb0 2024-04-20T11:20:09.840 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/qa/sv_0/4ad9e1fc-8db2-4eb2-9f1c-e52200117cc3 2024-04-20T11:20:09.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b180a3bb0 msgr2=0x7f4b180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b180a3bb0 0x7f4b180a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b240058c0 tx=0x7f4b24005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:09.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 msgr2=0x7f4b3419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 0x7f4b3419a9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4b20036040 tx=0x7f4b2002c720 comp rx=0 tx=0).stop 2024-04-20T11:20:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 shutdown_connections 2024-04-20T11:20:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4b180a3bb0 0x7f4b180a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b34105c50 0x7f4b341a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b34105290 0x7f4b3419a9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 --2- 172.21.15.44:0/912531676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b341040a0 0x7f4b3419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:09.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.843+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 >> 172.21.15.44:0/912531676 conn(0x7f4b340ff610 msgr2=0x7f4b34108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:09.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.844+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 shutdown_connections 2024-04-20T11:20:09.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:09.844+0000 7f4b33200640 1 -- 172.21.15.44:0/912531676 wait complete. 2024-04-20T11:20:09.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-20T11:20:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 -- 172.21.15.44:0/3230406681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105290 msgr2=0x7f6158105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 --2- 172.21.15.44:0/3230406681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105290 0x7f6158105710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6144009900 tx=0x7f614402bc40 comp rx=0 tx=0).stop 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 -- 172.21.15.44:0/3230406681 shutdown_connections 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 --2- 172.21.15.44:0/3230406681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105c50 0x7f615810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 --2- 172.21.15.44:0/3230406681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105290 0x7f6158105710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 --2- 172.21.15.44:0/3230406681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61581040a0 0x7f61581044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 -- 172.21.15.44:0/3230406681 >> 172.21.15.44:0/3230406681 conn(0x7f61580ff610 msgr2=0x7f6158101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 -- 172.21.15.44:0/3230406681 shutdown_connections 2024-04-20T11:20:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.043+0000 7f615ee00640 1 -- 172.21.15.44:0/3230406681 wait complete. 2024-04-20T11:20:10.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.044+0000 7f615ee00640 1 Processor -- start 2024-04-20T11:20:10.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.044+0000 7f615ee00640 1 -- start start 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61581040a0 0x7f615819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 0x7f615819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615ee00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 0x7f61581a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f615810fe50 con 0x7f6158105290 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615ee00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f615810fcd0 con 0x7f6158105c50 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f615810ffd0 con 0x7f61581040a0 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 0x7f615819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 0x7f61581a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 0x7f61581a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55158/0 (socket says 172.21.15.44:55158) 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 0x7f615819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50506/0 (socket says 172.21.15.44:50506) 2024-04-20T11:20:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615e400640 1 -- 172.21.15.44:0/912281001 learned_addr learned my addr 172.21.15.44:0/912281001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:10.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.045+0000 7f615da00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61581040a0 0x7f615819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615d000640 1 -- 172.21.15.44:0/912281001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61581040a0 msgr2=0x7f615819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615d000640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61581040a0 0x7f615819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615d000640 1 -- 172.21.15.44:0/912281001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 msgr2=0x7f61581a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615d000640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 0x7f61581a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615d000640 1 -- 172.21.15.44:0/912281001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61581a2050 con 0x7f6158105290 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615e400640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 0x7f61581a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615d000640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 0x7f615819a970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6144036040 tx=0x7f6144005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144043020 con 0x7f6158105290 2024-04-20T11:20:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61581a22e0 con 0x7f6158105290 2024-04-20T11:20:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f61440043d0 con 0x7f6158105290 2024-04-20T11:20:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144003c40 con 0x7f6158105290 2024-04-20T11:20:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.046+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a2800 con 0x7f6158105290 2024-04-20T11:20:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.048+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61580690f0 con 0x7f6158105290 2024-04-20T11:20:10.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.051+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6144046020 con 0x7f6158105290 2024-04-20T11:20:10.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.053+0000 7f6156000640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f61400a3bb0 0x7f61400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.053+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f61440f2ac0 con 0x7f6158105290 2024-04-20T11:20:10.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.053+0000 7f615da00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f61400a3bb0 0x7f61400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.053+0000 7f615da00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f61400a3bb0 0x7f61400a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61581050f0 tx=0x7f614c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.055+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f614404d020 con 0x7f6158105290 2024-04-20T11:20:10.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] up:active 2024-04-20T11:20:10.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-20T11:20:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:20:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: Cluster is now healthy 2024-04-20T11:20:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-04-20T11:20:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-20T11:20:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-04-20T11:20:10.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:09 smithi078 ceph-mon[21464]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] up:active 2024-04-20T11:20:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-20T11:20:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:20:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: Cluster is now healthy 2024-04-20T11:20:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-04-20T11:20:10.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-20T11:20:10.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-04-20T11:20:10.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:09 smithi044 ceph-mon[16855]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] up:active 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: Cluster is now healthy 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-20T11:20:10.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-04-20T11:20:10.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:09 smithi165 ceph-mon[21975]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:10.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.291+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f61580623a0 con 0x7f61400a3bb0 2024-04-20T11:20:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.327+0000 7f6156000640 1 -- 172.21.15.44:0/912281001 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f61580623a0 con 0x7f61400a3bb0 2024-04-20T11:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.329+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f61400a3bb0 msgr2=0x7f61400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f61400a3bb0 0x7f61400a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61581050f0 tx=0x7f614c009680 comp rx=0 tx=0).stop 2024-04-20T11:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 msgr2=0x7f615819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 0x7f615819a970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6144036040 tx=0x7f6144005e50 comp rx=0 tx=0).stop 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 shutdown_connections 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f61400a3bb0 0x7f61400a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6158105c50 0x7f61581a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6158105290 0x7f615819a970 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 --2- 172.21.15.44:0/912281001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61581040a0 0x7f615819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 >> 172.21.15.44:0/912281001 conn(0x7f61580ff610 msgr2=0x7f6158108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 shutdown_connections 2024-04-20T11:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.330+0000 7f615ee00640 1 -- 172.21.15.44:0/912281001 wait complete. 2024-04-20T11:20:10.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-20T11:20:10.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 -- 172.21.15.44:0/2459941627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 msgr2=0x7fbb98109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2459941627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb98109fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009be0 tx=0x7fbb9002bef0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 -- 172.21.15.44:0/2459941627 shutdown_connections 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2459941627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb98109fa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2459941627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb98105220 0x7fbb98107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2459941627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb981028f0 0x7fbb98104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 -- 172.21.15.44:0/2459941627 >> 172.21.15.44:0/2459941627 conn(0x7fbb980fc300 msgr2=0x7fbb980fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 -- 172.21.15.44:0/2459941627 shutdown_connections 2024-04-20T11:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.568+0000 7fbb9e000640 1 -- 172.21.15.44:0/2459941627 wait complete. 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.569+0000 7fbb9e000640 1 Processor -- start 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.569+0000 7fbb9e000640 1 -- start start 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 0x7fbb9819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb98105220 0x7fbb9819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb981a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9810dc80 con 0x7fbb98105220 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9e000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9810db00 con 0x7fbb98107b50 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9810de00 con 0x7fbb981028f0 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 0x7fbb9819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb981a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 0x7fbb9819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40708/0 (socket says 172.21.15.44:40708) 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9cc00640 1 -- 172.21.15.44:0/2906072588 learned_addr learned my addr 172.21.15.44:0/2906072588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb8fe00640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb98105220 0x7fbb9819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9d600640 1 -- 172.21.15.44:0/2906072588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 msgr2=0x7fbb9819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9d600640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 0x7fbb9819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9d600640 1 -- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb98105220 msgr2=0x7fbb9819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9d600640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb98105220 0x7fbb9819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9d600640 1 -- 172.21.15.44:0/2906072588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb981a2010 con 0x7fbb98107b50 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.570+0000 7fbb9cc00640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 0x7fbb9819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.571+0000 7fbb9d600640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb981a19a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb90036040 tx=0x7fbb90033820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.571+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90043020 con 0x7fbb98107b50 2024-04-20T11:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.571+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb90033990 con 0x7fbb98107b50 2024-04-20T11:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.571+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9003e740 con 0x7fbb98107b50 2024-04-20T11:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.571+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb981a22a0 con 0x7fbb98107b50 2024-04-20T11:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.571+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb981a2840 con 0x7fbb98107b50 2024-04-20T11:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.573+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb98109640 con 0x7fbb98107b50 2024-04-20T11:20:10.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.574+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb90046020 con 0x7fbb98107b50 2024-04-20T11:20:10.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.576+0000 7fbb96000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbb7c0a3c30 0x7fbb7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:10.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.576+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbb900f28d0 con 0x7fbb98107b50 2024-04-20T11:20:10.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.576+0000 7fbb9cc00640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbb7c0a3c30 0x7fbb7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:10.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.577+0000 7fbb9cc00640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbb7c0a3c30 0x7fbb7c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb88011550 tx=0x7fbb88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:10.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.580+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbb9004d020 con 0x7fbb98107b50 2024-04-20T11:20:10.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.825+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbb98062430 con 0x7fbb7c0a3c30 2024-04-20T11:20:10.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.839+0000 7fbb96000640 1 -- 172.21.15.44:0/2906072588 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbb98062430 con 0x7fbb7c0a3c30 2024-04-20T11:20:10.839 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/qa/sv_1/2d5dd744-448e-43b3-845f-21ec1fa93e09 2024-04-20T11:20:10.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbb7c0a3c30 msgr2=0x7fbb7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbb7c0a3c30 0x7fbb7c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb88011550 tx=0x7fbb88005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:10.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 msgr2=0x7fbb981a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:10.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb981a19a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb90036040 tx=0x7fbb90033820 comp rx=0 tx=0).stop 2024-04-20T11:20:10.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 shutdown_connections 2024-04-20T11:20:10.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fbb7c0a3c30 0x7fbb7c0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb98107b50 0x7fbb981a19a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb98105220 0x7fbb9819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 --2- 172.21.15.44:0/2906072588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb981028f0 0x7fbb9819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.841+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 >> 172.21.15.44:0/2906072588 conn(0x7fbb980fc300 msgr2=0x7fbb98101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.842+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 shutdown_connections 2024-04-20T11:20:10.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:10.842+0000 7fbb9e000640 1 -- 172.21.15.44:0/2906072588 wait complete. 2024-04-20T11:20:10.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:20:11.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 -- 172.21.15.44:0/2566556750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 msgr2=0x7f1ff410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 --2- 172.21.15.44:0/2566556750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff410c420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009ca0 tx=0x7f1fec02c100 comp rx=0 tx=0).stop 2024-04-20T11:20:11.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 -- 172.21.15.44:0/2566556750 shutdown_connections 2024-04-20T11:20:11.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 --2- 172.21.15.44:0/2566556750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff410c420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 --2- 172.21.15.44:0/2566556750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ff41051d0 0x7f1ff4105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 --2- 172.21.15.44:0/2566556750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff4103fe0 0x7f1ff4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 -- 172.21.15.44:0/2566556750 >> 172.21.15.44:0/2566556750 conn(0x7f1ff40ff570 msgr2=0x7f1ff4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:11.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 -- 172.21.15.44:0/2566556750 shutdown_connections 2024-04-20T11:20:11.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.047+0000 7f1ff3200640 1 -- 172.21.15.44:0/2566556750 wait complete. 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.048+0000 7f1ff3200640 1 Processor -- start 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.048+0000 7f1ff3200640 1 -- start start 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ff4103fe0 0x7f1ff419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 0x7f1ff419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff41a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410fd80 con 0x7f1ff41051d0 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410fc00 con 0x7f1ff4105b90 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410ff00 con 0x7f1ff4103fe0 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 0x7f1ff419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 0x7f1ff419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50538/0 (socket says 172.21.15.44:50538) 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff1400640 1 -- 172.21.15.44:0/1511675593 learned_addr learned my addr 172.21.15.44:0/1511675593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff2800640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff41a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.049+0000 7f1ff1e00640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ff4103fe0 0x7f1ff419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff2800640 1 -- 172.21.15.44:0/1511675593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ff4103fe0 msgr2=0x7f1ff419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff2800640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ff4103fe0 0x7f1ff419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff2800640 1 -- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 msgr2=0x7f1ff419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff2800640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 0x7f1ff419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff2800640 1 -- 172.21.15.44:0/1511675593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff41a1f50 con 0x7f1ff4105b90 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff1400640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 0x7f1ff419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff2800640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff41a18e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1fec037040 tx=0x7f1fec034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec039040 con 0x7f1ff4105b90 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1fec044070 con 0x7f1ff4105b90 2024-04-20T11:20:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41a21e0 con 0x7f1ff4105b90 2024-04-20T11:20:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004460 con 0x7f1ff4105b90 2024-04-20T11:20:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.050+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41a26a0 con 0x7f1ff4105b90 2024-04-20T11:20:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.051+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff410b2a0 con 0x7f1ff4105b90 2024-04-20T11:20:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.053+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1fec004600 con 0x7f1ff4105b90 2024-04-20T11:20:11.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.055+0000 7f1feaa00640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1fd80a3c40 0x7f1fd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.055+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1fec0f28f0 con 0x7f1ff4105b90 2024-04-20T11:20:11.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.055+0000 7f1ff1e00640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1fd80a3c40 0x7f1fd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.055+0000 7f1ff1e00640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1fd80a3c40 0x7f1fd80a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4105030 tx=0x7f1fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.059+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1fec0bf330 con 0x7f1ff4105b90 2024-04-20T11:20:11.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:10 smithi078 ceph-mon[21464]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:11.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:10 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] up:active 2024-04-20T11:20:11.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:10 smithi078 ceph-mon[21464]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-20T11:20:11.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:10 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:20:11.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:10 smithi078 ceph-mon[21464]: pgmap v325: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-04-20T11:20:11.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:10 smithi078 ceph-mon[21464]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:11.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:10 smithi044 ceph-mon[16855]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:11.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:10 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] up:active 2024-04-20T11:20:11.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:10 smithi044 ceph-mon[16855]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-20T11:20:11.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:10 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:20:11.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:10 smithi044 ceph-mon[16855]: pgmap v325: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-04-20T11:20:11.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:10 smithi044 ceph-mon[16855]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:10 smithi165 ceph-mon[21975]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:10 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] up:active 2024-04-20T11:20:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:10 smithi165 ceph-mon[21975]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-20T11:20:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:10 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:20:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:10 smithi165 ceph-mon[21975]: pgmap v325: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-04-20T11:20:11.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:10 smithi165 ceph-mon[21975]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:11.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.304+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ff402d5d0 con 0x7f1ff4105b90 2024-04-20T11:20:11.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.306+0000 7f1feaa00640 1 -- 172.21.15.44:0/1511675593 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f1fec0bec30 con 0x7f1ff4105b90 2024-04-20T11:20:11.305 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:11.305 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:20:09.840804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:20:11.306 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1fd80a3c40 msgr2=0x7f1fd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1fd80a3c40 0x7f1fd80a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4105030 tx=0x7f1fe4005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 msgr2=0x7f1ff41a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff41a18e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1fec037040 tx=0x7f1fec034670 comp rx=0 tx=0).stop 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 shutdown_connections 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f1fd80a3c40 0x7f1fd80a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ff4105b90 0x7f1ff41a18e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ff41051d0 0x7f1ff419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 --2- 172.21.15.44:0/1511675593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ff4103fe0 0x7f1ff419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 >> 172.21.15.44:0/1511675593 conn(0x7f1ff40ff570 msgr2=0x7f1ff4108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:11.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.308+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 shutdown_connections 2024-04-20T11:20:11.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.309+0000 7f1ff3200640 1 -- 172.21.15.44:0/1511675593 wait complete. 2024-04-20T11:20:11.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 -- 172.21.15.44:0/2746503802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 msgr2=0x7f54e010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2746503802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e010c600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009860 tx=0x7f54d802bb70 comp rx=0 tx=0).stop 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 -- 172.21.15.44:0/2746503802 shutdown_connections 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2746503802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e010c600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2746503802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01053b0 0x7f54e0105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2746503802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01041c0 0x7f54e01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 -- 172.21.15.44:0/2746503802 >> 172.21.15.44:0/2746503802 conn(0x7f54e00ff750 msgr2=0x7f54e0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 -- 172.21.15.44:0/2746503802 shutdown_connections 2024-04-20T11:20:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.533+0000 7f54e6e00640 1 -- 172.21.15.44:0/2746503802 wait complete. 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.534+0000 7f54e6e00640 1 Processor -- start 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.534+0000 7f54e6e00640 1 -- start start 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e6e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01041c0 0x7f54e019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 0x7f54e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e6e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e01a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010ff70 con 0x7f54e01053b0 2024-04-20T11:20:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e6e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010fdf0 con 0x7f54e0105d70 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e6e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01100f0 con 0x7f54e01041c0 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 0x7f54e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54e4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e01a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 0x7f54e019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50558/0 (socket says 172.21.15.44:50558) 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54d7400640 1 -- 172.21.15.44:0/2047148441 learned_addr learned my addr 172.21.15.44:0/2047148441 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.535+0000 7f54dfe00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01041c0 0x7f54e019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54d7400640 1 -- 172.21.15.44:0/2047148441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01041c0 msgr2=0x7f54e019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54d7400640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01041c0 0x7f54e019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54d7400640 1 -- 172.21.15.44:0/2047148441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 msgr2=0x7f54e01a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54d7400640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e01a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54d7400640 1 -- 172.21.15.44:0/2047148441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01a2170 con 0x7f54e01053b0 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54dfe00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01041c0 0x7f54e019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54e4e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54d7400640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 0x7f54e019aa90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54c800e9f0 tx=0x7f54c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c800cd80 con 0x7f54e01053b0 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54c8005520 con 0x7f54e01053b0 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8010690 con 0x7f54e01053b0 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a2460 con 0x7f54e01053b0 2024-04-20T11:20:11.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.536+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a2960 con 0x7f54e01053b0 2024-04-20T11:20:11.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.537+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e010b410 con 0x7f54e01053b0 2024-04-20T11:20:11.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.539+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f54c80046c0 con 0x7f54e01053b0 2024-04-20T11:20:11.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.542+0000 7f54dd600640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54c40a3c40 0x7f54c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:11.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.542+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f54c80d0990 con 0x7f54e01053b0 2024-04-20T11:20:11.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.542+0000 7f54dfe00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54c40a3c40 0x7f54c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:11.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.543+0000 7f54dfe00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54c40a3c40 0x7f54c40a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54e0105210 tx=0x7f54d00095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:11.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.547+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f54c809d2c0 con 0x7f54e01053b0 2024-04-20T11:20:11.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.760+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54e002d820 con 0x7f54e01053b0 2024-04-20T11:20:11.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.764+0000 7f54dd600640 1 -- 172.21.15.44:0/2047148441 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22171 (secure 0 0 0) 0x7f54c809cbc0 con 0x7f54e01053b0 2024-04-20T11:20:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:11.765 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":85,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:03.685700+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:20:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.767+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54c40a3c40 msgr2=0x7f54c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.767+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54c40a3c40 0x7f54c40a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54e0105210 tx=0x7f54d00095c0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.767+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 msgr2=0x7f54e019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:11.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.767+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 0x7f54e019aa90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54c800e9f0 tx=0x7f54c800edf0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.767+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 shutdown_connections 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f54c40a3c40 0x7f54c40a6100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e0105d70 0x7f54e01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54e01053b0 0x7f54e019aa90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 --2- 172.21.15.44:0/2047148441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54e01041c0 0x7f54e019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 >> 172.21.15.44:0/2047148441 conn(0x7f54e00ff750 msgr2=0x7f54e01085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 shutdown_connections 2024-04-20T11:20:11.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:11.768+0000 7f54e6e00640 1 -- 172.21.15.44:0/2047148441 wait complete. 2024-04-20T11:20:11.788 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T11:20:11.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:20:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 -- 172.21.15.44:0/1508698633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701085c0 msgr2=0x7f51701089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 --2- 172.21.15.44:0/1508698633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701085c0 0x7f51701089e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5164009900 tx=0x7f516402bc40 comp rx=0 tx=0).stop 2024-04-20T11:20:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 -- 172.21.15.44:0/1508698633 shutdown_connections 2024-04-20T11:20:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 --2- 172.21.15.44:0/1508698633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f517010a170 0x7f5170110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 --2- 172.21.15.44:0/1508698633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51701097b0 0x7f5170109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 --2- 172.21.15.44:0/1508698633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701085c0 0x7f51701089e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 -- 172.21.15.44:0/1508698633 >> 172.21.15.44:0/1508698633 conn(0x7f5170077580 msgr2=0x7f51700799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 -- 172.21.15.44:0/1508698633 shutdown_connections 2024-04-20T11:20:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.016+0000 7f5178400640 1 -- 172.21.15.44:0/1508698633 wait complete. 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 Processor -- start 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 -- start start 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51701085c0 0x7f517019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 0x7f517019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517010a170 0x7f51701a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5170114370 con 0x7f517010a170 2024-04-20T11:20:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51701141f0 con 0x7f51701097b0 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.018+0000 7f5178400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51701144f0 con 0x7f51701085c0 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 0x7f517019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 0x7f517019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55240/0 (socket says 172.21.15.44:55240) 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5176400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517010a170 0x7f51701a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175000640 1 -- 172.21.15.44:0/272983761 learned_addr learned my addr 172.21.15.44:0/272983761 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175a00640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51701085c0 0x7f517019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175a00640 1 -- 172.21.15.44:0/272983761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 msgr2=0x7f517019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175a00640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 0x7f517019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175a00640 1 -- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517010a170 msgr2=0x7f51701a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175a00640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517010a170 0x7f51701a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175a00640 1 -- 172.21.15.44:0/272983761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51701a65d0 con 0x7f51701085c0 2024-04-20T11:20:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.019+0000 7f5175000640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 0x7f517019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:12.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.020+0000 7f5175a00640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51701085c0 0x7f517019e9b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5164009230 tx=0x7f516402c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:12.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.020+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164043020 con 0x7f51701085c0 2024-04-20T11:20:12.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.020+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51701a6860 con 0x7f51701085c0 2024-04-20T11:20:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.020+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5164004350 con 0x7f51701085c0 2024-04-20T11:20:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.020+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164003b50 con 0x7f51701085c0 2024-04-20T11:20:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.020+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a6d80 con 0x7f51701085c0 2024-04-20T11:20:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.022+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517010f8e0 con 0x7f51701085c0 2024-04-20T11:20:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.024+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5164046020 con 0x7f51701085c0 2024-04-20T11:20:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.025+0000 7f516e000640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f51580a3c40 0x7f51580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.026+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f51640f2420 con 0x7f51701085c0 2024-04-20T11:20:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.026+0000 7f5175000640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f51580a3c40 0x7f51580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.027+0000 7f5175000640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f51580a3c40 0x7f51580a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f517019fec0 tx=0x7f515c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.030+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f516404d020 con 0x7f51701085c0 2024-04-20T11:20:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:11 smithi078 ceph-mon[21464]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:11 smithi078 ceph-mon[21464]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T11:20:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:11 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1511675593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:20:12.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:11 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2047148441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:11 smithi044 ceph-mon[16855]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:11 smithi044 ceph-mon[16855]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T11:20:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:11 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1511675593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:20:12.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:11 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2047148441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:12.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:11 smithi165 ceph-mon[21975]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:20:12.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:11 smithi165 ceph-mon[21975]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T11:20:12.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:11 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1511675593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:20:12.230 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:11 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2047148441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.237+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51700623a0 con 0x7f51580a3c40 2024-04-20T11:20:12.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.263+0000 7f516e000640 1 -- 172.21.15.44:0/272983761 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258956 (secure 0 0 0) 0x7f51700623a0 con 0x7f51580a3c40 2024-04-20T11:20:12.264 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:20:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f51580a3c40 msgr2=0x7f51580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f51580a3c40 0x7f51580a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f517019fec0 tx=0x7f515c00b4a0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51701085c0 msgr2=0x7f517019e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51701085c0 0x7f517019e9b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5164009230 tx=0x7f516402c7a0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 shutdown_connections 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f51580a3c40 0x7f51580a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517010a170 0x7f51701a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51701097b0 0x7f517019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 --2- 172.21.15.44:0/272983761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51701085c0 0x7f517019e9b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 >> 172.21.15.44:0/272983761 conn(0x7f5170077580 msgr2=0x7f517010cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 shutdown_connections 2024-04-20T11:20:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.269+0000 7f5178400640 1 -- 172.21.15.44:0/272983761 wait complete. 2024-04-20T11:20:12.289 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-20T11:20:12.047732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634141,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":181,"num_read_kb":190,"num_write":374,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360348,"kb_used_data":11196,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581476,"statfs":{"total":1151940427776,"available":1151571431424,"internally_reserved":0,"allocated":11464704,"data_stored":7091651,"data_compressed":109472,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43773,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":45,"num_read_kb":71,"num_write":193,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.379374"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891893+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:20:03.891893+0000","last_peered":"2024-04-20T11:20:03.891893+0000","last_clean":"2024-04-20T11:20:03.891893+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:20:03.891893+0000","last_undegraded":"2024-04-20T11:20:03.891893+0000","last_fullsized":"2024-04-20T11:20:03.891893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892531+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:20:03.892531+0000","last_peered":"2024-04-20T11:20:03.892531+0000","last_clean":"2024-04-20T11:20:03.892531+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:20:03.892531+0000","last_undegraded":"2024-04-20T11:20:03.892531+0000","last_fullsized":"2024-04-20T11:20:03.892531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892520+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:20:03.892520+0000","last_peered":"2024-04-20T11:20:03.892520+0000","last_clean":"2024-04-20T11:20:03.892520+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:20:03.892520+0000","last_undegraded":"2024-04-20T11:20:03.892520+0000","last_fullsized":"2024-04-20T11:20:03.892520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891747+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:20:03.891747+0000","last_peered":"2024-04-20T11:20:03.891747+0000","last_clean":"2024-04-20T11:20:03.891747+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:20:03.891747+0000","last_undegraded":"2024-04-20T11:20:03.891747+0000","last_fullsized":"2024-04-20T11:20:03.891747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699499+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:20:03.699499+0000","last_peered":"2024-04-20T11:20:03.699499+0000","last_clean":"2024-04-20T11:20:03.699499+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:20:03.699499+0000","last_undegraded":"2024-04-20T11:20:03.699499+0000","last_fullsized":"2024-04-20T11:20:03.699499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.693592+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:20:03.693592+0000","last_peered":"2024-04-20T11:20:03.693592+0000","last_clean":"2024-04-20T11:20:03.693592+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:20:03.693592+0000","last_undegraded":"2024-04-20T11:20:03.693592+0000","last_fullsized":"2024-04-20T11:20:03.693592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251290+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:20:04.251290+0000","last_peered":"2024-04-20T11:20:04.251290+0000","last_clean":"2024-04-20T11:20:04.251290+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:20:04.251290+0000","last_undegraded":"2024-04-20T11:20:04.251290+0000","last_fullsized":"2024-04-20T11:20:04.251290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.698816+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:20:03.698816+0000","last_peered":"2024-04-20T11:20:03.698816+0000","last_clean":"2024-04-20T11:20:03.698816+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:20:03.698816+0000","last_undegraded":"2024-04-20T11:20:03.698816+0000","last_fullsized":"2024-04-20T11:20:03.698816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891547+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:20:03.891547+0000","last_peered":"2024-04-20T11:20:03.891547+0000","last_clean":"2024-04-20T11:20:03.891547+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:20:03.891547+0000","last_undegraded":"2024-04-20T11:20:03.891547+0000","last_fullsized":"2024-04-20T11:20:03.891547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.693557+0000","last_change":"2024-04-20T11:20:02.697661+0000","last_active":"2024-04-20T11:20:03.693557+0000","last_peered":"2024-04-20T11:20:03.693557+0000","last_clean":"2024-04-20T11:20:03.693557+0000","last_became_active":"2024-04-20T11:20:02.697362+0000","last_became_peered":"2024-04-20T11:20:02.697362+0000","last_unstale":"2024-04-20T11:20:03.693557+0000","last_undegraded":"2024-04-20T11:20:03.693557+0000","last_fullsized":"2024-04-20T11:20:03.693557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:53:17.606523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251722+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:20:04.251722+0000","last_peered":"2024-04-20T11:20:04.251722+0000","last_clean":"2024-04-20T11:20:04.251722+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:20:04.251722+0000","last_undegraded":"2024-04-20T11:20:04.251722+0000","last_fullsized":"2024-04-20T11:20:04.251722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891744+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:20:03.891744+0000","last_peered":"2024-04-20T11:20:03.891744+0000","last_clean":"2024-04-20T11:20:03.891744+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:20:03.891744+0000","last_undegraded":"2024-04-20T11:20:03.891744+0000","last_fullsized":"2024-04-20T11:20:03.891744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891983+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:20:03.891983+0000","last_peered":"2024-04-20T11:20:03.891983+0000","last_clean":"2024-04-20T11:20:03.891983+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:20:03.891983+0000","last_undegraded":"2024-04-20T11:20:03.891983+0000","last_fullsized":"2024-04-20T11:20:03.891983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892220+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:20:03.892220+0000","last_peered":"2024-04-20T11:20:03.892220+0000","last_clean":"2024-04-20T11:20:03.892220+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:20:03.892220+0000","last_undegraded":"2024-04-20T11:20:03.892220+0000","last_fullsized":"2024-04-20T11:20:03.892220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891994+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:20:03.891994+0000","last_peered":"2024-04-20T11:20:03.891994+0000","last_clean":"2024-04-20T11:20:03.891994+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:20:03.891994+0000","last_undegraded":"2024-04-20T11:20:03.891994+0000","last_fullsized":"2024-04-20T11:20:03.891994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.890988+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:20:03.890988+0000","last_peered":"2024-04-20T11:20:03.890988+0000","last_clean":"2024-04-20T11:20:03.890988+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:20:03.890988+0000","last_undegraded":"2024-04-20T11:20:03.890988+0000","last_fullsized":"2024-04-20T11:20:03.890988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892088+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:20:03.892088+0000","last_peered":"2024-04-20T11:20:03.892088+0000","last_clean":"2024-04-20T11:20:03.892088+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:20:03.892088+0000","last_undegraded":"2024-04-20T11:20:03.892088+0000","last_fullsized":"2024-04-20T11:20:03.892088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690134+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:20:03.690134+0000","last_peered":"2024-04-20T11:20:03.690134+0000","last_clean":"2024-04-20T11:20:03.690134+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:20:03.690134+0000","last_undegraded":"2024-04-20T11:20:03.690134+0000","last_fullsized":"2024-04-20T11:20:03.690134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690758+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:20:03.690758+0000","last_peered":"2024-04-20T11:20:03.690758+0000","last_clean":"2024-04-20T11:20:03.690758+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:20:03.690758+0000","last_undegraded":"2024-04-20T11:20:03.690758+0000","last_fullsized":"2024-04-20T11:20:03.690758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.823191+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:20:07.823191+0000","last_peered":"2024-04-20T11:20:07.823191+0000","last_clean":"2024-04-20T11:20:07.823191+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:20:07.823191+0000","last_undegraded":"2024-04-20T11:20:07.823191+0000","last_fullsized":"2024-04-20T11:20:07.823191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699467+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:20:03.699467+0000","last_peered":"2024-04-20T11:20:03.699467+0000","last_clean":"2024-04-20T11:20:03.699467+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:20:03.699467+0000","last_undegraded":"2024-04-20T11:20:03.699467+0000","last_fullsized":"2024-04-20T11:20:03.699467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699551+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:20:03.699551+0000","last_peered":"2024-04-20T11:20:03.699551+0000","last_clean":"2024-04-20T11:20:03.699551+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:20:03.699551+0000","last_undegraded":"2024-04-20T11:20:03.699551+0000","last_fullsized":"2024-04-20T11:20:03.699551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892998+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:20:03.892998+0000","last_peered":"2024-04-20T11:20:03.892998+0000","last_clean":"2024-04-20T11:20:03.892998+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:20:03.892998+0000","last_undegraded":"2024-04-20T11:20:03.892998+0000","last_fullsized":"2024-04-20T11:20:03.892998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.820459+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:20:07.820459+0000","last_peered":"2024-04-20T11:20:07.820459+0000","last_clean":"2024-04-20T11:20:07.820459+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:20:07.820459+0000","last_undegraded":"2024-04-20T11:20:07.820459+0000","last_fullsized":"2024-04-20T11:20:07.820459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891206+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:20:03.891206+0000","last_peered":"2024-04-20T11:20:03.891206+0000","last_clean":"2024-04-20T11:20:03.891206+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:20:03.891206+0000","last_undegraded":"2024-04-20T11:20:03.891206+0000","last_fullsized":"2024-04-20T11:20:03.891206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699855+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:20:03.699855+0000","last_peered":"2024-04-20T11:20:03.699855+0000","last_clean":"2024-04-20T11:20:03.699855+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:20:03.699855+0000","last_undegraded":"2024-04-20T11:20:03.699855+0000","last_fullsized":"2024-04-20T11:20:03.699855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892149+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:20:03.892149+0000","last_peered":"2024-04-20T11:20:03.892149+0000","last_clean":"2024-04-20T11:20:03.892149+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:20:03.892149+0000","last_undegraded":"2024-04-20T11:20:03.892149+0000","last_fullsized":"2024-04-20T11:20:03.892149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.808083+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:20:07.808083+0000","last_peered":"2024-04-20T11:20:07.808083+0000","last_clean":"2024-04-20T11:20:07.808083+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:20:07.808083+0000","last_undegraded":"2024-04-20T11:20:07.808083+0000","last_fullsized":"2024-04-20T11:20:07.808083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699509+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:20:03.699509+0000","last_peered":"2024-04-20T11:20:03.699509+0000","last_clean":"2024-04-20T11:20:03.699509+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:20:03.699509+0000","last_undegraded":"2024-04-20T11:20:03.699509+0000","last_fullsized":"2024-04-20T11:20:03.699509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.825584+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:20:07.825584+0000","last_peered":"2024-04-20T11:20:07.825584+0000","last_clean":"2024-04-20T11:20:07.825584+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:20:07.825584+0000","last_undegraded":"2024-04-20T11:20:07.825584+0000","last_fullsized":"2024-04-20T11:20:07.825584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892088+0000","last_change":"2024-04-20T11:20:02.714927+0000","last_active":"2024-04-20T11:20:03.892088+0000","last_peered":"2024-04-20T11:20:03.892088+0000","last_clean":"2024-04-20T11:20:03.892088+0000","last_became_active":"2024-04-20T11:20:02.714633+0000","last_became_peered":"2024-04-20T11:20:02.714633+0000","last_unstale":"2024-04-20T11:20:03.892088+0000","last_undegraded":"2024-04-20T11:20:03.892088+0000","last_fullsized":"2024-04-20T11:20:03.892088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:02.606417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.082159+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:20:07.082159+0000","last_peered":"2024-04-20T11:20:07.082159+0000","last_clean":"2024-04-20T11:20:07.082159+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:20:07.082159+0000","last_undegraded":"2024-04-20T11:20:07.082159+0000","last_fullsized":"2024-04-20T11:20:07.082159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699450+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:20:03.699450+0000","last_peered":"2024-04-20T11:20:03.699450+0000","last_clean":"2024-04-20T11:20:03.699450+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:20:03.699450+0000","last_undegraded":"2024-04-20T11:20:03.699450+0000","last_fullsized":"2024-04-20T11:20:03.699450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.818942+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:20:07.818942+0000","last_peered":"2024-04-20T11:20:07.818942+0000","last_clean":"2024-04-20T11:20:07.818942+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:20:07.818942+0000","last_undegraded":"2024-04-20T11:20:07.818942+0000","last_fullsized":"2024-04-20T11:20:07.818942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690428+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:20:03.690428+0000","last_peered":"2024-04-20T11:20:03.690428+0000","last_clean":"2024-04-20T11:20:03.690428+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:20:03.690428+0000","last_undegraded":"2024-04-20T11:20:03.690428+0000","last_fullsized":"2024-04-20T11:20:03.690428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:10.811347+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:20:10.811347+0000","last_peered":"2024-04-20T11:20:10.811347+0000","last_clean":"2024-04-20T11:20:10.811347+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:20:10.811347+0000","last_undegraded":"2024-04-20T11:20:10.811347+0000","last_fullsized":"2024-04-20T11:20:10.811347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892077+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:20:03.892077+0000","last_peered":"2024-04-20T11:20:03.892077+0000","last_clean":"2024-04-20T11:20:03.892077+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:20:03.892077+0000","last_undegraded":"2024-04-20T11:20:03.892077+0000","last_fullsized":"2024-04-20T11:20:03.892077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.066606+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:20:06.066606+0000","last_peered":"2024-04-20T11:20:06.066606+0000","last_clean":"2024-04-20T11:20:06.066606+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:20:06.066606+0000","last_undegraded":"2024-04-20T11:20:06.066606+0000","last_fullsized":"2024-04-20T11:20:06.066606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892998+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:20:03.892998+0000","last_peered":"2024-04-20T11:20:03.892998+0000","last_clean":"2024-04-20T11:20:03.892998+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:20:03.892998+0000","last_undegraded":"2024-04-20T11:20:03.892998+0000","last_fullsized":"2024-04-20T11:20:03.892998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.807398+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:20:07.807398+0000","last_peered":"2024-04-20T11:20:07.807398+0000","last_clean":"2024-04-20T11:20:07.807398+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:20:07.807398+0000","last_undegraded":"2024-04-20T11:20:07.807398+0000","last_fullsized":"2024-04-20T11:20:07.807398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891475+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:20:03.891475+0000","last_peered":"2024-04-20T11:20:03.891475+0000","last_clean":"2024-04-20T11:20:03.891475+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:20:03.891475+0000","last_undegraded":"2024-04-20T11:20:03.891475+0000","last_fullsized":"2024-04-20T11:20:03.891475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'21","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:10.812258+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:20:10.812258+0000","last_peered":"2024-04-20T11:20:10.812258+0000","last_clean":"2024-04-20T11:20:10.812258+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:20:10.812258+0000","last_undegraded":"2024-04-20T11:20:10.812258+0000","last_fullsized":"2024-04-20T11:20:10.812258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":42,"num_write":23,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699819+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:20:03.699819+0000","last_peered":"2024-04-20T11:20:03.699819+0000","last_clean":"2024-04-20T11:20:03.699819+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:20:03.699819+0000","last_undegraded":"2024-04-20T11:20:03.699819+0000","last_fullsized":"2024-04-20T11:20:03.699819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.803704+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:20:07.803704+0000","last_peered":"2024-04-20T11:20:07.803704+0000","last_clean":"2024-04-20T11:20:07.803704+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:20:07.803704+0000","last_undegraded":"2024-04-20T11:20:07.803704+0000","last_fullsized":"2024-04-20T11:20:07.803704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891561+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:20:03.891561+0000","last_peered":"2024-04-20T11:20:03.891561+0000","last_clean":"2024-04-20T11:20:03.891561+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:20:03.891561+0000","last_undegraded":"2024-04-20T11:20:03.891561+0000","last_fullsized":"2024-04-20T11:20:03.891561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.825640+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:20:07.825640+0000","last_peered":"2024-04-20T11:20:07.825640+0000","last_clean":"2024-04-20T11:20:07.825640+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:20:07.825640+0000","last_undegraded":"2024-04-20T11:20:07.825640+0000","last_fullsized":"2024-04-20T11:20:07.825640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251554+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:20:04.251554+0000","last_peered":"2024-04-20T11:20:04.251554+0000","last_clean":"2024-04-20T11:20:04.251554+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:20:04.251554+0000","last_undegraded":"2024-04-20T11:20:04.251554+0000","last_fullsized":"2024-04-20T11:20:04.251554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.812072+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:20:08.812072+0000","last_peered":"2024-04-20T11:20:08.812072+0000","last_clean":"2024-04-20T11:20:08.812072+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:20:08.812072+0000","last_undegraded":"2024-04-20T11:20:08.812072+0000","last_fullsized":"2024-04-20T11:20:08.812072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251240+0000","last_change":"2024-04-20T11:20:02.691436+0000","last_active":"2024-04-20T11:20:04.251240+0000","last_peered":"2024-04-20T11:20:04.251240+0000","last_clean":"2024-04-20T11:20:04.251240+0000","last_became_active":"2024-04-20T11:20:02.691113+0000","last_became_peered":"2024-04-20T11:20:02.691113+0000","last_unstale":"2024-04-20T11:20:04.251240+0000","last_undegraded":"2024-04-20T11:20:04.251240+0000","last_fullsized":"2024-04-20T11:20:04.251240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:08:46.743698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.805202+0000","last_change":"2024-04-20T11:20:01.665426+0000","last_active":"2024-04-20T11:20:07.805202+0000","last_peered":"2024-04-20T11:20:07.805202+0000","last_clean":"2024-04-20T11:20:07.805202+0000","last_became_active":"2024-04-20T11:20:01.665149+0000","last_became_peered":"2024-04-20T11:20:01.665149+0000","last_unstale":"2024-04-20T11:20:07.805202+0000","last_undegraded":"2024-04-20T11:20:07.805202+0000","last_fullsized":"2024-04-20T11:20:07.805202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:58:25.728405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891708+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:20:03.891708+0000","last_peered":"2024-04-20T11:20:03.891708+0000","last_clean":"2024-04-20T11:20:03.891708+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:20:03.891708+0000","last_undegraded":"2024-04-20T11:20:03.891708+0000","last_fullsized":"2024-04-20T11:20:03.891708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699859+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:20:03.699859+0000","last_peered":"2024-04-20T11:20:03.699859+0000","last_clean":"2024-04-20T11:20:03.699859+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:20:03.699859+0000","last_undegraded":"2024-04-20T11:20:03.699859+0000","last_fullsized":"2024-04-20T11:20:03.699859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891669+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:20:03.891669+0000","last_peered":"2024-04-20T11:20:03.891669+0000","last_clean":"2024-04-20T11:20:03.891669+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:20:03.891669+0000","last_undegraded":"2024-04-20T11:20:03.891669+0000","last_fullsized":"2024-04-20T11:20:03.891669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690077+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:20:03.690077+0000","last_peered":"2024-04-20T11:20:03.690077+0000","last_clean":"2024-04-20T11:20:03.690077+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:20:03.690077+0000","last_undegraded":"2024-04-20T11:20:03.690077+0000","last_fullsized":"2024-04-20T11:20:03.690077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.111756+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:20:06.111756+0000","last_peered":"2024-04-20T11:20:06.111756+0000","last_clean":"2024-04-20T11:20:06.111756+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:20:06.111756+0000","last_undegraded":"2024-04-20T11:20:06.111756+0000","last_fullsized":"2024-04-20T11:20:06.111756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251545+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:20:04.251545+0000","last_peered":"2024-04-20T11:20:04.251545+0000","last_clean":"2024-04-20T11:20:04.251545+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:20:04.251545+0000","last_undegraded":"2024-04-20T11:20:04.251545+0000","last_fullsized":"2024-04-20T11:20:04.251545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.693364+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:20:03.693364+0000","last_peered":"2024-04-20T11:20:03.693364+0000","last_clean":"2024-04-20T11:20:03.693364+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:20:03.693364+0000","last_undegraded":"2024-04-20T11:20:03.693364+0000","last_fullsized":"2024-04-20T11:20:03.693364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891395+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:20:03.891395+0000","last_peered":"2024-04-20T11:20:03.891395+0000","last_clean":"2024-04-20T11:20:03.891395+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:20:03.891395+0000","last_undegraded":"2024-04-20T11:20:03.891395+0000","last_fullsized":"2024-04-20T11:20:03.891395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:09.813492+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:20:09.813492+0000","last_peered":"2024-04-20T11:20:09.813492+0000","last_clean":"2024-04-20T11:20:09.813492+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:20:09.813492+0000","last_undegraded":"2024-04-20T11:20:09.813492+0000","last_fullsized":"2024-04-20T11:20:09.813492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699134+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:20:03.699134+0000","last_peered":"2024-04-20T11:20:03.699134+0000","last_clean":"2024-04-20T11:20:03.699134+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:20:03.699134+0000","last_undegraded":"2024-04-20T11:20:03.699134+0000","last_fullsized":"2024-04-20T11:20:03.699134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.825301+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:20:07.825301+0000","last_peered":"2024-04-20T11:20:07.825301+0000","last_clean":"2024-04-20T11:20:07.825301+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:20:07.825301+0000","last_undegraded":"2024-04-20T11:20:07.825301+0000","last_fullsized":"2024-04-20T11:20:07.825301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251261+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:20:04.251261+0000","last_peered":"2024-04-20T11:20:04.251261+0000","last_clean":"2024-04-20T11:20:04.251261+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:20:04.251261+0000","last_undegraded":"2024-04-20T11:20:04.251261+0000","last_fullsized":"2024-04-20T11:20:04.251261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891313+0000","last_change":"2024-04-20T11:20:02.691443+0000","last_active":"2024-04-20T11:20:03.891313+0000","last_peered":"2024-04-20T11:20:03.891313+0000","last_clean":"2024-04-20T11:20:03.891313+0000","last_became_active":"2024-04-20T11:20:02.690998+0000","last_became_peered":"2024-04-20T11:20:02.690998+0000","last_unstale":"2024-04-20T11:20:03.891313+0000","last_undegraded":"2024-04-20T11:20:03.891313+0000","last_fullsized":"2024-04-20T11:20:03.891313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:05:29.485535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.076369+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:20:07.076369+0000","last_peered":"2024-04-20T11:20:07.076369+0000","last_clean":"2024-04-20T11:20:07.076369+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:20:07.076369+0000","last_undegraded":"2024-04-20T11:20:07.076369+0000","last_fullsized":"2024-04-20T11:20:07.076369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.823850+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:20:07.823850+0000","last_peered":"2024-04-20T11:20:07.823850+0000","last_clean":"2024-04-20T11:20:07.823850+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:20:07.823850+0000","last_undegraded":"2024-04-20T11:20:07.823850+0000","last_fullsized":"2024-04-20T11:20:07.823850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891538+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:20:03.891538+0000","last_peered":"2024-04-20T11:20:03.891538+0000","last_clean":"2024-04-20T11:20:03.891538+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:20:03.891538+0000","last_undegraded":"2024-04-20T11:20:03.891538+0000","last_fullsized":"2024-04-20T11:20:03.891538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.826204+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:20:07.826204+0000","last_peered":"2024-04-20T11:20:07.826204+0000","last_clean":"2024-04-20T11:20:07.826204+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:20:07.826204+0000","last_undegraded":"2024-04-20T11:20:07.826204+0000","last_fullsized":"2024-04-20T11:20:07.826204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699701+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:20:03.699701+0000","last_peered":"2024-04-20T11:20:03.699701+0000","last_clean":"2024-04-20T11:20:03.699701+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:20:03.699701+0000","last_undegraded":"2024-04-20T11:20:03.699701+0000","last_fullsized":"2024-04-20T11:20:03.699701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.849634+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:20:08.849634+0000","last_peered":"2024-04-20T11:20:08.849634+0000","last_clean":"2024-04-20T11:20:08.849634+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:20:08.849634+0000","last_undegraded":"2024-04-20T11:20:08.849634+0000","last_fullsized":"2024-04-20T11:20:08.849634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892629+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:20:03.892629+0000","last_peered":"2024-04-20T11:20:03.892629+0000","last_clean":"2024-04-20T11:20:03.892629+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:20:03.892629+0000","last_undegraded":"2024-04-20T11:20:03.892629+0000","last_fullsized":"2024-04-20T11:20:03.892629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.070422+0000","last_change":"2024-04-20T11:20:01.675692+0000","last_active":"2024-04-20T11:20:06.070422+0000","last_peered":"2024-04-20T11:20:06.070422+0000","last_clean":"2024-04-20T11:20:06.070422+0000","last_became_active":"2024-04-20T11:20:01.675373+0000","last_became_peered":"2024-04-20T11:20:01.675373+0000","last_unstale":"2024-04-20T11:20:06.070422+0000","last_undegraded":"2024-04-20T11:20:06.070422+0000","last_fullsized":"2024-04-20T11:20:06.070422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:57.667811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699348+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:20:03.699348+0000","last_peered":"2024-04-20T11:20:03.699348+0000","last_clean":"2024-04-20T11:20:03.699348+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:20:03.699348+0000","last_undegraded":"2024-04-20T11:20:03.699348+0000","last_fullsized":"2024-04-20T11:20:03.699348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.846708+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:20:08.846708+0000","last_peered":"2024-04-20T11:20:08.846708+0000","last_clean":"2024-04-20T11:20:08.846708+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:20:08.846708+0000","last_undegraded":"2024-04-20T11:20:08.846708+0000","last_fullsized":"2024-04-20T11:20:08.846708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891221+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:20:03.891221+0000","last_peered":"2024-04-20T11:20:03.891221+0000","last_clean":"2024-04-20T11:20:03.891221+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:20:03.891221+0000","last_undegraded":"2024-04-20T11:20:03.891221+0000","last_fullsized":"2024-04-20T11:20:03.891221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.062133+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:20:07.062133+0000","last_peered":"2024-04-20T11:20:07.062133+0000","last_clean":"2024-04-20T11:20:07.062133+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:20:07.062133+0000","last_undegraded":"2024-04-20T11:20:07.062133+0000","last_fullsized":"2024-04-20T11:20:07.062133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699157+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:20:03.699157+0000","last_peered":"2024-04-20T11:20:03.699157+0000","last_clean":"2024-04-20T11:20:03.699157+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:20:03.699157+0000","last_undegraded":"2024-04-20T11:20:03.699157+0000","last_fullsized":"2024-04-20T11:20:03.699157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.067422+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:20:06.067422+0000","last_peered":"2024-04-20T11:20:06.067422+0000","last_clean":"2024-04-20T11:20:06.067422+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:20:06.067422+0000","last_undegraded":"2024-04-20T11:20:06.067422+0000","last_fullsized":"2024-04-20T11:20:06.067422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251683+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:20:04.251683+0000","last_peered":"2024-04-20T11:20:04.251683+0000","last_clean":"2024-04-20T11:20:04.251683+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:20:04.251683+0000","last_undegraded":"2024-04-20T11:20:04.251683+0000","last_fullsized":"2024-04-20T11:20:04.251683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.822542+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:20:07.822542+0000","last_peered":"2024-04-20T11:20:07.822542+0000","last_clean":"2024-04-20T11:20:07.822542+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:20:07.822542+0000","last_undegraded":"2024-04-20T11:20:07.822542+0000","last_fullsized":"2024-04-20T11:20:07.822542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251831+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:20:04.251831+0000","last_peered":"2024-04-20T11:20:04.251831+0000","last_clean":"2024-04-20T11:20:04.251831+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:20:04.251831+0000","last_undegraded":"2024-04-20T11:20:04.251831+0000","last_fullsized":"2024-04-20T11:20:04.251831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.823128+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:20:07.823128+0000","last_peered":"2024-04-20T11:20:07.823128+0000","last_clean":"2024-04-20T11:20:07.823128+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:20:07.823128+0000","last_undegraded":"2024-04-20T11:20:07.823128+0000","last_fullsized":"2024-04-20T11:20:07.823128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251520+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:20:04.251520+0000","last_peered":"2024-04-20T11:20:04.251520+0000","last_clean":"2024-04-20T11:20:04.251520+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:20:04.251520+0000","last_undegraded":"2024-04-20T11:20:04.251520+0000","last_fullsized":"2024-04-20T11:20:04.251520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.849042+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:20:08.849042+0000","last_peered":"2024-04-20T11:20:08.849042+0000","last_clean":"2024-04-20T11:20:08.849042+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:20:08.849042+0000","last_undegraded":"2024-04-20T11:20:08.849042+0000","last_fullsized":"2024-04-20T11:20:08.849042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251768+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:20:04.251768+0000","last_peered":"2024-04-20T11:20:04.251768+0000","last_clean":"2024-04-20T11:20:04.251768+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:20:04.251768+0000","last_undegraded":"2024-04-20T11:20:04.251768+0000","last_fullsized":"2024-04-20T11:20:04.251768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891338+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:20:03.891338+0000","last_peered":"2024-04-20T11:20:03.891338+0000","last_clean":"2024-04-20T11:20:03.891338+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:20:03.891338+0000","last_undegraded":"2024-04-20T11:20:03.891338+0000","last_fullsized":"2024-04-20T11:20:03.891338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690545+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:20:03.690545+0000","last_peered":"2024-04-20T11:20:03.690545+0000","last_clean":"2024-04-20T11:20:03.690545+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:20:03.690545+0000","last_undegraded":"2024-04-20T11:20:03.690545+0000","last_fullsized":"2024-04-20T11:20:03.690545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690425+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:20:03.690425+0000","last_peered":"2024-04-20T11:20:03.690425+0000","last_clean":"2024-04-20T11:20:03.690425+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:20:03.690425+0000","last_undegraded":"2024-04-20T11:20:03.690425+0000","last_fullsized":"2024-04-20T11:20:03.690425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892411+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:20:03.892411+0000","last_peered":"2024-04-20T11:20:03.892411+0000","last_clean":"2024-04-20T11:20:03.892411+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:20:03.892411+0000","last_undegraded":"2024-04-20T11:20:03.892411+0000","last_fullsized":"2024-04-20T11:20:03.892411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891700+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:20:03.891700+0000","last_peered":"2024-04-20T11:20:03.891700+0000","last_clean":"2024-04-20T11:20:03.891700+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:20:03.891700+0000","last_undegraded":"2024-04-20T11:20:03.891700+0000","last_fullsized":"2024-04-20T11:20:03.891700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891564+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:20:03.891564+0000","last_peered":"2024-04-20T11:20:03.891564+0000","last_clean":"2024-04-20T11:20:03.891564+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:20:03.891564+0000","last_undegraded":"2024-04-20T11:20:03.891564+0000","last_fullsized":"2024-04-20T11:20:03.891564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690981+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:20:03.690981+0000","last_peered":"2024-04-20T11:20:03.690981+0000","last_clean":"2024-04-20T11:20:03.690981+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:20:03.690981+0000","last_undegraded":"2024-04-20T11:20:03.690981+0000","last_fullsized":"2024-04-20T11:20:03.690981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699545+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:20:03.699545+0000","last_peered":"2024-04-20T11:20:03.699545+0000","last_clean":"2024-04-20T11:20:03.699545+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:20:03.699545+0000","last_undegraded":"2024-04-20T11:20:03.699545+0000","last_fullsized":"2024-04-20T11:20:03.699545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891513+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:20:03.891513+0000","last_peered":"2024-04-20T11:20:03.891513+0000","last_clean":"2024-04-20T11:20:03.891513+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:20:03.891513+0000","last_undegraded":"2024-04-20T11:20:03.891513+0000","last_fullsized":"2024-04-20T11:20:03.891513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699751+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:20:03.699751+0000","last_peered":"2024-04-20T11:20:03.699751+0000","last_clean":"2024-04-20T11:20:03.699751+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:20:03.699751+0000","last_undegraded":"2024-04-20T11:20:03.699751+0000","last_fullsized":"2024-04-20T11:20:03.699751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891198+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:20:03.891198+0000","last_peered":"2024-04-20T11:20:03.891198+0000","last_clean":"2024-04-20T11:20:03.891198+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:20:03.891198+0000","last_undegraded":"2024-04-20T11:20:03.891198+0000","last_fullsized":"2024-04-20T11:20:03.891198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891897+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:20:03.891897+0000","last_peered":"2024-04-20T11:20:03.891897+0000","last_clean":"2024-04-20T11:20:03.891897+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:20:03.891897+0000","last_undegraded":"2024-04-20T11:20:03.891897+0000","last_fullsized":"2024-04-20T11:20:03.891897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891452+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:20:03.891452+0000","last_peered":"2024-04-20T11:20:03.891452+0000","last_clean":"2024-04-20T11:20:03.891452+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:20:03.891452+0000","last_undegraded":"2024-04-20T11:20:03.891452+0000","last_fullsized":"2024-04-20T11:20:03.891452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282095,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43656,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":45,"num_read_kb":71,"num_write":192,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":65017,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":168,"ondisk_log_size":168,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1069326,"data_compressed":37716,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":439429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":443703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":440587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":441483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":439185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1031127,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":9964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":40320,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:20:12.294 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 -- 172.21.15.44:0/1357674574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 msgr2=0x7f3b14103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 --2- 172.21.15.44:0/1357674574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14103640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009900 tx=0x7f3b0002bc40 comp rx=0 tx=0).stop 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 -- 172.21.15.44:0/1357674574 shutdown_connections 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 --2- 172.21.15.44:0/1357674574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14103b80 0x7f3b1410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 --2- 172.21.15.44:0/1357674574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14103640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 --2- 172.21.15.44:0/1357674574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b14101fd0 0x7f3b141023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 -- 172.21.15.44:0/1357674574 >> 172.21.15.44:0/1357674574 conn(0x7f3b140fd540 msgr2=0x7f3b140ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 -- 172.21.15.44:0/1357674574 shutdown_connections 2024-04-20T11:20:12.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.548+0000 7f3b1b800640 1 -- 172.21.15.44:0/1357674574 wait complete. 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.549+0000 7f3b1b800640 1 Processor -- start 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.549+0000 7f3b1b800640 1 -- start start 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b1b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14101fd0 0x7f3b14075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b1b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b1b800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b14103b80 0x7f3b14074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b1b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1410dd80 con 0x7f3b141031c0 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b1b800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1410dc00 con 0x7f3b14103b80 2024-04-20T11:20:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b1b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1410df00 con 0x7f3b14101fd0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50584/0 (socket says 172.21.15.44:50584) 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b18e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14101fd0 0x7f3b14075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b13e00640 1 -- 172.21.15.44:0/3794713857 learned_addr learned my addr 172.21.15.44:0/3794713857 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.550+0000 7f3b19800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b14103b80 0x7f3b14074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b13e00640 1 -- 172.21.15.44:0/3794713857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14101fd0 msgr2=0x7f3b14075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b13e00640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14101fd0 0x7f3b14075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b13e00640 1 -- 172.21.15.44:0/3794713857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b14103b80 msgr2=0x7f3b14074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b13e00640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b14103b80 0x7f3b14074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b13e00640 1 -- 172.21.15.44:0/3794713857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b14075120 con 0x7f3b141031c0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b13e00640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14074350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009230 tx=0x7f3b00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00043020 con 0x7f3b141031c0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b00004250 con 0x7f3b141031c0 2024-04-20T11:20:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00003b50 con 0x7f3b141031c0 2024-04-20T11:20:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.551+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b141a24b0 con 0x7f3b141031c0 2024-04-20T11:20:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.552+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b141a29b0 con 0x7f3b141031c0 2024-04-20T11:20:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.553+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b141092f0 con 0x7f3b141031c0 2024-04-20T11:20:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.555+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b00046020 con 0x7f3b141031c0 2024-04-20T11:20:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.557+0000 7f3b11600640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3af80a3cc0 0x7f3af80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.557+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3b000f2400 con 0x7f3b141031c0 2024-04-20T11:20:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.557+0000 7f3b18e00640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3af80a3cc0 0x7f3af80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:12.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.558+0000 7f3b18e00640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3af80a3cc0 0x7f3af80a6180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b040095f0 tx=0x7f3b04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.561+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3b0004d020 con 0x7f3b141031c0 2024-04-20T11:20:12.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.771+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b140623a0 con 0x7f3af80a3cc0 2024-04-20T11:20:12.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.787+0000 7f3b11600640 1 -- 172.21.15.44:0/3794713857 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258956 (secure 0 0 0) 0x7f3b140623a0 con 0x7f3af80a3cc0 2024-04-20T11:20:12.787 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:20:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.793+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3af80a3cc0 msgr2=0x7f3af80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.793+0000 7f3b1b800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3af80a3cc0 0x7f3af80a6180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b040095f0 tx=0x7f3b04005f90 comp rx=0 tx=0).stop 2024-04-20T11:20:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.793+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 msgr2=0x7f3b14074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.793+0000 7f3b1b800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14074350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009230 tx=0x7f3b00004ab0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 shutdown_connections 2024-04-20T11:20:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3af80a3cc0 0x7f3af80a6180 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b14103b80 0x7f3b14074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b141031c0 0x7f3b14074350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 --2- 172.21.15.44:0/3794713857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b14101fd0 0x7f3b14075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 >> 172.21.15.44:0/3794713857 conn(0x7f3b140fd540 msgr2=0x7f3b141064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 shutdown_connections 2024-04-20T11:20:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:12.794+0000 7f3b1b800640 1 -- 172.21.15.44:0/3794713857 wait complete. 2024-04-20T11:20:12.814 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-20T11:20:12.047732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634141,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":181,"num_read_kb":190,"num_write":374,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360348,"kb_used_data":11196,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581476,"statfs":{"total":1151940427776,"available":1151571431424,"internally_reserved":0,"allocated":11464704,"data_stored":7091651,"data_compressed":109472,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43773,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":45,"num_read_kb":71,"num_write":193,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.379374"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891893+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:20:03.891893+0000","last_peered":"2024-04-20T11:20:03.891893+0000","last_clean":"2024-04-20T11:20:03.891893+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:20:03.891893+0000","last_undegraded":"2024-04-20T11:20:03.891893+0000","last_fullsized":"2024-04-20T11:20:03.891893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892531+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:20:03.892531+0000","last_peered":"2024-04-20T11:20:03.892531+0000","last_clean":"2024-04-20T11:20:03.892531+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:20:03.892531+0000","last_undegraded":"2024-04-20T11:20:03.892531+0000","last_fullsized":"2024-04-20T11:20:03.892531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892520+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:20:03.892520+0000","last_peered":"2024-04-20T11:20:03.892520+0000","last_clean":"2024-04-20T11:20:03.892520+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:20:03.892520+0000","last_undegraded":"2024-04-20T11:20:03.892520+0000","last_fullsized":"2024-04-20T11:20:03.892520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891747+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:20:03.891747+0000","last_peered":"2024-04-20T11:20:03.891747+0000","last_clean":"2024-04-20T11:20:03.891747+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:20:03.891747+0000","last_undegraded":"2024-04-20T11:20:03.891747+0000","last_fullsized":"2024-04-20T11:20:03.891747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699499+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:20:03.699499+0000","last_peered":"2024-04-20T11:20:03.699499+0000","last_clean":"2024-04-20T11:20:03.699499+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:20:03.699499+0000","last_undegraded":"2024-04-20T11:20:03.699499+0000","last_fullsized":"2024-04-20T11:20:03.699499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.693592+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:20:03.693592+0000","last_peered":"2024-04-20T11:20:03.693592+0000","last_clean":"2024-04-20T11:20:03.693592+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:20:03.693592+0000","last_undegraded":"2024-04-20T11:20:03.693592+0000","last_fullsized":"2024-04-20T11:20:03.693592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251290+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:20:04.251290+0000","last_peered":"2024-04-20T11:20:04.251290+0000","last_clean":"2024-04-20T11:20:04.251290+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:20:04.251290+0000","last_undegraded":"2024-04-20T11:20:04.251290+0000","last_fullsized":"2024-04-20T11:20:04.251290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.698816+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:20:03.698816+0000","last_peered":"2024-04-20T11:20:03.698816+0000","last_clean":"2024-04-20T11:20:03.698816+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:20:03.698816+0000","last_undegraded":"2024-04-20T11:20:03.698816+0000","last_fullsized":"2024-04-20T11:20:03.698816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891547+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:20:03.891547+0000","last_peered":"2024-04-20T11:20:03.891547+0000","last_clean":"2024-04-20T11:20:03.891547+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:20:03.891547+0000","last_undegraded":"2024-04-20T11:20:03.891547+0000","last_fullsized":"2024-04-20T11:20:03.891547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.693557+0000","last_change":"2024-04-20T11:20:02.697661+0000","last_active":"2024-04-20T11:20:03.693557+0000","last_peered":"2024-04-20T11:20:03.693557+0000","last_clean":"2024-04-20T11:20:03.693557+0000","last_became_active":"2024-04-20T11:20:02.697362+0000","last_became_peered":"2024-04-20T11:20:02.697362+0000","last_unstale":"2024-04-20T11:20:03.693557+0000","last_undegraded":"2024-04-20T11:20:03.693557+0000","last_fullsized":"2024-04-20T11:20:03.693557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:53:17.606523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251722+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:20:04.251722+0000","last_peered":"2024-04-20T11:20:04.251722+0000","last_clean":"2024-04-20T11:20:04.251722+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:20:04.251722+0000","last_undegraded":"2024-04-20T11:20:04.251722+0000","last_fullsized":"2024-04-20T11:20:04.251722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891744+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:20:03.891744+0000","last_peered":"2024-04-20T11:20:03.891744+0000","last_clean":"2024-04-20T11:20:03.891744+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:20:03.891744+0000","last_undegraded":"2024-04-20T11:20:03.891744+0000","last_fullsized":"2024-04-20T11:20:03.891744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891983+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:20:03.891983+0000","last_peered":"2024-04-20T11:20:03.891983+0000","last_clean":"2024-04-20T11:20:03.891983+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:20:03.891983+0000","last_undegraded":"2024-04-20T11:20:03.891983+0000","last_fullsized":"2024-04-20T11:20:03.891983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892220+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:20:03.892220+0000","last_peered":"2024-04-20T11:20:03.892220+0000","last_clean":"2024-04-20T11:20:03.892220+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:20:03.892220+0000","last_undegraded":"2024-04-20T11:20:03.892220+0000","last_fullsized":"2024-04-20T11:20:03.892220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891994+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:20:03.891994+0000","last_peered":"2024-04-20T11:20:03.891994+0000","last_clean":"2024-04-20T11:20:03.891994+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:20:03.891994+0000","last_undegraded":"2024-04-20T11:20:03.891994+0000","last_fullsized":"2024-04-20T11:20:03.891994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.890988+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:20:03.890988+0000","last_peered":"2024-04-20T11:20:03.890988+0000","last_clean":"2024-04-20T11:20:03.890988+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:20:03.890988+0000","last_undegraded":"2024-04-20T11:20:03.890988+0000","last_fullsized":"2024-04-20T11:20:03.890988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892088+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:20:03.892088+0000","last_peered":"2024-04-20T11:20:03.892088+0000","last_clean":"2024-04-20T11:20:03.892088+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:20:03.892088+0000","last_undegraded":"2024-04-20T11:20:03.892088+0000","last_fullsized":"2024-04-20T11:20:03.892088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690134+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:20:03.690134+0000","last_peered":"2024-04-20T11:20:03.690134+0000","last_clean":"2024-04-20T11:20:03.690134+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:20:03.690134+0000","last_undegraded":"2024-04-20T11:20:03.690134+0000","last_fullsized":"2024-04-20T11:20:03.690134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690758+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:20:03.690758+0000","last_peered":"2024-04-20T11:20:03.690758+0000","last_clean":"2024-04-20T11:20:03.690758+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:20:03.690758+0000","last_undegraded":"2024-04-20T11:20:03.690758+0000","last_fullsized":"2024-04-20T11:20:03.690758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.823191+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:20:07.823191+0000","last_peered":"2024-04-20T11:20:07.823191+0000","last_clean":"2024-04-20T11:20:07.823191+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:20:07.823191+0000","last_undegraded":"2024-04-20T11:20:07.823191+0000","last_fullsized":"2024-04-20T11:20:07.823191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699467+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:20:03.699467+0000","last_peered":"2024-04-20T11:20:03.699467+0000","last_clean":"2024-04-20T11:20:03.699467+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:20:03.699467+0000","last_undegraded":"2024-04-20T11:20:03.699467+0000","last_fullsized":"2024-04-20T11:20:03.699467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699551+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:20:03.699551+0000","last_peered":"2024-04-20T11:20:03.699551+0000","last_clean":"2024-04-20T11:20:03.699551+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:20:03.699551+0000","last_undegraded":"2024-04-20T11:20:03.699551+0000","last_fullsized":"2024-04-20T11:20:03.699551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892998+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:20:03.892998+0000","last_peered":"2024-04-20T11:20:03.892998+0000","last_clean":"2024-04-20T11:20:03.892998+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:20:03.892998+0000","last_undegraded":"2024-04-20T11:20:03.892998+0000","last_fullsized":"2024-04-20T11:20:03.892998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.820459+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:20:07.820459+0000","last_peered":"2024-04-20T11:20:07.820459+0000","last_clean":"2024-04-20T11:20:07.820459+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:20:07.820459+0000","last_undegraded":"2024-04-20T11:20:07.820459+0000","last_fullsized":"2024-04-20T11:20:07.820459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891206+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:20:03.891206+0000","last_peered":"2024-04-20T11:20:03.891206+0000","last_clean":"2024-04-20T11:20:03.891206+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:20:03.891206+0000","last_undegraded":"2024-04-20T11:20:03.891206+0000","last_fullsized":"2024-04-20T11:20:03.891206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699855+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:20:03.699855+0000","last_peered":"2024-04-20T11:20:03.699855+0000","last_clean":"2024-04-20T11:20:03.699855+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:20:03.699855+0000","last_undegraded":"2024-04-20T11:20:03.699855+0000","last_fullsized":"2024-04-20T11:20:03.699855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892149+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:20:03.892149+0000","last_peered":"2024-04-20T11:20:03.892149+0000","last_clean":"2024-04-20T11:20:03.892149+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:20:03.892149+0000","last_undegraded":"2024-04-20T11:20:03.892149+0000","last_fullsized":"2024-04-20T11:20:03.892149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.808083+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:20:07.808083+0000","last_peered":"2024-04-20T11:20:07.808083+0000","last_clean":"2024-04-20T11:20:07.808083+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:20:07.808083+0000","last_undegraded":"2024-04-20T11:20:07.808083+0000","last_fullsized":"2024-04-20T11:20:07.808083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699509+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:20:03.699509+0000","last_peered":"2024-04-20T11:20:03.699509+0000","last_clean":"2024-04-20T11:20:03.699509+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:20:03.699509+0000","last_undegraded":"2024-04-20T11:20:03.699509+0000","last_fullsized":"2024-04-20T11:20:03.699509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.825584+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:20:07.825584+0000","last_peered":"2024-04-20T11:20:07.825584+0000","last_clean":"2024-04-20T11:20:07.825584+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:20:07.825584+0000","last_undegraded":"2024-04-20T11:20:07.825584+0000","last_fullsized":"2024-04-20T11:20:07.825584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892088+0000","last_change":"2024-04-20T11:20:02.714927+0000","last_active":"2024-04-20T11:20:03.892088+0000","last_peered":"2024-04-20T11:20:03.892088+0000","last_clean":"2024-04-20T11:20:03.892088+0000","last_became_active":"2024-04-20T11:20:02.714633+0000","last_became_peered":"2024-04-20T11:20:02.714633+0000","last_unstale":"2024-04-20T11:20:03.892088+0000","last_undegraded":"2024-04-20T11:20:03.892088+0000","last_fullsized":"2024-04-20T11:20:03.892088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:02.606417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.082159+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:20:07.082159+0000","last_peered":"2024-04-20T11:20:07.082159+0000","last_clean":"2024-04-20T11:20:07.082159+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:20:07.082159+0000","last_undegraded":"2024-04-20T11:20:07.082159+0000","last_fullsized":"2024-04-20T11:20:07.082159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699450+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:20:03.699450+0000","last_peered":"2024-04-20T11:20:03.699450+0000","last_clean":"2024-04-20T11:20:03.699450+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:20:03.699450+0000","last_undegraded":"2024-04-20T11:20:03.699450+0000","last_fullsized":"2024-04-20T11:20:03.699450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.818942+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:20:07.818942+0000","last_peered":"2024-04-20T11:20:07.818942+0000","last_clean":"2024-04-20T11:20:07.818942+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:20:07.818942+0000","last_undegraded":"2024-04-20T11:20:07.818942+0000","last_fullsized":"2024-04-20T11:20:07.818942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690428+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:20:03.690428+0000","last_peered":"2024-04-20T11:20:03.690428+0000","last_clean":"2024-04-20T11:20:03.690428+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:20:03.690428+0000","last_undegraded":"2024-04-20T11:20:03.690428+0000","last_fullsized":"2024-04-20T11:20:03.690428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:10.811347+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:20:10.811347+0000","last_peered":"2024-04-20T11:20:10.811347+0000","last_clean":"2024-04-20T11:20:10.811347+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:20:10.811347+0000","last_undegraded":"2024-04-20T11:20:10.811347+0000","last_fullsized":"2024-04-20T11:20:10.811347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892077+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:20:03.892077+0000","last_peered":"2024-04-20T11:20:03.892077+0000","last_clean":"2024-04-20T11:20:03.892077+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:20:03.892077+0000","last_undegraded":"2024-04-20T11:20:03.892077+0000","last_fullsized":"2024-04-20T11:20:03.892077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.066606+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:20:06.066606+0000","last_peered":"2024-04-20T11:20:06.066606+0000","last_clean":"2024-04-20T11:20:06.066606+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:20:06.066606+0000","last_undegraded":"2024-04-20T11:20:06.066606+0000","last_fullsized":"2024-04-20T11:20:06.066606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892998+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:20:03.892998+0000","last_peered":"2024-04-20T11:20:03.892998+0000","last_clean":"2024-04-20T11:20:03.892998+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:20:03.892998+0000","last_undegraded":"2024-04-20T11:20:03.892998+0000","last_fullsized":"2024-04-20T11:20:03.892998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.807398+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:20:07.807398+0000","last_peered":"2024-04-20T11:20:07.807398+0000","last_clean":"2024-04-20T11:20:07.807398+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:20:07.807398+0000","last_undegraded":"2024-04-20T11:20:07.807398+0000","last_fullsized":"2024-04-20T11:20:07.807398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891475+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:20:03.891475+0000","last_peered":"2024-04-20T11:20:03.891475+0000","last_clean":"2024-04-20T11:20:03.891475+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:20:03.891475+0000","last_undegraded":"2024-04-20T11:20:03.891475+0000","last_fullsized":"2024-04-20T11:20:03.891475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'21","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:10.812258+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:20:10.812258+0000","last_peered":"2024-04-20T11:20:10.812258+0000","last_clean":"2024-04-20T11:20:10.812258+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:20:10.812258+0000","last_undegraded":"2024-04-20T11:20:10.812258+0000","last_fullsized":"2024-04-20T11:20:10.812258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":42,"num_write":23,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699819+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:20:03.699819+0000","last_peered":"2024-04-20T11:20:03.699819+0000","last_clean":"2024-04-20T11:20:03.699819+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:20:03.699819+0000","last_undegraded":"2024-04-20T11:20:03.699819+0000","last_fullsized":"2024-04-20T11:20:03.699819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.803704+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:20:07.803704+0000","last_peered":"2024-04-20T11:20:07.803704+0000","last_clean":"2024-04-20T11:20:07.803704+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:20:07.803704+0000","last_undegraded":"2024-04-20T11:20:07.803704+0000","last_fullsized":"2024-04-20T11:20:07.803704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891561+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:20:03.891561+0000","last_peered":"2024-04-20T11:20:03.891561+0000","last_clean":"2024-04-20T11:20:03.891561+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:20:03.891561+0000","last_undegraded":"2024-04-20T11:20:03.891561+0000","last_fullsized":"2024-04-20T11:20:03.891561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.825640+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:20:07.825640+0000","last_peered":"2024-04-20T11:20:07.825640+0000","last_clean":"2024-04-20T11:20:07.825640+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:20:07.825640+0000","last_undegraded":"2024-04-20T11:20:07.825640+0000","last_fullsized":"2024-04-20T11:20:07.825640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251554+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:20:04.251554+0000","last_peered":"2024-04-20T11:20:04.251554+0000","last_clean":"2024-04-20T11:20:04.251554+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:20:04.251554+0000","last_undegraded":"2024-04-20T11:20:04.251554+0000","last_fullsized":"2024-04-20T11:20:04.251554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.812072+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:20:08.812072+0000","last_peered":"2024-04-20T11:20:08.812072+0000","last_clean":"2024-04-20T11:20:08.812072+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:20:08.812072+0000","last_undegraded":"2024-04-20T11:20:08.812072+0000","last_fullsized":"2024-04-20T11:20:08.812072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251240+0000","last_change":"2024-04-20T11:20:02.691436+0000","last_active":"2024-04-20T11:20:04.251240+0000","last_peered":"2024-04-20T11:20:04.251240+0000","last_clean":"2024-04-20T11:20:04.251240+0000","last_became_active":"2024-04-20T11:20:02.691113+0000","last_became_peered":"2024-04-20T11:20:02.691113+0000","last_unstale":"2024-04-20T11:20:04.251240+0000","last_undegraded":"2024-04-20T11:20:04.251240+0000","last_fullsized":"2024-04-20T11:20:04.251240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:08:46.743698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.805202+0000","last_change":"2024-04-20T11:20:01.665426+0000","last_active":"2024-04-20T11:20:07.805202+0000","last_peered":"2024-04-20T11:20:07.805202+0000","last_clean":"2024-04-20T11:20:07.805202+0000","last_became_active":"2024-04-20T11:20:01.665149+0000","last_became_peered":"2024-04-20T11:20:01.665149+0000","last_unstale":"2024-04-20T11:20:07.805202+0000","last_undegraded":"2024-04-20T11:20:07.805202+0000","last_fullsized":"2024-04-20T11:20:07.805202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:58:25.728405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891708+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:20:03.891708+0000","last_peered":"2024-04-20T11:20:03.891708+0000","last_clean":"2024-04-20T11:20:03.891708+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:20:03.891708+0000","last_undegraded":"2024-04-20T11:20:03.891708+0000","last_fullsized":"2024-04-20T11:20:03.891708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699859+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:20:03.699859+0000","last_peered":"2024-04-20T11:20:03.699859+0000","last_clean":"2024-04-20T11:20:03.699859+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:20:03.699859+0000","last_undegraded":"2024-04-20T11:20:03.699859+0000","last_fullsized":"2024-04-20T11:20:03.699859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891669+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:20:03.891669+0000","last_peered":"2024-04-20T11:20:03.891669+0000","last_clean":"2024-04-20T11:20:03.891669+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:20:03.891669+0000","last_undegraded":"2024-04-20T11:20:03.891669+0000","last_fullsized":"2024-04-20T11:20:03.891669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690077+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:20:03.690077+0000","last_peered":"2024-04-20T11:20:03.690077+0000","last_clean":"2024-04-20T11:20:03.690077+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:20:03.690077+0000","last_undegraded":"2024-04-20T11:20:03.690077+0000","last_fullsized":"2024-04-20T11:20:03.690077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.111756+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:20:06.111756+0000","last_peered":"2024-04-20T11:20:06.111756+0000","last_clean":"2024-04-20T11:20:06.111756+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:20:06.111756+0000","last_undegraded":"2024-04-20T11:20:06.111756+0000","last_fullsized":"2024-04-20T11:20:06.111756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251545+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:20:04.251545+0000","last_peered":"2024-04-20T11:20:04.251545+0000","last_clean":"2024-04-20T11:20:04.251545+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:20:04.251545+0000","last_undegraded":"2024-04-20T11:20:04.251545+0000","last_fullsized":"2024-04-20T11:20:04.251545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.693364+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:20:03.693364+0000","last_peered":"2024-04-20T11:20:03.693364+0000","last_clean":"2024-04-20T11:20:03.693364+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:20:03.693364+0000","last_undegraded":"2024-04-20T11:20:03.693364+0000","last_fullsized":"2024-04-20T11:20:03.693364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891395+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:20:03.891395+0000","last_peered":"2024-04-20T11:20:03.891395+0000","last_clean":"2024-04-20T11:20:03.891395+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:20:03.891395+0000","last_undegraded":"2024-04-20T11:20:03.891395+0000","last_fullsized":"2024-04-20T11:20:03.891395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:09.813492+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:20:09.813492+0000","last_peered":"2024-04-20T11:20:09.813492+0000","last_clean":"2024-04-20T11:20:09.813492+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:20:09.813492+0000","last_undegraded":"2024-04-20T11:20:09.813492+0000","last_fullsized":"2024-04-20T11:20:09.813492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699134+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:20:03.699134+0000","last_peered":"2024-04-20T11:20:03.699134+0000","last_clean":"2024-04-20T11:20:03.699134+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:20:03.699134+0000","last_undegraded":"2024-04-20T11:20:03.699134+0000","last_fullsized":"2024-04-20T11:20:03.699134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.825301+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:20:07.825301+0000","last_peered":"2024-04-20T11:20:07.825301+0000","last_clean":"2024-04-20T11:20:07.825301+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:20:07.825301+0000","last_undegraded":"2024-04-20T11:20:07.825301+0000","last_fullsized":"2024-04-20T11:20:07.825301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251261+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:20:04.251261+0000","last_peered":"2024-04-20T11:20:04.251261+0000","last_clean":"2024-04-20T11:20:04.251261+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:20:04.251261+0000","last_undegraded":"2024-04-20T11:20:04.251261+0000","last_fullsized":"2024-04-20T11:20:04.251261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891313+0000","last_change":"2024-04-20T11:20:02.691443+0000","last_active":"2024-04-20T11:20:03.891313+0000","last_peered":"2024-04-20T11:20:03.891313+0000","last_clean":"2024-04-20T11:20:03.891313+0000","last_became_active":"2024-04-20T11:20:02.690998+0000","last_became_peered":"2024-04-20T11:20:02.690998+0000","last_unstale":"2024-04-20T11:20:03.891313+0000","last_undegraded":"2024-04-20T11:20:03.891313+0000","last_fullsized":"2024-04-20T11:20:03.891313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:05:29.485535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.076369+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:20:07.076369+0000","last_peered":"2024-04-20T11:20:07.076369+0000","last_clean":"2024-04-20T11:20:07.076369+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:20:07.076369+0000","last_undegraded":"2024-04-20T11:20:07.076369+0000","last_fullsized":"2024-04-20T11:20:07.076369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.823850+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:20:07.823850+0000","last_peered":"2024-04-20T11:20:07.823850+0000","last_clean":"2024-04-20T11:20:07.823850+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:20:07.823850+0000","last_undegraded":"2024-04-20T11:20:07.823850+0000","last_fullsized":"2024-04-20T11:20:07.823850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891538+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:20:03.891538+0000","last_peered":"2024-04-20T11:20:03.891538+0000","last_clean":"2024-04-20T11:20:03.891538+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:20:03.891538+0000","last_undegraded":"2024-04-20T11:20:03.891538+0000","last_fullsized":"2024-04-20T11:20:03.891538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.826204+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:20:07.826204+0000","last_peered":"2024-04-20T11:20:07.826204+0000","last_clean":"2024-04-20T11:20:07.826204+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:20:07.826204+0000","last_undegraded":"2024-04-20T11:20:07.826204+0000","last_fullsized":"2024-04-20T11:20:07.826204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699701+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:20:03.699701+0000","last_peered":"2024-04-20T11:20:03.699701+0000","last_clean":"2024-04-20T11:20:03.699701+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:20:03.699701+0000","last_undegraded":"2024-04-20T11:20:03.699701+0000","last_fullsized":"2024-04-20T11:20:03.699701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.849634+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:20:08.849634+0000","last_peered":"2024-04-20T11:20:08.849634+0000","last_clean":"2024-04-20T11:20:08.849634+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:20:08.849634+0000","last_undegraded":"2024-04-20T11:20:08.849634+0000","last_fullsized":"2024-04-20T11:20:08.849634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892629+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:20:03.892629+0000","last_peered":"2024-04-20T11:20:03.892629+0000","last_clean":"2024-04-20T11:20:03.892629+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:20:03.892629+0000","last_undegraded":"2024-04-20T11:20:03.892629+0000","last_fullsized":"2024-04-20T11:20:03.892629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.070422+0000","last_change":"2024-04-20T11:20:01.675692+0000","last_active":"2024-04-20T11:20:06.070422+0000","last_peered":"2024-04-20T11:20:06.070422+0000","last_clean":"2024-04-20T11:20:06.070422+0000","last_became_active":"2024-04-20T11:20:01.675373+0000","last_became_peered":"2024-04-20T11:20:01.675373+0000","last_unstale":"2024-04-20T11:20:06.070422+0000","last_undegraded":"2024-04-20T11:20:06.070422+0000","last_fullsized":"2024-04-20T11:20:06.070422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:57.667811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699348+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:20:03.699348+0000","last_peered":"2024-04-20T11:20:03.699348+0000","last_clean":"2024-04-20T11:20:03.699348+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:20:03.699348+0000","last_undegraded":"2024-04-20T11:20:03.699348+0000","last_fullsized":"2024-04-20T11:20:03.699348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.846708+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:20:08.846708+0000","last_peered":"2024-04-20T11:20:08.846708+0000","last_clean":"2024-04-20T11:20:08.846708+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:20:08.846708+0000","last_undegraded":"2024-04-20T11:20:08.846708+0000","last_fullsized":"2024-04-20T11:20:08.846708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891221+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:20:03.891221+0000","last_peered":"2024-04-20T11:20:03.891221+0000","last_clean":"2024-04-20T11:20:03.891221+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:20:03.891221+0000","last_undegraded":"2024-04-20T11:20:03.891221+0000","last_fullsized":"2024-04-20T11:20:03.891221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.062133+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:20:07.062133+0000","last_peered":"2024-04-20T11:20:07.062133+0000","last_clean":"2024-04-20T11:20:07.062133+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:20:07.062133+0000","last_undegraded":"2024-04-20T11:20:07.062133+0000","last_fullsized":"2024-04-20T11:20:07.062133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699157+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:20:03.699157+0000","last_peered":"2024-04-20T11:20:03.699157+0000","last_clean":"2024-04-20T11:20:03.699157+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:20:03.699157+0000","last_undegraded":"2024-04-20T11:20:03.699157+0000","last_fullsized":"2024-04-20T11:20:03.699157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:06.067422+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:20:06.067422+0000","last_peered":"2024-04-20T11:20:06.067422+0000","last_clean":"2024-04-20T11:20:06.067422+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:20:06.067422+0000","last_undegraded":"2024-04-20T11:20:06.067422+0000","last_fullsized":"2024-04-20T11:20:06.067422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251683+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:20:04.251683+0000","last_peered":"2024-04-20T11:20:04.251683+0000","last_clean":"2024-04-20T11:20:04.251683+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:20:04.251683+0000","last_undegraded":"2024-04-20T11:20:04.251683+0000","last_fullsized":"2024-04-20T11:20:04.251683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.822542+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:20:07.822542+0000","last_peered":"2024-04-20T11:20:07.822542+0000","last_clean":"2024-04-20T11:20:07.822542+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:20:07.822542+0000","last_undegraded":"2024-04-20T11:20:07.822542+0000","last_fullsized":"2024-04-20T11:20:07.822542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251831+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:20:04.251831+0000","last_peered":"2024-04-20T11:20:04.251831+0000","last_clean":"2024-04-20T11:20:04.251831+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:20:04.251831+0000","last_undegraded":"2024-04-20T11:20:04.251831+0000","last_fullsized":"2024-04-20T11:20:04.251831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:07.823128+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:20:07.823128+0000","last_peered":"2024-04-20T11:20:07.823128+0000","last_clean":"2024-04-20T11:20:07.823128+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:20:07.823128+0000","last_undegraded":"2024-04-20T11:20:07.823128+0000","last_fullsized":"2024-04-20T11:20:07.823128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251520+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:20:04.251520+0000","last_peered":"2024-04-20T11:20:04.251520+0000","last_clean":"2024-04-20T11:20:04.251520+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:20:04.251520+0000","last_undegraded":"2024-04-20T11:20:04.251520+0000","last_fullsized":"2024-04-20T11:20:04.251520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:08.849042+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:20:08.849042+0000","last_peered":"2024-04-20T11:20:08.849042+0000","last_clean":"2024-04-20T11:20:08.849042+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:20:08.849042+0000","last_undegraded":"2024-04-20T11:20:08.849042+0000","last_fullsized":"2024-04-20T11:20:08.849042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:04.251768+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:20:04.251768+0000","last_peered":"2024-04-20T11:20:04.251768+0000","last_clean":"2024-04-20T11:20:04.251768+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:20:04.251768+0000","last_undegraded":"2024-04-20T11:20:04.251768+0000","last_fullsized":"2024-04-20T11:20:04.251768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891338+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:20:03.891338+0000","last_peered":"2024-04-20T11:20:03.891338+0000","last_clean":"2024-04-20T11:20:03.891338+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:20:03.891338+0000","last_undegraded":"2024-04-20T11:20:03.891338+0000","last_fullsized":"2024-04-20T11:20:03.891338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690545+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:20:03.690545+0000","last_peered":"2024-04-20T11:20:03.690545+0000","last_clean":"2024-04-20T11:20:03.690545+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:20:03.690545+0000","last_undegraded":"2024-04-20T11:20:03.690545+0000","last_fullsized":"2024-04-20T11:20:03.690545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690425+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:20:03.690425+0000","last_peered":"2024-04-20T11:20:03.690425+0000","last_clean":"2024-04-20T11:20:03.690425+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:20:03.690425+0000","last_undegraded":"2024-04-20T11:20:03.690425+0000","last_fullsized":"2024-04-20T11:20:03.690425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.892411+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:20:03.892411+0000","last_peered":"2024-04-20T11:20:03.892411+0000","last_clean":"2024-04-20T11:20:03.892411+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:20:03.892411+0000","last_undegraded":"2024-04-20T11:20:03.892411+0000","last_fullsized":"2024-04-20T11:20:03.892411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891700+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:20:03.891700+0000","last_peered":"2024-04-20T11:20:03.891700+0000","last_clean":"2024-04-20T11:20:03.891700+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:20:03.891700+0000","last_undegraded":"2024-04-20T11:20:03.891700+0000","last_fullsized":"2024-04-20T11:20:03.891700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891564+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:20:03.891564+0000","last_peered":"2024-04-20T11:20:03.891564+0000","last_clean":"2024-04-20T11:20:03.891564+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:20:03.891564+0000","last_undegraded":"2024-04-20T11:20:03.891564+0000","last_fullsized":"2024-04-20T11:20:03.891564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.690981+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:20:03.690981+0000","last_peered":"2024-04-20T11:20:03.690981+0000","last_clean":"2024-04-20T11:20:03.690981+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:20:03.690981+0000","last_undegraded":"2024-04-20T11:20:03.690981+0000","last_fullsized":"2024-04-20T11:20:03.690981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699545+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:20:03.699545+0000","last_peered":"2024-04-20T11:20:03.699545+0000","last_clean":"2024-04-20T11:20:03.699545+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:20:03.699545+0000","last_undegraded":"2024-04-20T11:20:03.699545+0000","last_fullsized":"2024-04-20T11:20:03.699545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891513+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:20:03.891513+0000","last_peered":"2024-04-20T11:20:03.891513+0000","last_clean":"2024-04-20T11:20:03.891513+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:20:03.891513+0000","last_undegraded":"2024-04-20T11:20:03.891513+0000","last_fullsized":"2024-04-20T11:20:03.891513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.699751+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:20:03.699751+0000","last_peered":"2024-04-20T11:20:03.699751+0000","last_clean":"2024-04-20T11:20:03.699751+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:20:03.699751+0000","last_undegraded":"2024-04-20T11:20:03.699751+0000","last_fullsized":"2024-04-20T11:20:03.699751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891198+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:20:03.891198+0000","last_peered":"2024-04-20T11:20:03.891198+0000","last_clean":"2024-04-20T11:20:03.891198+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:20:03.891198+0000","last_undegraded":"2024-04-20T11:20:03.891198+0000","last_fullsized":"2024-04-20T11:20:03.891198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891897+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:20:03.891897+0000","last_peered":"2024-04-20T11:20:03.891897+0000","last_clean":"2024-04-20T11:20:03.891897+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:20:03.891897+0000","last_undegraded":"2024-04-20T11:20:03.891897+0000","last_fullsized":"2024-04-20T11:20:03.891897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T11:20:03.891452+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:20:03.891452+0000","last_peered":"2024-04-20T11:20:03.891452+0000","last_clean":"2024-04-20T11:20:03.891452+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:20:03.891452+0000","last_undegraded":"2024-04-20T11:20:03.891452+0000","last_fullsized":"2024-04-20T11:20:03.891452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282095,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43656,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":45,"num_read_kb":71,"num_write":192,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":65017,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":168,"ondisk_log_size":168,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1069326,"data_compressed":37716,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":439429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":443703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":440587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":441483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":439185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":439578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1031127,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":9964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":40320,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:20:12.819 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T11:20:12.819 INFO:tasks.cephadm:Setting up client nodes... 2024-04-20T11:20:12.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-20T11:20:12.864 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:20:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:20:12.572+0000 7ff8f2550700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-20T11:20:12.864 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:20:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:20:12.572+0000 7ff8f2550700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-20T11:20:12.864 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:20:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:20:12.572+0000 7ff8f2550700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-20T11:20:13.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:12 smithi078 ceph-mon[21464]: pgmap v326: 97 pgs: 97 active+clean; 619 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-20T11:20:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:12 smithi078 ceph-mon[21464]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:20:13.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:12 smithi078 ceph-mon[21464]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:20:13.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:12 smithi044 ceph-mon[16855]: pgmap v326: 97 pgs: 97 active+clean; 619 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-20T11:20:13.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:12 smithi044 ceph-mon[16855]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:20:13.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:12 smithi044 ceph-mon[16855]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:20:13.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:12 smithi165 ceph-mon[21975]: pgmap v326: 97 pgs: 97 active+clean; 619 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-20T11:20:13.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:12 smithi165 ceph-mon[21975]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:20:13.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:12 smithi165 ceph-mon[21975]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:20:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:15.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:15 smithi044 ceph-mon[16855]: pgmap v327: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-20T11:20:15.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:15 smithi078 ceph-mon[21464]: pgmap v327: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-20T11:20:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:15 smithi165 ceph-mon[21975]: pgmap v327: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-20T11:20:15.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 -- 172.21.15.44:0/836715901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f8103d00 msgr2=0x7f04f8104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 --2- 172.21.15.44:0/836715901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f8103d00 0x7f04f8104160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f04e40099c0 tx=0x7f04e4009cd0 comp rx=0 tx=0).stop 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 -- 172.21.15.44:0/836715901 shutdown_connections 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 --2- 172.21.15.44:0/836715901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f81046a0 0x7f04f810af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 --2- 172.21.15.44:0/836715901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8102b00 0x7f04f8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 --2- 172.21.15.44:0/836715901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f8103d00 0x7f04f8104160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 -- 172.21.15.44:0/836715901 >> 172.21.15.44:0/836715901 conn(0x7f04f80fe0a0 msgr2=0x7f04f81004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 -- 172.21.15.44:0/836715901 shutdown_connections 2024-04-20T11:20:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.957+0000 7f04fd000700 1 -- 172.21.15.44:0/836715901 wait complete. 2024-04-20T11:20:15.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 Processor -- start 2024-04-20T11:20:15.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 -- start start 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 0x7f04f819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8103d00 0x7f04f819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f81046a0 0x7f04f81a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810e7b0 con 0x7f04f8102b00 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810e630 con 0x7f04f81046a0 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.959+0000 7f04fd000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810e930 con 0x7f04f8103d00 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 0x7f04f819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8103d00 0x7f04f819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 0x7f04f819aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50598/0 (socket says 172.21.15.44:50598) 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 -- 172.21.15.44:0/2588162772 learned_addr learned my addr 172.21.15.44:0/2588162772 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f6800700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f81046a0 0x7f04f81a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 -- 172.21.15.44:0/2588162772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8103d00 msgr2=0x7f04f819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8103d00 0x7f04f819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 -- 172.21.15.44:0/2588162772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f81046a0 msgr2=0x7f04f81a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f81046a0 0x7f04f81a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 -- 172.21.15.44:0/2588162772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81a2b40 con 0x7f04f8102b00 2024-04-20T11:20:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5400700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8103d00 0x7f04f819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.960+0000 7f04f5e00700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 0x7f04f819aee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04e800ea80 tx=0x7f04e800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.961+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e800cbd0 con 0x7f04f8102b00 2024-04-20T11:20:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.961+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04e800cd70 con 0x7f04f8102b00 2024-04-20T11:20:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.961+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f81a2e30 con 0x7f04f8102b00 2024-04-20T11:20:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.961+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8018840 con 0x7f04f8102b00 2024-04-20T11:20:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.961+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81a3290 con 0x7f04f8102b00 2024-04-20T11:20:15.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.962+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f8050fc0 con 0x7f04f8102b00 2024-04-20T11:20:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.966+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04e801f020 con 0x7f04f8102b00 2024-04-20T11:20:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.967+0000 7f04eea00700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04dc0a3e00 0x7f04dc0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.968+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f04e80d0570 con 0x7f04f8102b00 2024-04-20T11:20:15.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.968+0000 7f04f5400700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04dc0a3e00 0x7f04dc0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:15.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.968+0000 7f04f5400700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04dc0a3e00 0x7f04dc0a62c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04e4006010 tx=0x7f04e4003a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:15.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:15.971+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f04f8050fc0 con 0x7f04f8102b00 2024-04-20T11:20:16.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.296+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f04f81a37d0 con 0x7f04f8102b00 2024-04-20T11:20:16.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.300+0000 7f04eea00700 1 -- 172.21.15.44:0/2588162772 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f04e809cea0 con 0x7f04f8102b00 2024-04-20T11:20:16.300 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-04-20T11:20:16.300 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDwpCNmJU6+ERAAufFJOCCeERL2MlKJsuC0rg== 2024-04-20T11:20:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04dc0a3e00 msgr2=0x7f04dc0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04dc0a3e00 0x7f04dc0a62c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04e4006010 tx=0x7f04e4003a00 comp rx=0 tx=0).stop 2024-04-20T11:20:16.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 msgr2=0x7f04f819aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:16.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 0x7f04f819aee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04e800ea80 tx=0x7f04e800ed90 comp rx=0 tx=0).stop 2024-04-20T11:20:16.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 shutdown_connections 2024-04-20T11:20:16.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04dc0a3e00 0x7f04dc0a62c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f81046a0 0x7f04f81a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04f8102b00 0x7f04f819aee0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 --2- 172.21.15.44:0/2588162772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04f8103d00 0x7f04f819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.303+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 >> 172.21.15.44:0/2588162772 conn(0x7f04f80fe0a0 msgr2=0x7f04f8074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.304+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 shutdown_connections 2024-04-20T11:20:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:16.304+0000 7f04fd000700 1 -- 172.21.15.44:0/2588162772 wait complete. 2024-04-20T11:20:17.248 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:20:17.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-20T11:20:17.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T11:20:17.372 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:17 smithi044 ceph-mon[16855]: pgmap v328: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-20T11:20:17.372 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2588162772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:20:17.372 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2588162772' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:20:17.390 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T11:20:17.390 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-20T11:20:17.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph mgr dump --format=json 2024-04-20T11:20:17.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:17 smithi078 ceph-mon[21464]: pgmap v328: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-20T11:20:17.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2588162772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:20:17.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2588162772' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:17 smithi165 ceph-mon[21975]: pgmap v328: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-20T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2588162772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2588162772' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T11:20:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:19.377 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:19 smithi044 ceph-mon[16855]: pgmap v329: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-20T11:20:19.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:19 smithi078 ceph-mon[21464]: pgmap v329: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-20T11:20:19.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:19 smithi165 ceph-mon[21975]: pgmap v329: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-20T11:20:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.250+0000 7fadd0e00700 1 -- 172.21.15.44:0/695398654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc109c90 msgr2=0x7fadcc10dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.250+0000 7fadd0e00700 1 --2- 172.21.15.44:0/695398654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc109c90 0x7fadcc10dce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadb80099c0 tx=0x7fadb8009cd0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.250+0000 7fadd0e00700 1 -- 172.21.15.44:0/695398654 shutdown_connections 2024-04-20T11:20:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.250+0000 7fadd0e00700 1 --2- 172.21.15.44:0/695398654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadcc10e3c0 0x7fadcc115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.250+0000 7fadd0e00700 1 --2- 172.21.15.44:0/695398654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.250+0000 7fadd0e00700 1 --2- 172.21.15.44:0/695398654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc109c90 0x7fadcc10dce0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.251+0000 7fadd0e00700 1 -- 172.21.15.44:0/695398654 >> 172.21.15.44:0/695398654 conn(0x7fadcc076220 msgr2=0x7fadcc107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.251+0000 7fadd0e00700 1 -- 172.21.15.44:0/695398654 shutdown_connections 2024-04-20T11:20:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.251+0000 7fadd0e00700 1 -- 172.21.15.44:0/695398654 wait complete. 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.252+0000 7fadd0e00700 1 Processor -- start 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.252+0000 7fadd0e00700 1 -- start start 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadd0e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadd0e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadcc109c90 0x7fadcc1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadd0e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc10e3c0 0x7fadcc19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadd0e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc119470 con 0x7fadcc10e3c0 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadd0e00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc1192f0 con 0x7fadcc109c90 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadd0e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc1195f0 con 0x7fadcc1092e0 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadc9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadc9400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadcc109c90 0x7fadcc1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:20.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadc9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:56942/0 (socket says 172.21.15.44:56942) 2024-04-20T11:20:20.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadc9e00700 1 -- 172.21.15.44:0/1022267720 learned_addr learned my addr 172.21.15.44:0/1022267720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:20.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.253+0000 7fadca800700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc10e3c0 0x7fadcc19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:20.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.254+0000 7fadc9e00700 1 -- 172.21.15.44:0/1022267720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadcc109c90 msgr2=0x7fadcc1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.254+0000 7fadc9e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadcc109c90 0x7fadcc1a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.254+0000 7fadc9e00700 1 -- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc10e3c0 msgr2=0x7fadcc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.254+0000 7fadc9e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc10e3c0 0x7fadcc19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.254+0000 7fadc9e00700 1 -- 172.21.15.44:0/1022267720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc19fff0 con 0x7fadcc1092e0 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.254+0000 7fadc9e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1a55f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00eb30 tx=0x7fadbc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.255+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00cbd0 con 0x7fadcc1092e0 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.255+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadbc004520 con 0x7fadcc1092e0 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.255+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc1a02e0 con 0x7fadcc1092e0 2024-04-20T11:20:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.255+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc010430 con 0x7fadcc1092e0 2024-04-20T11:20:20.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.255+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc1ac570 con 0x7fadcc1092e0 2024-04-20T11:20:20.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.256+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc10a7c0 con 0x7fadcc1092e0 2024-04-20T11:20:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.258+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadbc004000 con 0x7fadcc1092e0 2024-04-20T11:20:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.259+0000 7fadc2a00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadb00a3d50 0x7fadb00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.260+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fadbc0d0730 con 0x7fadcc1092e0 2024-04-20T11:20:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.260+0000 7fadc9400700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadb00a3d50 0x7fadb00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.260+0000 7fadc9400700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadb00a3d50 0x7fadb00a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000c00 tx=0x7fadb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:20.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.264+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fadbc09d060 con 0x7fadcc1092e0 2024-04-20T11:20:20.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.560+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fadcc1a0bf0 con 0x7fadcc1092e0 2024-04-20T11:20:20.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.566+0000 7fadc2a00700 1 -- 172.21.15.44:0/1022267720 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7fadbc00cd70 con 0x7fadcc1092e0 2024-04-20T11:20:20.571 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadb00a3d50 msgr2=0x7fadb00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadb00a3d50 0x7fadb00a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000c00 tx=0x7fadb8005fb0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 msgr2=0x7fadcc1a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1a55f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00eb30 tx=0x7fadbc00ee40 comp rx=0 tx=0).stop 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 shutdown_connections 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fadb00a3d50 0x7fadb00a6210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc10e3c0 0x7fadcc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadcc1092e0 0x7fadcc1a55f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 --2- 172.21.15.44:0/1022267720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadcc109c90 0x7fadcc1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 >> 172.21.15.44:0/1022267720 conn(0x7fadcc076220 msgr2=0x7fadcc0767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 shutdown_connections 2024-04-20T11:20:20.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:20.571+0000 7fadd0e00700 1 -- 172.21.15.44:0/1022267720 wait complete. 2024-04-20T11:20:21.331 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:21 smithi044 ceph-mon[16855]: pgmap v330: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 KiB/s wr, 22 op/s 2024-04-20T11:20:21.331 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:21 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1022267720' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T11:20:21.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:21 smithi078 ceph-mon[21464]: pgmap v330: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 KiB/s wr, 22 op/s 2024-04-20T11:20:21.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:21 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1022267720' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T11:20:21.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:21 smithi165 ceph-mon[21975]: pgmap v330: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 KiB/s wr, 22 op/s 2024-04-20T11:20:21.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:21 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1022267720' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T11:20:21.565 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2425002048},{"type":"v1","addr":"172.21.15.44:6801","nonce":2425002048}]},"active_addr":"172.21.15.44:6801/2425002048","active_change":"2024-04-20T11:11:49.892771+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.44:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2316836529}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":464946469}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":748971044}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2017822852}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2528447246}]}]} 2024-04-20T11:20:21.572 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-20T11:20:21.572 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-20T11:20:21.572 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T11:20:22.045 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:23.421 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:23 smithi044 ceph-mon[16855]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 18 op/s 2024-04-20T11:20:23.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:23 smithi078 ceph-mon[21464]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 18 op/s 2024-04-20T11:20:23.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:23 smithi165 ceph-mon[21975]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 18 op/s 2024-04-20T11:20:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 -- 172.21.15.44:0/3979910426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0075690 msgr2=0x7f3fd010ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 --2- 172.21.15.44:0/3979910426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0075690 0x7f3fd010ed00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009990 tx=0x7f3fb8009ca0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 -- 172.21.15.44:0/3979910426 shutdown_connections 2024-04-20T11:20:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 --2- 172.21.15.44:0/3979910426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd010f240 0x7f3fd0111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 --2- 172.21.15.44:0/3979910426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd0074d70 0x7f3fd0075150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 --2- 172.21.15.44:0/3979910426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0075690 0x7f3fd010ed00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 -- 172.21.15.44:0/3979910426 >> 172.21.15.44:0/3979910426 conn(0x7f3fd00fe4d0 msgr2=0x7f3fd01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 -- 172.21.15.44:0/3979910426 shutdown_connections 2024-04-20T11:20:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.272+0000 7f3fcf200700 1 -- 172.21.15.44:0/3979910426 wait complete. 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.273+0000 7f3fcf200700 1 Processor -- start 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 -- start start 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0074d70 0x7f3fd019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 0x7f3fd019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd010f240 0x7f3fd019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0114fd0 con 0x7f3fd010f240 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0114e50 con 0x7f3fd0074d70 2024-04-20T11:20:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.274+0000 7f3fcf200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0115150 con 0x7f3fd0075690 2024-04-20T11:20:24.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 0x7f3fd019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:24.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fcde00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0074d70 0x7f3fd019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 0x7f3fd019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:56966/0 (socket says 172.21.15.44:56966) 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 -- 172.21.15.44:0/890567250 learned_addr learned my addr 172.21.15.44:0/890567250 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fce800700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd010f240 0x7f3fd019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 -- 172.21.15.44:0/890567250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0074d70 msgr2=0x7f3fd019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0074d70 0x7f3fd019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 -- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd010f240 msgr2=0x7f3fd019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd010f240 0x7f3fd019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 -- 172.21.15.44:0/890567250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd01a03d0 con 0x7f3fd0075690 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fce800700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd010f240 0x7f3fd019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fcde00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0074d70 0x7f3fd019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.275+0000 7f3fc7e00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 0x7f3fd019b900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000c00 tx=0x7f3fb800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.276+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8027070 con 0x7f3fd0075690 2024-04-20T11:20:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.276+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd01a0660 con 0x7f3fd0075690 2024-04-20T11:20:24.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.276+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3fb8022470 con 0x7f3fd0075690 2024-04-20T11:20:24.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.276+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8017630 con 0x7f3fd0075690 2024-04-20T11:20:24.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.276+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd01a7f40 con 0x7f3fd0075690 2024-04-20T11:20:24.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.277+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd0069740 con 0x7f3fd0075690 2024-04-20T11:20:24.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.279+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3fb802a070 con 0x7f3fd0075690 2024-04-20T11:20:24.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.281+0000 7f3fc6a00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3fb40a3d50 0x7f3fb40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:24.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.281+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3fb80d13b0 con 0x7f3fd0075690 2024-04-20T11:20:24.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.281+0000 7f3fcde00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3fb40a3d50 0x7f3fb40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:24.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.282+0000 7f3fcde00700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3fb40a3d50 0x7f3fb40a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0011490 tx=0x7f3fc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:24.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.285+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3fb805eff0 con 0x7f3fd0075690 2024-04-20T11:20:24.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.518+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fd01a8110 con 0x7f3fd0075690 2024-04-20T11:20:24.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.520+0000 7f3fc6a00700 1 -- 172.21.15.44:0/890567250 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22171 (secure 0 0 0) 0x7f3fb8013070 con 0x7f3fd0075690 2024-04-20T11:20:24.521 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:24.522 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":85,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:03.685700+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:20:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3fb40a3d50 msgr2=0x7f3fb40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3fb40a3d50 0x7f3fb40a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0011490 tx=0x7f3fc0009260 comp rx=0 tx=0).stop 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 msgr2=0x7f3fd019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 0x7f3fd019b900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000c00 tx=0x7f3fb800b610 comp rx=0 tx=0).stop 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 shutdown_connections 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3fb40a3d50 0x7f3fb40a6210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd010f240 0x7f3fd019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fd0074d70 0x7f3fd019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 --2- 172.21.15.44:0/890567250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fd0075690 0x7f3fd019b900 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 >> 172.21.15.44:0/890567250 conn(0x7f3fd00fe4d0 msgr2=0x7f3fd0111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 shutdown_connections 2024-04-20T11:20:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:24.523+0000 7f3fcf200700 1 -- 172.21.15.44:0/890567250 wait complete. 2024-04-20T11:20:25.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:25 smithi044 ceph-mon[16855]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-20T11:20:25.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:25 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/890567250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:25.378 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-20T11:20:25.378 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T11:20:25.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:25 smithi078 ceph-mon[21464]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-20T11:20:25.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:25 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/890567250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:25 smithi165 ceph-mon[21975]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-20T11:20:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:25 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/890567250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:25.863 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:27.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:27 smithi044 ceph-mon[16855]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-20T11:20:27.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:27 smithi078 ceph-mon[21464]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-20T11:20:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:27 smithi165 ceph-mon[21975]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-20T11:20:28.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 -- 172.21.15.44:0/711521475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24104dc0 msgr2=0x7fad241051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:28.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 --2- 172.21.15.44:0/711521475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24104dc0 0x7fad241051a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fad180099c0 tx=0x7fad18009cd0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 -- 172.21.15.44:0/711521475 shutdown_connections 2024-04-20T11:20:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 --2- 172.21.15.44:0/711521475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24109ea0 0x7fad24111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 --2- 172.21.15.44:0/711521475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24104dc0 0x7fad241051a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 --2- 172.21.15.44:0/711521475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24105770 0x7fad241097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 -- 172.21.15.44:0/711521475 >> 172.21.15.44:0/711521475 conn(0x7fad24100630 msgr2=0x7fad24102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.064+0000 7fad2a600700 1 -- 172.21.15.44:0/711521475 shutdown_connections 2024-04-20T11:20:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.065+0000 7fad2a600700 1 -- 172.21.15.44:0/711521475 wait complete. 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 Processor -- start 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 -- start start 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24104dc0 0x7fad2419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24105770 0x7fad2419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24109ea0 0x7fad2419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad24115050 con 0x7fad24109ea0 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad24114ed0 con 0x7fad24105770 2024-04-20T11:20:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.066+0000 7fad2a600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad241151d0 con 0x7fad24104dc0 2024-04-20T11:20:28.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad29200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24104dc0 0x7fad2419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:28.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad29c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24109ea0 0x7fad2419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:28.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad29200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24104dc0 0x7fad2419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:56980/0 (socket says 172.21.15.44:56980) 2024-04-20T11:20:28.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24105770 0x7fad2419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:28.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad29200700 1 -- 172.21.15.44:0/1310942049 learned_addr learned my addr 172.21.15.44:0/1310942049 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad29c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24109ea0 0x7fad2419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36884/0 (socket says 172.21.15.44:36884) 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 -- 172.21.15.44:0/1310942049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24104dc0 msgr2=0x7fad2419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24104dc0 0x7fad2419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 -- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24109ea0 msgr2=0x7fad2419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24109ea0 0x7fad2419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 -- 172.21.15.44:0/1310942049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad241a0450 con 0x7fad24105770 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.067+0000 7fad23e00700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24105770 0x7fad2419b980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d720 tx=0x7fad1000da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.068+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10016070 con 0x7fad24105770 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.068+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad10005020 con 0x7fad24105770 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.068+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad241a0740 con 0x7fad24105770 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.068+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1000fc30 con 0x7fad24105770 2024-04-20T11:20:28.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.068+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad241a7fc0 con 0x7fad24105770 2024-04-20T11:20:28.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.069+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad04005330 con 0x7fad24105770 2024-04-20T11:20:28.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.071+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad10004a30 con 0x7fad24105770 2024-04-20T11:20:28.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.073+0000 7fad21600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad0c0a3d50 0x7fad0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:28.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.073+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fad100d0150 con 0x7fad24105770 2024-04-20T11:20:28.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.073+0000 7fad29200700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad0c0a3d50 0x7fad0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:28.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.074+0000 7fad29200700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad0c0a3d50 0x7fad0c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad18009f50 tx=0x7fad18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:28.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.075+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fad1009ca80 con 0x7fad24105770 2024-04-20T11:20:28.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.307+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad040058f0 con 0x7fad24105770 2024-04-20T11:20:28.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.309+0000 7fad21600700 1 -- 172.21.15.44:0/1310942049 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22171 (secure 0 0 0) 0x7fad1009c380 con 0x7fad24105770 2024-04-20T11:20:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:20:28.311 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":85,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:03.685700+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad0c0a3d50 msgr2=0x7fad0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad0c0a3d50 0x7fad0c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad18009f50 tx=0x7fad18005fb0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24105770 msgr2=0x7fad2419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24105770 0x7fad2419b980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d720 tx=0x7fad1000da30 comp rx=0 tx=0).stop 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 shutdown_connections 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fad0c0a3d50 0x7fad0c0a6210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad24109ea0 0x7fad2419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad24104dc0 0x7fad2419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 --2- 172.21.15.44:0/1310942049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad24105770 0x7fad2419b980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 >> 172.21.15.44:0/1310942049 conn(0x7fad24100630 msgr2=0x7fad241011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 shutdown_connections 2024-04-20T11:20:28.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:28.314+0000 7fad2a600700 1 -- 172.21.15.44:0/1310942049 wait complete. 2024-04-20T11:20:29.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-20T11:20:29.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-20T11:20:29.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-20T11:20:29.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-20T11:20:29.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-20T11:20:29.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-20T11:20:29.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-20T11:20:29.163 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-20T11:20:29.163 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-20T11:20:29.163 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-20T11:20:29.164 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-20T11:20:29.164 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-20T11:20:29.351 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:29 smithi044 ceph-mon[16855]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-20T11:20:29.351 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:29 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1310942049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:29.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:29 smithi078 ceph-mon[21464]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-20T11:20:29.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:29 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1310942049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:29 smithi165 ceph-mon[21975]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-20T11:20:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:29 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1310942049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:20:30.208 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.256 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.257 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.277 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.285 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.292 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.298 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:30.348 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:31.433 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:31 smithi044 ceph-mon[16855]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-20T11:20:31.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:31 smithi078 ceph-mon[21464]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-20T11:20:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:31 smithi165 ceph-mon[21975]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-20T11:20:33.150 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:33 smithi044 ceph-mon[16855]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-20T11:20:33.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:33 smithi078 ceph-mon[21464]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-20T11:20:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:33 smithi165 ceph-mon[21975]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-20T11:20:35.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:35 smithi078 ceph-mon[21464]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:35.439 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:35 smithi044 ceph-mon[16855]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:35.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:35 smithi165 ceph-mon[21975]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:37.160 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:37 smithi044 ceph-mon[16855]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-20T11:20:37.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:37 smithi078 ceph-mon[21464]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-20T11:20:37.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:37 smithi165 ceph-mon[21975]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-20T11:20:39.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:39 smithi078 ceph-mon[21464]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:39.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:39 smithi044 ceph-mon[16855]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:39.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:39 smithi165 ceph-mon[21975]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:41.222 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:41 smithi044 ceph-mon[16855]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:41.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:41 smithi078 ceph-mon[21464]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:41.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:41 smithi165 ceph-mon[21975]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 -- 172.21.15.44:0/2582933438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 msgr2=0x7f7f6c10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 --2- 172.21.15.44:0/2582933438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 0x7f7f6c10ef10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009990 tx=0x7f7f54009ca0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 -- 172.21.15.44:0/2582933438 shutdown_connections 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 --2- 172.21.15.44:0/2582933438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c10f450 0x7f7f6c111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 --2- 172.21.15.44:0/2582933438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c074c80 0x7f7f6c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 --2- 172.21.15.44:0/2582933438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 0x7f7f6c10ef10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 -- 172.21.15.44:0/2582933438 >> 172.21.15.44:0/2582933438 conn(0x7f7f6c079bb0 msgr2=0x7f7f6c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:42.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 -- 172.21.15.44:0/2582933438 shutdown_connections 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.709+0000 7f7f71000700 1 -- 172.21.15.44:0/2582933438 wait complete. 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 Processor -- start 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 -- start start 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c074c80 0x7f7f6c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 0x7f7f6c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c10f450 0x7f7f6c19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c115100 con 0x7f7f6c0755a0 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c114f80 con 0x7f7f6c074c80 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.711+0000 7f7f71000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c115280 con 0x7f7f6c10f450 2024-04-20T11:20:42.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c074c80 0x7f7f6c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f63e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 0x7f7f6c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c074c80 0x7f7f6c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37086/0 (socket says 172.21.15.44:37086) 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 -- 172.21.15.44:0/2254515028 learned_addr learned my addr 172.21.15.44:0/2254515028 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f6a800700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c10f450 0x7f7f6c19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 -- 172.21.15.44:0/2254515028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c10f450 msgr2=0x7f7f6c19fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c10f450 0x7f7f6c19fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 -- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 msgr2=0x7f7f6c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c0755a0 0x7f7f6c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 -- 172.21.15.44:0/2254515028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1a05e0 con 0x7f7f6c074c80 2024-04-20T11:20:42.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f6a800700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c10f450 0x7f7f6c19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.712+0000 7f7f69e00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c074c80 0x7f7f6c19b5d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ef30 tx=0x7f7f5c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.713+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0092e0 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.713+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7f5c004bb0 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.713+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1a0870 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.713+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c004260 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.713+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1a8150 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.714+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f6c101770 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.716+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f5c01b030 con 0x7f7f6c074c80 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.717+0000 7f7f62a00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7f500a3cd0 0x7f7f500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.717+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7f5c0d00f0 con 0x7f7f6c074c80 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.717+0000 7f7f63e00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7f500a3cd0 0x7f7f500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.718+0000 7f7f62a00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] conn(0x7f7f500ad4b0 0x7f7f500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.718+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 --> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f500aff80 con 0x7f7f500ad4b0 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.718+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c0d7050 con 0x7f7f6c074c80 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.718+0000 7f7f6a800700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] conn(0x7f7f500ad4b0 0x7f7f500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.718+0000 7f7f63e00700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7f500a3cd0 0x7f7f500a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c101c50 tx=0x7f7f5400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.719+0000 7f7f6a800700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] conn(0x7f7f500ad4b0 0x7f7f500af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.720+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== osd.3 v2:172.21.15.44:6810/3780220117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f500aff80 con 0x7f7f500ad4b0 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.739+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 --> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7f6c076370 con 0x7f7f500ad4b0 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.740+0000 7f7f62a00700 1 -- 172.21.15.44:0/2254515028 <== osd.3 v2:172.21.15.44:6810/3780220117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7f6c076370 con 0x7f7f500ad4b0 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.740+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] conn(0x7f7f500ad4b0 msgr2=0x7f7f500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.740+0000 7f7f71000700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] conn(0x7f7f500ad4b0 0x7f7f500af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7f500a3cd0 msgr2=0x7f7f500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7f500a3cd0 0x7f7f500a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c101c50 tx=0x7f7f5400b260 comp rx=0 tx=0).stop 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c074c80 msgr2=0x7f7f6c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:42.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 --2- 172.21.15.44:0/2254515028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f6c074c80 0x7f7f6c19b5d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ef30 tx=0x7f7f5c00c3a0 comp rx=0 tx=0).stop 2024-04-20T11:20:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f6a800700 1 -- 172.21.15.44:0/2254515028 reap_dead start 2024-04-20T11:20:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 shutdown_connections 2024-04-20T11:20:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 >> 172.21.15.44:0/2254515028 conn(0x7f7f6c079bb0 msgr2=0x7f7f6c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 shutdown_connections 2024-04-20T11:20:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.741+0000 7f7f71000700 1 -- 172.21.15.44:0/2254515028 wait complete. 2024-04-20T11:20:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:20:43.426 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:43 smithi044 ceph-mon[16855]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:43.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:43 smithi078 ceph-mon[21464]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:43 smithi165 ceph-mon[21975]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.910+0000 7f7975400700 1 -- 172.21.15.44:0/4190399719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79701092e0 msgr2=0x7f79701096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.910+0000 7f7975400700 1 --2- 172.21.15.44:0/4190399719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79701092e0 0x7f79701096c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7960009920 tx=0x7f7960009c30 comp rx=0 tx=0).stop 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 -- 172.21.15.44:0/4190399719 shutdown_connections 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 --2- 172.21.15.44:0/4190399719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 0x7f7970115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 --2- 172.21.15.44:0/4190399719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79701092e0 0x7f79701096c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 --2- 172.21.15.44:0/4190399719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7970109c90 0x7f797010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 -- 172.21.15.44:0/4190399719 >> 172.21.15.44:0/4190399719 conn(0x7f7970076220 msgr2=0x7f7970107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 -- 172.21.15.44:0/4190399719 shutdown_connections 2024-04-20T11:20:44.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.911+0000 7f7975400700 1 -- 172.21.15.44:0/4190399719 wait complete. 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 Processor -- start 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 -- start start 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79701092e0 0x7f79701a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7970109c90 0x7f79701a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 0x7f797019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7970119530 con 0x7f797010e3c0 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79701193b0 con 0x7f79701092e0 2024-04-20T11:20:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.912+0000 7f7975400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f79701196b0 con 0x7f7970109c90 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 0x7f797019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f7965e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7970109c90 0x7f79701a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 0x7f797019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39462/0 (socket says 172.21.15.44:39462) 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 -- 172.21.15.44:0/3294574361 learned_addr learned my addr 172.21.15.44:0/3294574361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796e800700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79701092e0 0x7f79701a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 -- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7970109c90 msgr2=0x7f79701a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7970109c90 0x7f79701a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 -- 172.21.15.44:0/3294574361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79701092e0 msgr2=0x7f79701a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79701092e0 0x7f79701a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.913+0000 7f796f200700 1 -- 172.21.15.44:0/3294574361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797019ffb0 con 0x7f797010e3c0 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.914+0000 7f796f200700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 0x7f797019f7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f796800eb80 tx=0x7f796800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.914+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796800cc90 con 0x7f797010e3c0 2024-04-20T11:20:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.914+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7968004530 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.914+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79701a02a0 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.914+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968010690 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.914+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79701ac4b0 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.915+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79700631e0 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.917+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7968003e40 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.918+0000 7f7967e00700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.919+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f79680d0a40 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.919+0000 7f796e800700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.919+0000 7f7967e00700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] conn(0x7f79540ad4b0 0x7f79540af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.919+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 --> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79540aff80 con 0x7f79540ad4b0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.919+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f79680d0df0 con 0x7f797010e3c0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.919+0000 7f7965e00700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] conn(0x7f79540ad4b0 0x7f79540af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.920+0000 7f796e800700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f796000c010 tx=0x7f7960019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.920+0000 7f7965e00700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] conn(0x7f79540ad4b0 0x7f79540af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.921+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== osd.5 v2:172.21.15.165:6808/2633955642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79540aff80 con 0x7f79540ad4b0 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.940+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 --> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7970063660 con 0x7f79540ad4b0 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7967e00700 1 -- 172.21.15.44:0/3294574361 <== osd.5 v2:172.21.15.165:6808/2633955642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7970063660 con 0x7f79540ad4b0 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] conn(0x7f79540ad4b0 msgr2=0x7f79540af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7975400700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] conn(0x7f79540ad4b0 0x7f79540af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:44.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 msgr2=0x7f79540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7975400700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f796000c010 tx=0x7f7960019040 comp rx=0 tx=0).stop 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 msgr2=0x7f797019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f7975400700 1 --2- 172.21.15.44:0/3294574361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f797010e3c0 0x7f797019f7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f796800eb80 tx=0x7f796800ee90 comp rx=0 tx=0).stop 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.941+0000 7f796f200700 1 -- 172.21.15.44:0/3294574361 reap_dead start 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.942+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 shutdown_connections 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.942+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 >> 172.21.15.44:0/3294574361 conn(0x7f7970076220 msgr2=0x7f79700767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.942+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 shutdown_connections 2024-04-20T11:20:44.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:43.942+0000 7f7975400700 1 -- 172.21.15.44:0/3294574361 wait complete. 2024-04-20T11:20:45.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 -- 172.21.15.44:0/4024998788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 msgr2=0x7fca9010f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 --2- 172.21.15.44:0/4024998788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 0x7fca9010f8c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b0a0 tx=0x7fca8800b3b0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 -- 172.21.15.44:0/4024998788 shutdown_connections 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 --2- 172.21.15.44:0/4024998788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 0x7fca9010f8c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 --2- 172.21.15.44:0/4024998788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca90108a60 0x7fca90108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 --2- 172.21.15.44:0/4024998788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca90102aa0 0x7fca90102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 -- 172.21.15.44:0/4024998788 >> 172.21.15.44:0/4024998788 conn(0x7fca90079bb0 msgr2=0x7fca90079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 -- 172.21.15.44:0/4024998788 shutdown_connections 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.213+0000 7fca97a00700 1 -- 172.21.15.44:0/4024998788 wait complete. 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.214+0000 7fca97a00700 1 Processor -- start 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.214+0000 7fca97a00700 1 -- start start 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.214+0000 7fca97a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca90102aa0 0x7fca90074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.214+0000 7fca97a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 0x7fca90075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca97a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca90108a60 0x7fca900757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca97a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca90112ea0 con 0x7fca90103460 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca97a00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca90112d20 con 0x7fca90102aa0 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca97a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca90113020 con 0x7fca90108a60 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca8fe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 0x7fca90075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca90108a60 0x7fca900757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca90102aa0 0x7fca90074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca8fe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 0x7fca90075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39480/0 (socket says 172.21.15.44:39480) 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca8fe00700 1 -- 172.21.15.44:0/4287484741 learned_addr learned my addr 172.21.15.44:0/4287484741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95a00700 1 -- 172.21.15.44:0/4287484741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca90102aa0 msgr2=0x7fca90074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95a00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca90102aa0 0x7fca90074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95a00700 1 -- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 msgr2=0x7fca90075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95a00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca90103460 0x7fca90075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.215+0000 7fca95a00700 1 -- 172.21.15.44:0/4287484741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca901a77b0 con 0x7fca90108a60 2024-04-20T11:20:45.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.216+0000 7fca95a00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca90108a60 0x7fca900757d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b0a0 tx=0x7fca88004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.216+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88021070 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.216+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca88007b80 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.216+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901a7a40 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.216+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88005040 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.216+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901a7fd0 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.217+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca90063120 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.219+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fca88024070 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.220+0000 7fca8d600700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fca780a3d50 0x7fca780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.221+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fca8800f020 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.221+0000 7fca95000700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fca780a3d50 0x7fca780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.221+0000 7fca8d600700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] conn(0x7fca780ad530 0x7fca780af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.221+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 --> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca8800c910 con 0x7fca780ad530 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.221+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fca880d1db0 con 0x7fca90108a60 2024-04-20T11:20:45.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.221+0000 7fca8fe00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] conn(0x7fca780ad530 0x7fca780af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.222+0000 7fca95000700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fca780a3d50 0x7fca780a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca84009680 tx=0x7fca84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.222+0000 7fca8fe00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] conn(0x7fca780ad530 0x7fca780af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.223+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== osd.9 v2:172.21.15.44:6826/3770179466 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca8800c910 con 0x7fca780ad530 2024-04-20T11:20:45.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.242+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 --> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca90063520 con 0x7fca780ad530 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.243+0000 7fca8d600700 1 -- 172.21.15.44:0/4287484741 <== osd.9 v2:172.21.15.44:6826/3770179466 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fca90063520 con 0x7fca780ad530 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.243+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] conn(0x7fca780ad530 msgr2=0x7fca780af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.243+0000 7fca97a00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] conn(0x7fca780ad530 0x7fca780af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fca780a3d50 msgr2=0x7fca780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fca780a3d50 0x7fca780a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca84009680 tx=0x7fca84009260 comp rx=0 tx=0).stop 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca90108a60 msgr2=0x7fca900757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 --2- 172.21.15.44:0/4287484741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca90108a60 0x7fca900757d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b0a0 tx=0x7fca88004d00 comp rx=0 tx=0).stop 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca95a00700 1 -- 172.21.15.44:0/4287484741 reap_dead start 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 shutdown_connections 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 >> 172.21.15.44:0/4287484741 conn(0x7fca90079bb0 msgr2=0x7fca901016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 shutdown_connections 2024-04-20T11:20:45.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.244+0000 7fca97a00700 1 -- 172.21.15.44:0/4287484741 wait complete. 2024-04-20T11:20:45.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:45 smithi044 ceph-mon[16855]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-20T11:20:45.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:45 smithi078 ceph-mon[21464]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-20T11:20:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:45 smithi165 ceph-mon[21975]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 -- 172.21.15.44:0/2689735996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c107980 msgr2=0x7fa25c06a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 --2- 172.21.15.44:0/2689735996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c107980 0x7fa25c06a760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009990 tx=0x7fa24c009ca0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 -- 172.21.15.44:0/2689735996 shutdown_connections 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 --2- 172.21.15.44:0/2689735996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa25c105490 0x7fa25c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 --2- 172.21.15.44:0/2689735996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c107980 0x7fa25c06a760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 --2- 172.21.15.44:0/2689735996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa25c06aca0 0x7fa25c06b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 -- 172.21.15.44:0/2689735996 >> 172.21.15.44:0/2689735996 conn(0x7fa25c078900 msgr2=0x7fa25c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 -- 172.21.15.44:0/2689735996 shutdown_connections 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.971+0000 7fa262800700 1 -- 172.21.15.44:0/2689735996 wait complete. 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 Processor -- start 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 -- start start 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c06aca0 0x7fa25c1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa25c105490 0x7fa25c1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa25c107980 0x7fa25c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c1194a0 con 0x7fa25c06aca0 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c119320 con 0x7fa25c107980 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.973+0000 7fa262800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c119620 con 0x7fa25c105490 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa25b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c06aca0 0x7fa25c1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa25b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c06aca0 0x7fa25c1a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39498/0 (socket says 172.21.15.44:39498) 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa25b400700 1 -- 172.21.15.44:0/2788902854 learned_addr learned my addr 172.21.15.44:0/2788902854 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:45.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa252a00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa25c105490 0x7fa25c1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa25be00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa25c107980 0x7fa25c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa252a00700 1 -- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa25c107980 msgr2=0x7fa25c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa252a00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa25c107980 0x7fa25c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa252a00700 1 -- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c06aca0 msgr2=0x7fa25c1a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa252a00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c06aca0 0x7fa25c1a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa252a00700 1 -- 172.21.15.44:0/2788902854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c1a0000 con 0x7fa25c105490 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.974+0000 7fa25b400700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa25c06aca0 0x7fa25c1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:45.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.975+0000 7fa252a00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa25c105490 0x7fa25c1a5b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e9f0 tx=0x7fa24400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:45.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.975+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400cb70 con 0x7fa25c105490 2024-04-20T11:20:45.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.975+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c1a02f0 con 0x7fa25c105490 2024-04-20T11:20:45.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.975+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa24400ee90 con 0x7fa25c105490 2024-04-20T11:20:45.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.975+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244010430 con 0x7fa25c105490 2024-04-20T11:20:45.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.975+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1ac580 con 0x7fa25c105490 2024-04-20T11:20:45.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.976+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa25c101360 con 0x7fa25c105490 2024-04-20T11:20:45.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.978+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa244003e90 con 0x7fa25c105490 2024-04-20T11:20:45.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.980+0000 7fa258c00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa2400a3cd0 0x7fa2400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.980+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa2440d0930 con 0x7fa25c105490 2024-04-20T11:20:45.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.980+0000 7fa25b400700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa2400a3cd0 0x7fa2400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.980+0000 7fa258c00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] conn(0x7fa2400ad4b0 0x7fa2400af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:45.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.980+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 --> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2400aff80 con 0x7fa2400ad4b0 2024-04-20T11:20:45.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.980+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa2440d0ce0 con 0x7fa25c105490 2024-04-20T11:20:45.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.981+0000 7fa25be00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] conn(0x7fa2400ad4b0 0x7fa2400af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:45.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.981+0000 7fa25b400700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa2400a3cd0 0x7fa2400a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009750 tx=0x7fa24c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.981+0000 7fa25be00700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] conn(0x7fa2400ad4b0 0x7fa2400af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:45.982+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== osd.1 v2:172.21.15.78:6800/2898064203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2400aff80 con 0x7fa2400ad4b0 2024-04-20T11:20:46.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.001+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 --> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa25c101f70 con 0x7fa2400ad4b0 2024-04-20T11:20:46.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.002+0000 7fa258c00700 1 -- 172.21.15.44:0/2788902854 <== osd.1 v2:172.21.15.78:6800/2898064203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa25c101f70 con 0x7fa2400ad4b0 2024-04-20T11:20:46.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.002+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] conn(0x7fa2400ad4b0 msgr2=0x7fa2400af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:46.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.002+0000 7fa262800700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] conn(0x7fa2400ad4b0 0x7fa2400af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa2400a3cd0 msgr2=0x7fa2400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa2400a3cd0 0x7fa2400a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009750 tx=0x7fa24c00b280 comp rx=0 tx=0).stop 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa25c105490 msgr2=0x7fa25c1a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 --2- 172.21.15.44:0/2788902854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa25c105490 0x7fa25c1a5b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e9f0 tx=0x7fa24400ed00 comp rx=0 tx=0).stop 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa25be00700 1 -- 172.21.15.44:0/2788902854 reap_dead start 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 shutdown_connections 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 >> 172.21.15.44:0/2788902854 conn(0x7fa25c078900 msgr2=0x7fa25c100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 shutdown_connections 2024-04-20T11:20:46.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:46.003+0000 7fa262800700 1 -- 172.21.15.44:0/2788902854 wait complete. 2024-04-20T11:20:47.227 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:47 smithi044 ceph-mon[16855]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-20T11:20:47.290 INFO:teuthology.orchestra.run.smithi044.stdout:163208757306 2024-04-20T11:20:47.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-20T11:20:47.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:47 smithi078 ceph-mon[21464]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-20T11:20:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.972+0000 7fe0f5400700 1 -- 172.21.15.44:0/39986446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f01092e0 msgr2=0x7fe0f01096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.972+0000 7fe0f5400700 1 --2- 172.21.15.44:0/39986446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f01092e0 0x7fe0f01096c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00099c0 tx=0x7fe0e0009cd0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 -- 172.21.15.44:0/39986446 shutdown_connections 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 --2- 172.21.15.44:0/39986446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0f010e3c0 0x7fe0f0115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 --2- 172.21.15.44:0/39986446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f01092e0 0x7fe0f01096c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 --2- 172.21.15.44:0/39986446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0f0109c90 0x7fe0f010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 -- 172.21.15.44:0/39986446 >> 172.21.15.44:0/39986446 conn(0x7fe0f0076220 msgr2=0x7fe0f0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 -- 172.21.15.44:0/39986446 shutdown_connections 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.973+0000 7fe0f5400700 1 -- 172.21.15.44:0/39986446 wait complete. 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 Processor -- start 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 -- start start 2024-04-20T11:20:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0f01092e0 0x7fe0f01a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f0109c90 0x7fe0f01a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0f010e3c0 0x7fe0f019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0119510 con 0x7fe0f0109c90 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0119390 con 0x7fe0f010e3c0 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.974+0000 7fe0f5400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0119690 con 0x7fe0f01092e0 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f0109c90 0x7fe0f01a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0ee800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0f01092e0 0x7fe0f01a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f0109c90 0x7fe0f01a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39454/0 (socket says 172.21.15.44:39454) 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0ee800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0f01092e0 0x7fe0f01a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41592/0 (socket says 172.21.15.44:41592) 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 -- 172.21.15.44:0/3940181066 learned_addr learned my addr 172.21.15.44:0/3940181066 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0ef200700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0f010e3c0 0x7fe0f019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 -- 172.21.15.44:0/3940181066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0f01092e0 msgr2=0x7fe0f01a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0f01092e0 0x7fe0f01a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 -- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0f010e3c0 msgr2=0x7fe0f019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0f010e3c0 0x7fe0f019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.975+0000 7fe0e7e00700 1 -- 172.21.15.44:0/3940181066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f019ffb0 con 0x7fe0f0109c90 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.976+0000 7fe0e7e00700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f0109c90 0x7fe0f01a5b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800ea30 tx=0x7fe0d800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.976+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800cbd0 con 0x7fe0f0109c90 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.976+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0d8004520 con 0x7fe0f0109c90 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.976+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8010430 con 0x7fe0f0109c90 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.976+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a02a0 con 0x7fe0f0109c90 2024-04-20T11:20:47.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.976+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01ac4b0 con 0x7fe0f0109c90 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.977+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0f0063120 con 0x7fe0f0109c90 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.979+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0d8003ef0 con 0x7fe0f0109c90 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.980+0000 7fe0e7400700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0d40a3d50 0x7fe0d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.981+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe0d80d0820 con 0x7fe0f0109c90 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.981+0000 7fe0ee800700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0d40a3d50 0x7fe0d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.981+0000 7fe0e7400700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] conn(0x7fe0d40ad530 0x7fe0d40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:47.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.981+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 --> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0d8011e50 con 0x7fe0d40ad530 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.981+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fe0d80d0bd0 con 0x7fe0f0109c90 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.981+0000 7fe0ef200700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] conn(0x7fe0d40ad530 0x7fe0d40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.982+0000 7fe0ee800700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0d40a3d50 0x7fe0d40a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0000c00 tx=0x7fe0e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.982+0000 7fe0ef200700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] conn(0x7fe0d40ad530 0x7fe0d40af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:41.983+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== osd.4 v2:172.21.15.78:6808/856268583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0d8011e50 con 0x7fe0d40ad530 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.001+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 --> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0f00635f0 con 0x7fe0d40ad530 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.002+0000 7fe0e7400700 1 -- 172.21.15.44:0/3940181066 <== osd.4 v2:172.21.15.78:6808/856268583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe0f00635f0 con 0x7fe0d40ad530 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] conn(0x7fe0d40ad530 msgr2=0x7fe0d40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] conn(0x7fe0d40ad530 0x7fe0d40af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0d40a3d50 msgr2=0x7fe0d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe0d40a3d50 0x7fe0d40a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0000c00 tx=0x7fe0e0005fb0 comp rx=0 tx=0).stop 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f0109c90 msgr2=0x7fe0f01a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 --2- 172.21.15.44:0/3940181066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f0109c90 0x7fe0f01a5b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800ea30 tx=0x7fe0d800ed40 comp rx=0 tx=0).stop 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0ef200700 1 -- 172.21.15.44:0/3940181066 reap_dead start 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 shutdown_connections 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 >> 172.21.15.44:0/3940181066 conn(0x7fe0f0076220 msgr2=0x7fe0f00767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 shutdown_connections 2024-04-20T11:20:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:42.003+0000 7fe0f5400700 1 -- 172.21.15.44:0/3940181066 wait complete. 2024-04-20T11:20:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:47 smithi165 ceph-mon[21975]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-20T11:20:47.632 INFO:teuthology.orchestra.run.smithi044.stdout:107374182468 2024-04-20T11:20:47.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-20T11:20:49.309 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:49 smithi044 ceph-mon[16855]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-20T11:20:49.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:49 smithi165 ceph-mon[21975]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-20T11:20:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:49 smithi078 ceph-mon[21464]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-20T11:20:49.816 INFO:teuthology.orchestra.run.smithi044.stdout:141733920832 2024-04-20T11:20:49.816 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-20T11:20:50.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-20T11:20:50.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-20T11:20:50.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-20T11:20:50.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-20T11:20:50.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-20T11:20:50.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:50 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-20T11:20:50.551 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-20T11:20:50.551 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-20T11:20:50.551 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-20T11:20:50.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-20T11:20:50.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-20T11:20:50.552 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:50 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-20T11:20:50.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-20T11:20:50.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-20T11:20:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-20T11:20:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-20T11:20:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-20T11:20:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:50 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-20T11:20:51.247 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:51 smithi044 ceph-mon[16855]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 -- 172.21.15.44:0/1354883867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa644102310 msgr2=0x7fa64410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 --2- 172.21.15.44:0/1354883867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa644102310 0x7fa64410e800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00b0a0 tx=0x7fa63c00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 -- 172.21.15.44:0/1354883867 shutdown_connections 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 --2- 172.21.15.44:0/1354883867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa644102310 0x7fa64410e800 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 --2- 172.21.15.44:0/1354883867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6441079a0 0x7fa644107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 --2- 172.21.15.44:0/1354883867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa644101950 0x7fa644101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 -- 172.21.15.44:0/1354883867 >> 172.21.15.44:0/1354883867 conn(0x7fa644078900 msgr2=0x7fa644078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 -- 172.21.15.44:0/1354883867 shutdown_connections 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.487+0000 7fa64ae00700 1 -- 172.21.15.44:0/1354883867 wait complete. 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.488+0000 7fa64ae00700 1 Processor -- start 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.488+0000 7fa64ae00700 1 -- start start 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa64ae00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa644101950 0x7fa64419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa64ae00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa644102310 0x7fa64419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa64ae00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441079a0 0x7fa64419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa64ae00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa644111e20 con 0x7fa644102310 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa64ae00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa644111ca0 con 0x7fa6441079a0 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa64ae00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa644111fa0 con 0x7fa644101950 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa63be00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa644102310 0x7fa64419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa644101950 0x7fa64419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa644101950 0x7fa64419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47778/0 (socket says 172.21.15.44:47778) 2024-04-20T11:20:51.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 -- 172.21.15.44:0/2326499909 learned_addr learned my addr 172.21.15.44:0/2326499909 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa648e00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441079a0 0x7fa64419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 -- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441079a0 msgr2=0x7fa64419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441079a0 0x7fa64419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 -- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa644102310 msgr2=0x7fa64419b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa644102310 0x7fa64419b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.489+0000 7fa643e00700 1 -- 172.21.15.44:0/2326499909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6441a04c0 con 0x7fa644101950 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.490+0000 7fa643e00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa644101950 0x7fa64419b4b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e9c0 tx=0x7fa63400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:51.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.490+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63400cbd0 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.490+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa63400ee60 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.490+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6441a07b0 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.490+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634010430 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.490+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441a8030 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.491+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa64406b190 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.493+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa634004750 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.494+0000 7fa641600700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6280a3cd0 0x7fa6280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.495+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa6340d0b50 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.495+0000 7fa63be00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6280a3cd0 0x7fa6280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.495+0000 7fa641600700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] conn(0x7fa6280ad4b0 0x7fa6280af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.495+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 --> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa63401ee50 con 0x7fa6280ad4b0 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.495+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa6340d0f00 con 0x7fa644101950 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.495+0000 7fa648e00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] conn(0x7fa6280ad4b0 0x7fa6280af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:51.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.496+0000 7fa63be00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6280a3cd0 0x7fa6280a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa644101860 tx=0x7fa62c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.496+0000 7fa648e00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] conn(0x7fa6280ad4b0 0x7fa6280af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.497+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== osd.10 v2:172.21.15.78:6824/259959588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa63401ee50 con 0x7fa6280ad4b0 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.515+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 --> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa644103070 con 0x7fa6280ad4b0 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.516+0000 7fa641600700 1 -- 172.21.15.44:0/2326499909 <== osd.10 v2:172.21.15.78:6824/259959588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa644103070 con 0x7fa6280ad4b0 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] conn(0x7fa6280ad4b0 msgr2=0x7fa6280af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] conn(0x7fa6280ad4b0 0x7fa6280af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6280a3cd0 msgr2=0x7fa6280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa6280a3cd0 0x7fa6280a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa644101860 tx=0x7fa62c0073d0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa644101950 msgr2=0x7fa64419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 --2- 172.21.15.44:0/2326499909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa644101950 0x7fa64419b4b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e9c0 tx=0x7fa63400ecd0 comp rx=0 tx=0).stop 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa648e00700 1 -- 172.21.15.44:0/2326499909 reap_dead start 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 shutdown_connections 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 >> 172.21.15.44:0/2326499909 conn(0x7fa644078900 msgr2=0x7fa64406a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 shutdown_connections 2024-04-20T11:20:51.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:49.517+0000 7fa64ae00700 1 -- 172.21.15.44:0/2326499909 wait complete. 2024-04-20T11:20:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-20T11:20:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-20T11:20:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-20T11:20:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-20T11:20:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-20T11:20:51.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-20T11:20:51.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-20T11:20:51.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:51 smithi165 ceph-mon[21975]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T11:20:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-20T11:20:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:51 smithi078 ceph-mon[21464]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T11:20:51.716 INFO:teuthology.orchestra.run.smithi044.stdout:287762808866 2024-04-20T11:20:51.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-20T11:20:52.458 INFO:teuthology.orchestra.run.smithi044.stdout:55834574930 2024-04-20T11:20:52.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-20T11:20:52.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:52 smithi165 ceph-mon[21975]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T11:20:52.529 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:52 smithi044 ceph-mon[16855]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T11:20:52.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 -- 172.21.15.44:0/3323734787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580075710 msgr2=0x7f7580113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:52.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 --2- 172.21.15.44:0/3323734787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580075710 0x7f7580113450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756c009990 tx=0x7f756c009ca0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 -- 172.21.15.44:0/3323734787 shutdown_connections 2024-04-20T11:20:52.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 --2- 172.21.15.44:0/3323734787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7580113990 0x7f7580115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 --2- 172.21.15.44:0/3323734787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7580074df0 0x7f75800751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 --2- 172.21.15.44:0/3323734787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580075710 0x7f7580113450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 -- 172.21.15.44:0/3323734787 >> 172.21.15.44:0/3323734787 conn(0x7f75800fe4d0 msgr2=0x7f75801008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 -- 172.21.15.44:0/3323734787 shutdown_connections 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.249+0000 7f757fe00700 1 -- 172.21.15.44:0/3323734787 wait complete. 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.250+0000 7f757fe00700 1 Processor -- start 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.250+0000 7f757fe00700 1 -- start start 2024-04-20T11:20:52.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757fe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580074df0 0x7f75801a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757fe00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7580075710 0x7f75801a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7580113990 0x7f758019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757fe00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7580119400 con 0x7f7580074df0 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757fe00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7580119280 con 0x7f7580075710 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757fe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7580119580 con 0x7f7580113990 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757ea00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580074df0 0x7f75801a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757ea00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580074df0 0x7f75801a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34042/0 (socket says 172.21.15.44:34042) 2024-04-20T11:20:52.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757e000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7580075710 0x7f75801a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757ea00700 1 -- 172.21.15.44:0/2910182167 learned_addr learned my addr 172.21.15.44:0/2910182167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757f400700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7580113990 0x7f758019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757e000700 1 -- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7580113990 msgr2=0x7f758019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757e000700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7580113990 0x7f758019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757e000700 1 -- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580074df0 msgr2=0x7f75801a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757e000700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580074df0 0x7f75801a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.251+0000 7f757e000700 1 -- 172.21.15.44:0/2910182167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758019ff20 con 0x7f7580075710 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.252+0000 7f757e000700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7580075710 0x7f75801a5ab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f756c006010 tx=0x7f756c005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:52.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.252+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c027070 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.252+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f756c0056b0 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.252+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0173f0 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.252+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75801a01b0 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.252+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801ac320 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.253+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75800631a0 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.255+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f756c02a070 con 0x7f7580075710 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.257+0000 7f7584a00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75640a3cd0 0x7f75640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:52.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.257+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f756c0d12e0 con 0x7f7580075710 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.257+0000 7f757ea00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75640a3cd0 0x7f75640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.257+0000 7f7584a00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] conn(0x7f75640ad710 0x7f75640afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.257+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 --> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75640b0280 con 0x7f75640ad710 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.257+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f756c0d8050 con 0x7f7580075710 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.258+0000 7f757f400700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] conn(0x7f75640ad710 0x7f75640afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.258+0000 7f757ea00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75640a3cd0 0x7f75640a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7570009d20 tx=0x7f7570009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:52.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.258+0000 7f757f400700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] conn(0x7f75640ad710 0x7f75640afbb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.259+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== osd.2 v2:172.21.15.165:6800/1225247316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75640b0280 con 0x7f75640ad710 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.283+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 --> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7580063620 con 0x7f75640ad710 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.283+0000 7f7584a00700 1 -- 172.21.15.44:0/2910182167 <== osd.2 v2:172.21.15.165:6800/1225247316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7580063620 con 0x7f75640ad710 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] conn(0x7f75640ad710 msgr2=0x7f75640afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] conn(0x7f75640ad710 0x7f75640afbb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75640a3cd0 msgr2=0x7f75640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f75640a3cd0 0x7f75640a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7570009d20 tx=0x7f7570009260 comp rx=0 tx=0).stop 2024-04-20T11:20:52.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7580075710 msgr2=0x7f75801a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 --2- 172.21.15.44:0/2910182167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7580075710 0x7f75801a5ab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f756c006010 tx=0x7f756c005590 comp rx=0 tx=0).stop 2024-04-20T11:20:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757f400700 1 -- 172.21.15.44:0/2910182167 reap_dead start 2024-04-20T11:20:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 shutdown_connections 2024-04-20T11:20:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 >> 172.21.15.44:0/2910182167 conn(0x7f75800fe4d0 msgr2=0x7f75800ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 shutdown_connections 2024-04-20T11:20:52.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:52.284+0000 7f757fe00700 1 -- 172.21.15.44:0/2910182167 wait complete. 2024-04-20T11:20:52.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:52 smithi078 ceph-mon[21464]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T11:20:53.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:53 smithi165 ceph-mon[21975]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-20T11:20:53.562 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:53 smithi044 ceph-mon[16855]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-20T11:20:53.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.602+0000 7f162a800700 1 -- 172.21.15.44:0/1325440273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 msgr2=0x7f16241098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:53.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.602+0000 7f162a800700 1 --2- 172.21.15.44:0/1325440273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f16241098c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f160c009990 tx=0x7f160c009ca0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 -- 172.21.15.44:0/1325440273 shutdown_connections 2024-04-20T11:20:53.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 --2- 172.21.15.44:0/1325440273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624109fa0 0x7f1624111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 --2- 172.21.15.44:0/1325440273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1624104ec0 0x7f16241052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 --2- 172.21.15.44:0/1325440273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f16241098c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 -- 172.21.15.44:0/1325440273 >> 172.21.15.44:0/1325440273 conn(0x7f1624100730 msgr2=0x7f1624102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:53.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 -- 172.21.15.44:0/1325440273 shutdown_connections 2024-04-20T11:20:53.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.603+0000 7f162a800700 1 -- 172.21.15.44:0/1325440273 wait complete. 2024-04-20T11:20:53.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.604+0000 7f162a800700 1 Processor -- start 2024-04-20T11:20:53.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 -- start start 2024-04-20T11:20:53.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1624104ec0 0x7f162419b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f162419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624109fa0 0x7f162419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1624115120 con 0x7f1624109fa0 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1624114fa0 con 0x7f1624105870 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f162a800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16241152a0 con 0x7f1624104ec0 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f161be00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f162419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f1623e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624109fa0 0x7f162419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f161be00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f162419ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:49524/0 (socket says 172.21.15.44:49524) 2024-04-20T11:20:53.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f161be00700 1 -- 172.21.15.44:0/2140624300 learned_addr learned my addr 172.21.15.44:0/2140624300 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.605+0000 7f1623400700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1624104ec0 0x7f162419b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f161be00700 1 -- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1624104ec0 msgr2=0x7f162419b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f161be00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1624104ec0 0x7f162419b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f161be00700 1 -- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624109fa0 msgr2=0x7f162419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f161be00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624109fa0 0x7f162419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f161be00700 1 -- 172.21.15.44:0/2140624300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16241a0540 con 0x7f1624105870 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f161be00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f162419ba70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f160c000c00 tx=0x7f160c00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c026070 con 0x7f1624105870 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f160c003d90 con 0x7f1624105870 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c021410 con 0x7f1624105870 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.606+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16241a07d0 con 0x7f1624105870 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.607+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16241a80b0 con 0x7f1624105870 2024-04-20T11:20:53.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.607+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16240631e0 con 0x7f1624105870 2024-04-20T11:20:53.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.610+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f160c029070 con 0x7f1624105870 2024-04-20T11:20:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.611+0000 7f1620c00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f16080a3cd0 0x7f16080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:53.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.611+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f160c0d1990 con 0x7f1624105870 2024-04-20T11:20:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.611+0000 7f1623400700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f16080a3cd0 0x7f16080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.611+0000 7f1620c00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] conn(0x7f16080ad710 0x7f16080afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.611+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 --> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f160c00dec0 con 0x7f16080ad710 2024-04-20T11:20:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.611+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f160c0d1d40 con 0x7f1624105870 2024-04-20T11:20:53.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.612+0000 7f1623e00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] conn(0x7f16080ad710 0x7f16080afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:53.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.612+0000 7f1623400700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f16080a3cd0 0x7f16080a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1614009c60 tx=0x7f1614009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:53.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.612+0000 7f1623e00700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] conn(0x7f16080ad710 0x7f16080afbb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:53.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.613+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== osd.8 v2:172.21.15.165:6816/2878265181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f160c00dec0 con 0x7f16080ad710 2024-04-20T11:20:53.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.632+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 --> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1624063660 con 0x7f16080ad710 2024-04-20T11:20:53.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f1620c00700 1 -- 172.21.15.44:0/2140624300 <== osd.8 v2:172.21.15.165:6816/2878265181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1624063660 con 0x7f16080ad710 2024-04-20T11:20:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] conn(0x7f16080ad710 msgr2=0x7f16080afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f162a800700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] conn(0x7f16080ad710 0x7f16080afbb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f16080a3cd0 msgr2=0x7f16080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f162a800700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f16080a3cd0 0x7f16080a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1614009c60 tx=0x7f1614009260 comp rx=0 tx=0).stop 2024-04-20T11:20:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 msgr2=0x7f162419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f162a800700 1 --2- 172.21.15.44:0/2140624300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1624105870 0x7f162419ba70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f160c000c00 tx=0x7f160c00bb20 comp rx=0 tx=0).stop 2024-04-20T11:20:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.633+0000 7f1623e00700 1 -- 172.21.15.44:0/2140624300 reap_dead start 2024-04-20T11:20:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.634+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 shutdown_connections 2024-04-20T11:20:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.634+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 >> 172.21.15.44:0/2140624300 conn(0x7f1624100730 msgr2=0x7f1624101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:53.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.634+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 shutdown_connections 2024-04-20T11:20:53.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:53.634+0000 7f162a800700 1 -- 172.21.15.44:0/2140624300 wait complete. 2024-04-20T11:20:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:53 smithi078 ceph-mon[21464]: pgmap v348: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-20T11:20:55.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:55 smithi044 ceph-mon[16855]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-04-20T11:20:55.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:55 smithi165 ceph-mon[21975]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-04-20T11:20:55.593 INFO:teuthology.orchestra.run.smithi044.stdout:309237645340 2024-04-20T11:20:55.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-20T11:20:55.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:55 smithi078 ceph-mon[21464]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-04-20T11:20:55.941 INFO:teuthology.orchestra.run.smithi044.stdout:77309411406 2024-04-20T11:20:55.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-20T11:20:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.076+0000 7f255be00700 1 -- 172.21.15.44:0/2968558002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c102150 msgr2=0x7f255c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.076+0000 7f255be00700 1 --2- 172.21.15.44:0/2968558002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c102150 0x7f255c10ee10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25480098f0 tx=0x7f2548009c00 comp rx=0 tx=0).stop 2024-04-20T11:20:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 -- 172.21.15.44:0/2968558002 shutdown_connections 2024-04-20T11:20:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 --2- 172.21.15.44:0/2968558002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255c10f350 0x7f255c111740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 --2- 172.21.15.44:0/2968558002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255c1017a0 0x7f255c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 --2- 172.21.15.44:0/2968558002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c102150 0x7f255c10ee10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 -- 172.21.15.44:0/2968558002 >> 172.21.15.44:0/2968558002 conn(0x7f255c0fd3e0 msgr2=0x7f255c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 -- 172.21.15.44:0/2968558002 shutdown_connections 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.077+0000 7f255be00700 1 -- 172.21.15.44:0/2968558002 wait complete. 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.078+0000 7f255be00700 1 Processor -- start 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 -- start start 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c1017a0 0x7f255c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255c102150 0x7f255c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255c10f350 0x7f255c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1150b0 con 0x7f255c10f350 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f255c114f30 con 0x7f255c1017a0 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255be00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f255c115230 con 0x7f255c102150 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.079+0000 7f255b400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255c10f350 0x7f255c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c1017a0 0x7f255c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c1017a0 0x7f255c19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:49556/0 (socket says 172.21.15.44:49556) 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 -- 172.21.15.44:0/2720038945 learned_addr learned my addr 172.21.15.44:0/2720038945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255a000700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255c102150 0x7f255c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 -- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255c102150 msgr2=0x7f255c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255c102150 0x7f255c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 -- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255c10f350 msgr2=0x7f255c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255c10f350 0x7f255c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 -- 172.21.15.44:0/2720038945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c1a04f0 con 0x7f255c1017a0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255b400700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255c10f350 0x7f255c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.080+0000 7f255aa00700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c1017a0 0x7f255c19b4e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f254c00e9a0 tx=0x7f254c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.081+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c00ca10 con 0x7f255c1017a0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.081+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f254c00cbb0 con 0x7f255c1017a0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.081+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c018710 con 0x7f255c1017a0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.081+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255c1a07e0 con 0x7f255c1017a0 2024-04-20T11:20:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.081+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f255c1a8060 con 0x7f255c1017a0 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.082+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f255c063160 con 0x7f255c1017a0 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.085+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f254c010430 con 0x7f255c1017a0 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.086+0000 7f2553400700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f25400a3d50 0x7f25400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.086+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f254c0d0e60 con 0x7f255c1017a0 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.087+0000 7f255a000700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f25400a3d50 0x7f25400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.087+0000 7f2553400700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] conn(0x7f25400ad790 0x7f25400afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.087+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 --> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25400b0300 con 0x7f25400ad790 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.087+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f254c0d1210 con 0x7f255c1017a0 2024-04-20T11:20:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.087+0000 7f255b400700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] conn(0x7f25400ad790 0x7f25400afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.087+0000 7f255a000700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f25400a3d50 0x7f25400a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2548009810 tx=0x7f2548019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.088+0000 7f255b400700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] conn(0x7f25400ad790 0x7f25400afc30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.089+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== osd.11 v2:172.21.15.165:6824/1018684667 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25400b0300 con 0x7f25400ad790 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.107+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 --> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f255c063560 con 0x7f25400ad790 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.108+0000 7f2553400700 1 -- 172.21.15.44:0/2720038945 <== osd.11 v2:172.21.15.165:6824/1018684667 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f255c063560 con 0x7f25400ad790 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] conn(0x7f25400ad790 msgr2=0x7f25400afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] conn(0x7f25400ad790 0x7f25400afc30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f25400a3d50 msgr2=0x7f25400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f25400a3d50 0x7f25400a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2548009810 tx=0x7f2548019040 comp rx=0 tx=0).stop 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c1017a0 msgr2=0x7f255c19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 --2- 172.21.15.44:0/2720038945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255c1017a0 0x7f255c19b4e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f254c00e9a0 tx=0x7f254c00ecb0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255b400700 1 -- 172.21.15.44:0/2720038945 reap_dead start 2024-04-20T11:20:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 shutdown_connections 2024-04-20T11:20:56.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.109+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 >> 172.21.15.44:0/2720038945 conn(0x7f255c0fd3e0 msgr2=0x7f255c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:56.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.110+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 shutdown_connections 2024-04-20T11:20:56.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:55.110+0000 7f255be00700 1 -- 172.21.15.44:0/2720038945 wait complete. 2024-04-20T11:20:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 -- 172.21.15.44:0/3387750065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0105880 msgr2=0x7f07f01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 --2- 172.21.15.44:0/3387750065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0105880 0x7f07f01098d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0099c0 tx=0x7f07dc009cd0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 -- 172.21.15.44:0/3387750065 shutdown_connections 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 --2- 172.21.15.44:0/3387750065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07f0109fb0 0x7f07f0111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 --2- 172.21.15.44:0/3387750065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f0104ed0 0x7f07f01052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 --2- 172.21.15.44:0/3387750065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0105880 0x7f07f01098d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 -- 172.21.15.44:0/3387750065 >> 172.21.15.44:0/3387750065 conn(0x7f07f0100760 msgr2=0x7f07f0102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 -- 172.21.15.44:0/3387750065 shutdown_connections 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.681+0000 7f07f5000700 1 -- 172.21.15.44:0/3387750065 wait complete. 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.682+0000 7f07f5000700 1 Processor -- start 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.682+0000 7f07f5000700 1 -- start start 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07f5000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07f0104ed0 0x7f07f019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07f5000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f0105880 0x7f07f019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07f5000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0109fb0 0x7f07f019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07f5000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01150b0 con 0x7f07f0109fb0 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07f5000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0114f30 con 0x7f07f0104ed0 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07f5000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0115230 con 0x7f07f0105880 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07ee800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0109fb0 0x7f07f019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07ee800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0109fb0 0x7f07f019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34122/0 (socket says 172.21.15.44:34122) 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07ee800700 1 -- 172.21.15.44:0/952572321 learned_addr learned my addr 172.21.15.44:0/952572321 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07ede00700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07f0104ed0 0x7f07f019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07ed400700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f0105880 0x7f07f019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.683+0000 7f07ee800700 1 -- 172.21.15.44:0/952572321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f0105880 msgr2=0x7f07f019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ee800700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f0105880 0x7f07f019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ee800700 1 -- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07f0104ed0 msgr2=0x7f07f019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ee800700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07f0104ed0 0x7f07f019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ee800700 1 -- 172.21.15.44:0/952572321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f01a0450 con 0x7f07f0109fb0 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ede00700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07f0104ed0 0x7f07f019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:56.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ed400700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f0105880 0x7f07f019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07ee800700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0109fb0 0x7f07f019fd10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07e800ef30 tx=0x7f07e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e80092e0 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07e8004bb0 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f01a06e0 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8004280 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.684+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f01a7fc0 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.685+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07f0101b10 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.688+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f07e801b030 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.689+0000 7f07e6a00700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f07d40a3cd0 0x7f07d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.689+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f07e80d0960 con 0x7f07f0109fb0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.689+0000 7f07ede00700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f07d40a3cd0 0x7f07d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.689+0000 7f07e6a00700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] conn(0x7f07d40ad710 0x7f07d40afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.690+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 --> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07d40b0280 con 0x7f07d40ad710 2024-04-20T11:20:56.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.690+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f07e80d0d10 con 0x7f07f0109fb0 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.690+0000 7f07ed400700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] conn(0x7f07d40ad710 0x7f07d40afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.690+0000 7f07ede00700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f07d40a3cd0 0x7f07d40a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07e000afd0 tx=0x7f07e000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.691+0000 7f07ed400700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] conn(0x7f07d40ad710 0x7f07d40afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.691+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== osd.7 v2:172.21.15.78:6816/1412804643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07d40b0280 con 0x7f07d40ad710 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.710+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 --> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07f01063b0 con 0x7f07d40ad710 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07e6a00700 1 -- 172.21.15.44:0/952572321 <== osd.7 v2:172.21.15.78:6816/1412804643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07f01063b0 con 0x7f07d40ad710 2024-04-20T11:20:56.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] conn(0x7f07d40ad710 msgr2=0x7f07d40afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07f5000700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] conn(0x7f07d40ad710 0x7f07d40afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f07d40a3cd0 msgr2=0x7f07d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07f5000700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f07d40a3cd0 0x7f07d40a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07e000afd0 tx=0x7f07e000a380 comp rx=0 tx=0).stop 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0109fb0 msgr2=0x7f07f019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07f5000700 1 --2- 172.21.15.44:0/952572321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f0109fb0 0x7f07f019fd10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07e800ef30 tx=0x7f07e800c3a0 comp rx=0 tx=0).stop 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.711+0000 7f07ee800700 1 -- 172.21.15.44:0/952572321 reap_dead start 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.712+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 shutdown_connections 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.712+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 >> 172.21.15.44:0/952572321 conn(0x7f07f0100760 msgr2=0x7f07f01010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.712+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 shutdown_connections 2024-04-20T11:20:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:56.712+0000 7f07f5000700 1 -- 172.21.15.44:0/952572321 wait complete. 2024-04-20T11:20:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.401+0000 7f6cf8000700 1 -- 172.21.15.44:0/874030706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 msgr2=0x7f6cf010dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.401+0000 7f6cf8000700 1 --2- 172.21.15.44:0/874030706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf010dce0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098f0 tx=0x7f6cdc009c00 comp rx=0 tx=0).stop 2024-04-20T11:20:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 -- 172.21.15.44:0/874030706 shutdown_connections 2024-04-20T11:20:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 --2- 172.21.15.44:0/874030706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf010e3c0 0x7f6cf0115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 --2- 172.21.15.44:0/874030706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cf01092e0 0x7f6cf01096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 --2- 172.21.15.44:0/874030706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf010dce0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 -- 172.21.15.44:0/874030706 >> 172.21.15.44:0/874030706 conn(0x7f6cf0076220 msgr2=0x7f6cf0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 -- 172.21.15.44:0/874030706 shutdown_connections 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.402+0000 7f6cf8000700 1 -- 172.21.15.44:0/874030706 wait complete. 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.403+0000 7f6cf8000700 1 Processor -- start 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.403+0000 7f6cf8000700 1 -- start start 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf8000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf01092e0 0x7f6cf01a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf8000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf01a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf8000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cf010e3c0 0x7f6cf019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf8000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0119510 con 0x7f6cf01092e0 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf8000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0119390 con 0x7f6cf0109c90 2024-04-20T11:20:57.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf8000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0119690 con 0x7f6cf010e3c0 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf4c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf01a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf5600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf01092e0 0x7f6cf01a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf4c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf01a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:49544/0 (socket says 172.21.15.44:49544) 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.404+0000 7f6cf4c00700 1 -- 172.21.15.44:0/3563596951 learned_addr learned my addr 172.21.15.44:0/3563596951 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf6000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cf010e3c0 0x7f6cf019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf4c00700 1 -- 172.21.15.44:0/3563596951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cf010e3c0 msgr2=0x7f6cf019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf4c00700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cf010e3c0 0x7f6cf019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf4c00700 1 -- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf01092e0 msgr2=0x7f6cf01a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf4c00700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf01092e0 0x7f6cf01a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf4c00700 1 -- 172.21.15.44:0/3563596951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf01a0090 con 0x7f6cf0109c90 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf4c00700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf01a5bd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c010 tx=0x7f6cdc010fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00b7c0 con 0x7f6cf0109c90 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6cdc00f960 con 0x7f6cf0109c90 2024-04-20T11:20:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.405+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf01a0320 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.406+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc003b30 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.406+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf01ac610 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.406+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6cf00631a0 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.409+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6cdc003cd0 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.410+0000 7f6cee000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6cd80a3d50 0x7f6cd80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.410+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f6cdc0d1910 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.410+0000 7f6cf5600700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6cd80a3d50 0x7f6cd80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.410+0000 7f6cee000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] conn(0x7f6cd80ad790 0x7f6cd80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.410+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 --> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cdc00be00 con 0x7f6cd80ad790 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.410+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6cdc0d1cc0 con 0x7f6cf0109c90 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.411+0000 7f6cf6000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] conn(0x7f6cd80ad790 0x7f6cd80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.411+0000 7f6cf5600700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6cd80a3d50 0x7f6cd80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009c60 tx=0x7f6ce4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.411+0000 7f6cf6000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] conn(0x7f6cd80ad790 0x7f6cd80afc30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.412+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== osd.6 v2:172.21.15.44:6818/2930726508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6cdc00be00 con 0x7f6cd80ad790 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.432+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 --> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6cf00635a0 con 0x7f6cd80ad790 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.432+0000 7f6cee000700 1 -- 172.21.15.44:0/3563596951 <== osd.6 v2:172.21.15.44:6818/2930726508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6cf00635a0 con 0x7f6cd80ad790 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] conn(0x7f6cd80ad790 msgr2=0x7f6cd80afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] conn(0x7f6cd80ad790 0x7f6cd80afc30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6cd80a3d50 msgr2=0x7f6cd80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6cd80a3d50 0x7f6cd80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009c60 tx=0x7f6ce4009260 comp rx=0 tx=0).stop 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 msgr2=0x7f6cf01a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 --2- 172.21.15.44:0/3563596951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cf0109c90 0x7f6cf01a5bd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c010 tx=0x7f6cdc010fa0 comp rx=0 tx=0).stop 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf6000700 1 -- 172.21.15.44:0/3563596951 reap_dead start 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 shutdown_connections 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 >> 172.21.15.44:0/3563596951 conn(0x7f6cf0076220 msgr2=0x7f6cf00767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 shutdown_connections 2024-04-20T11:20:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:54.433+0000 7f6cf8000700 1 -- 172.21.15.44:0/3563596951 wait complete. 2024-04-20T11:20:57.650 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:57 smithi044 ceph-mon[16855]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-20T11:20:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:57 smithi078 ceph-mon[21464]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-20T11:20:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:57 smithi165 ceph-mon[21975]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-20T11:20:58.681 INFO:teuthology.orchestra.run.smithi044.stdout:197568495669 2024-04-20T11:20:58.682 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-20T11:20:58.812 INFO:teuthology.orchestra.run.smithi044.stdout:253403070506 2024-04-20T11:20:58.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-20T11:20:59.657 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:20:59 smithi044 ceph-mon[16855]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-20T11:20:59.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:20:59 smithi078 ceph-mon[21464]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-20T11:20:59.722 INFO:teuthology.orchestra.run.smithi044.stdout:343597383704 2024-04-20T11:20:59.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-20T11:20:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:20:59 smithi165 ceph-mon[21975]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-20T11:20:59.832 INFO:teuthology.orchestra.run.smithi044.stdout:219043332144 2024-04-20T11:20:59.833 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-20T11:20:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 -- 172.21.15.44:0/2650208169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93701092e0 msgr2=0x7f93701096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 --2- 172.21.15.44:0/2650208169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93701092e0 0x7f93701096c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93600099c0 tx=0x7f9360009cd0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 -- 172.21.15.44:0/2650208169 shutdown_connections 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 --2- 172.21.15.44:0/2650208169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f937010e3c0 0x7f9370115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 --2- 172.21.15.44:0/2650208169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93701092e0 0x7f93701096c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 --2- 172.21.15.44:0/2650208169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9370109c90 0x7f937010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 -- 172.21.15.44:0/2650208169 >> 172.21.15.44:0/2650208169 conn(0x7f9370076220 msgr2=0x7f9370107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 -- 172.21.15.44:0/2650208169 shutdown_connections 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.567+0000 7f9376800700 1 -- 172.21.15.44:0/2650208169 wait complete. 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.568+0000 7f9376800700 1 Processor -- start 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.568+0000 7f9376800700 1 -- start start 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9376800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93701092e0 0x7f93701a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9376800700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9370109c90 0x7f93701a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9376800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f937010e3c0 0x7f937019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9376800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9370119500 con 0x7f937010e3c0 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9376800700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9370119380 con 0x7f9370109c90 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9376800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9370119680 con 0x7f93701092e0 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f936f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93701092e0 0x7f93701a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f936fe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f937010e3c0 0x7f937019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f9367e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9370109c90 0x7f93701a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f936f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93701092e0 0x7f93701a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54214/0 (socket says 172.21.15.44:54214) 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.569+0000 7f936f400700 1 -- 172.21.15.44:0/3809173701 learned_addr learned my addr 172.21.15.44:0/3809173701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936f400700 1 -- 172.21.15.44:0/3809173701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9370109c90 msgr2=0x7f93701a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936f400700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9370109c90 0x7f93701a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936f400700 1 -- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f937010e3c0 msgr2=0x7f937019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:59.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936f400700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f937010e3c0 0x7f937019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936f400700 1 -- 172.21.15.44:0/3809173701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937019ff80 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936fe00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f937010e3c0 0x7f937019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936f400700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93701092e0 0x7f93701a5620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9360006010 tx=0x7f9360003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360009e60 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f936000be40 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93701a0210 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.570+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600173f0 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.571+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93701ac480 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.571+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93700631e0 con 0x7f93701092e0 2024-04-20T11:20:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.574+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f936000f680 con 0x7f93701092e0 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.575+0000 7f936cc00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f93540a3cd0 0x7f93540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.575+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f93600d1470 con 0x7f93701092e0 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.575+0000 7f9367e00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f93540a3cd0 0x7f93540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.575+0000 7f936cc00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] conn(0x7f93540ad710 0x7f93540afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.576+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 --> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9360024ec0 con 0x7f93540ad710 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.576+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f93600d1820 con 0x7f93701092e0 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.576+0000 7f936fe00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] conn(0x7f93540ad710 0x7f93540afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:20:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.576+0000 7f9367e00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f93540a3cd0 0x7f93540a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9370108890 tx=0x7f9358009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.576+0000 7f936fe00700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] conn(0x7f93540ad710 0x7f93540afbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:20:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.577+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== osd.0 v2:172.21.15.44:6802/694121710 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9360024ec0 con 0x7f93540ad710 2024-04-20T11:20:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.596+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 --> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9370063660 con 0x7f93540ad710 2024-04-20T11:20:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f936cc00700 1 -- 172.21.15.44:0/3809173701 <== osd.0 v2:172.21.15.44:6802/694121710 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9370063660 con 0x7f93540ad710 2024-04-20T11:20:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] conn(0x7f93540ad710 msgr2=0x7f93540afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f9376800700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] conn(0x7f93540ad710 0x7f93540afbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:20:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f93540a3cd0 msgr2=0x7f93540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f9376800700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f93540a3cd0 0x7f93540a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9370108890 tx=0x7f9358009260 comp rx=0 tx=0).stop 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93701092e0 msgr2=0x7f93701a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.597+0000 7f9376800700 1 --2- 172.21.15.44:0/3809173701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93701092e0 0x7f93701a5620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9360006010 tx=0x7f9360003c30 comp rx=0 tx=0).stop 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.598+0000 7f936fe00700 1 -- 172.21.15.44:0/3809173701 reap_dead start 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.598+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 shutdown_connections 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.598+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 >> 172.21.15.44:0/3809173701 conn(0x7f9370076220 msgr2=0x7f9370076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.598+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 shutdown_connections 2024-04-20T11:20:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:20:58.598+0000 7f9376800700 1 -- 172.21.15.44:0/3809173701 wait complete. 2024-04-20T11:21:00.322 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:00.661 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:00.682 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:00.682 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:00.706 INFO:teuthology.orchestra.run.smithi044.stdout:34359738455 2024-04-20T11:21:00.706 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-20T11:21:00.779 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:01.174 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:01.440 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:01 smithi044 ceph-mon[16855]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 11 B/s, 0 objects/s recovering 2024-04-20T11:21:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:01.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:01 smithi078 ceph-mon[21464]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 11 B/s, 0 objects/s recovering 2024-04-20T11:21:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:01 smithi165 ceph-mon[21975]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 11 B/s, 0 objects/s recovering 2024-04-20T11:21:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:03.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:03 smithi044 ceph-mon[16855]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-20T11:21:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:03 smithi165 ceph-mon[21975]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-20T11:21:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:03 smithi078 ceph-mon[21464]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-20T11:21:05.425 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:05 smithi165 ceph-mon[21975]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T11:21:05.425 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:05 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:21:05.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:05 smithi078 ceph-mon[21464]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T11:21:05.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:05 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:21:05.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:05 smithi044 ceph-mon[16855]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T11:21:05.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:05 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:21:06.433 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:06.491 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:07.178 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.178 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.179 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:06 smithi044 ceph-mon[16855]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T11:21:07.179 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.179 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:06 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.240 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.240 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.240 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:06 smithi078 ceph-mon[21464]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T11:21:07.240 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.240 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:06 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:06 smithi165 ceph-mon[21975]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T11:21:07.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:07.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:06 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:09.422 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:09 smithi078 ceph-mon[21464]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:09.431 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:09 smithi044 ceph-mon[16855]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:09.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:09 smithi165 ceph-mon[21975]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:11.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:11 smithi078 ceph-mon[21464]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:11.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:11 smithi044 ceph-mon[16855]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:11.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:11 smithi165 ceph-mon[21975]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:13.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:13 smithi044 ceph-mon[16855]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:13.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:13 smithi078 ceph-mon[21464]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:13.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:13 smithi165 ceph-mon[21975]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:13.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 -- 172.21.15.44:0/492757979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 msgr2=0x7f2c0c075060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:13.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 --2- 172.21.15.44:0/492757979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 0x7f2c0c075060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009990 tx=0x7f2bfc009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 -- 172.21.15.44:0/492757979 shutdown_connections 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 --2- 172.21.15.44:0/492757979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 0x7f2c0c111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 --2- 172.21.15.44:0/492757979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 0x7f2c0c075060 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 --2- 172.21.15.44:0/492757979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 -- 172.21.15.44:0/492757979 >> 172.21.15.44:0/492757979 conn(0x7f2c0c079bb0 msgr2=0x7f2c0c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 -- 172.21.15.44:0/492757979 shutdown_connections 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.569+0000 7f2c12400700 1 -- 172.21.15.44:0/492757979 wait complete. 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.570+0000 7f2c12400700 1 Processor -- start 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.570+0000 7f2c12400700 1 -- start start 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c12400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 0x7f2c0c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c12400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:13.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c12400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 0x7f2c0c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c12400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1150a0 con 0x7f2c0c0755a0 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c12400700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c114f20 con 0x7f2c0c10f450 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c12400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c115220 con 0x7f2c0c074c80 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c02a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c02a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c19b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42208/0 (socket says 172.21.15.44:42208) 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c02a00700 1 -- 172.21.15.44:0/1558842003 learned_addr learned my addr 172.21.15.44:0/1558842003 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c0be00700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 0x7f2c0c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.571+0000 7f2c0b400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 0x7f2c0c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c02a00700 1 -- 172.21.15.44:0/1558842003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 msgr2=0x7f2c0c19b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c02a00700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 0x7f2c0c19b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c02a00700 1 -- 172.21.15.44:0/1558842003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 msgr2=0x7f2c0c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c02a00700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 0x7f2c0c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c02a00700 1 -- 172.21.15.44:0/1558842003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c1a0450 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c0be00700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 0x7f2c0c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c02a00700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c19b9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e970 tx=0x7f2bf400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4004220 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2bf400cce0 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c1a0740 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4010430 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.572+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c1a7fc0 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.573+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c069740 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.576+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2bf400c7e0 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.577+0000 7f2c08c00700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2bf00a3cd0 0x7f2bf00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.577+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f2bf40d0740 con 0x7f2c0c0755a0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.577+0000 7f2c0b400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2bf00a3cd0 0x7f2bf00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.578+0000 7f2c0b400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2bf00a3cd0 0x7f2bf00a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0098e0 tx=0x7f2bfc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:13.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.582+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2bf409cfc0 con 0x7f2c0c0755a0 2024-04-20T11:21:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.828+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2c0c1a8220 con 0x7f2c0c0755a0 2024-04-20T11:21:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.829+0000 7f2c08c00700 1 -- 172.21.15.44:0/1558842003 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2bf409c8c0 con 0x7f2c0c0755a0 2024-04-20T11:21:13.829 INFO:teuthology.orchestra.run.smithi044.stdout:141733920838 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2bf00a3cd0 msgr2=0x7f2bf00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2bf00a3cd0 0x7f2bf00a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0098e0 tx=0x7f2bfc005960 comp rx=0 tx=0).stop 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 msgr2=0x7f2c0c19b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c19b9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e970 tx=0x7f2bf400ec80 comp rx=0 tx=0).stop 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 shutdown_connections 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2bf00a3cd0 0x7f2bf00a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c0c10f450 0x7f2c0c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c0c074c80 0x7f2c0c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 --2- 172.21.15.44:0/1558842003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c0c0755a0 0x7f2c0c19b9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 >> 172.21.15.44:0/1558842003 conn(0x7f2c0c079bb0 msgr2=0x7f2c0c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.832+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 shutdown_connections 2024-04-20T11:21:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:13.833+0000 7f2c12400700 1 -- 172.21.15.44:0/1558842003 wait complete. 2024-04-20T11:21:14.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:14 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1558842003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T11:21:14.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:14 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1558842003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T11:21:14.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:14 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1558842003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T11:21:14.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 -- 172.21.15.44:0/3355810739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038108f90 msgr2=0x7f3038109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 --2- 172.21.15.44:0/3355810739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038108f90 0x7f3038109410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3024009990 tx=0x7f3024009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 -- 172.21.15.44:0/3355810739 shutdown_connections 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 --2- 172.21.15.44:0/3355810739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038109950 0x7f3038115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 --2- 172.21.15.44:0/3355810739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 0x7f303810f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 --2- 172.21.15.44:0/3355810739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038108f90 0x7f3038109410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.497+0000 7f303d400700 1 -- 172.21.15.44:0/3355810739 >> 172.21.15.44:0/3355810739 conn(0x7f30380787e0 msgr2=0x7f3038107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.498+0000 7f303d400700 1 -- 172.21.15.44:0/3355810739 shutdown_connections 2024-04-20T11:21:14.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.498+0000 7f303d400700 1 -- 172.21.15.44:0/3355810739 wait complete. 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 Processor -- start 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 -- start start 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 0x7f303819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038109950 0x7f303819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 0x7f30381a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30381193e0 con 0x7f303810f040 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3038119260 con 0x7f3038109950 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.499+0000 7f303d400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3038119560 con 0x7f3038108f90 2024-04-20T11:21:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f303ca00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 0x7f30381a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 0x7f303819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 0x7f303819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58558/0 (socket says 172.21.15.44:58558) 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037e00700 1 -- 172.21.15.44:0/351268674 learned_addr learned my addr 172.21.15.44:0/351268674 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038109950 0x7f303819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037e00700 1 -- 172.21.15.44:0/351268674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038109950 msgr2=0x7f303819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037e00700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038109950 0x7f303819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.500+0000 7f3037e00700 1 -- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 msgr2=0x7f30381a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f3037e00700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 0x7f30381a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f3037e00700 1 -- 172.21.15.44:0/351268674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30381a4750 con 0x7f3038108f90 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f303ca00700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 0x7f30381a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f3037e00700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 0x7f303819f790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f302800e9a0 tx=0x7f302800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302800ca10 con 0x7f3038108f90 2024-04-20T11:21:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f302800cbb0 con 0x7f3038108f90 2024-04-20T11:21:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30381a4a40 con 0x7f3038108f90 2024-04-20T11:21:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028018710 con 0x7f3038108f90 2024-04-20T11:21:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.501+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30381ac2c0 con 0x7f3038108f90 2024-04-20T11:21:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.502+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3038111390 con 0x7f3038108f90 2024-04-20T11:21:14.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.506+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3028010430 con 0x7f3038108f90 2024-04-20T11:21:14.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.508+0000 7f3034c00700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f301c0a3cd0 0x7f301c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.508+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f30280d0b30 con 0x7f3038108f90 2024-04-20T11:21:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.508+0000 7f3037400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f301c0a3cd0 0x7f301c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.508+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f302809d3b0 con 0x7f3038108f90 2024-04-20T11:21:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.509+0000 7f3037400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f301c0a3cd0 0x7f301c0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f303810e760 tx=0x7f302400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.801+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f30381a0aa0 con 0x7f3038108f90 2024-04-20T11:21:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.802+0000 7f3034c00700 1 -- 172.21.15.44:0/351268674 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f302809ccb0 con 0x7f3038108f90 2024-04-20T11:21:14.801 INFO:teuthology.orchestra.run.smithi044.stdout:219043332148 2024-04-20T11:21:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f301c0a3cd0 msgr2=0x7f301c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f301c0a3cd0 0x7f301c0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f303810e760 tx=0x7f302400b260 comp rx=0 tx=0).stop 2024-04-20T11:21:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 msgr2=0x7f303819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 0x7f303819f790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f302800e9a0 tx=0x7f302800ecb0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 shutdown_connections 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f301c0a3cd0 0x7f301c0a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f303810f040 0x7f30381a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3038108f90 0x7f303819f790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 --2- 172.21.15.44:0/351268674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3038109950 0x7f303819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 >> 172.21.15.44:0/351268674 conn(0x7f30380787e0 msgr2=0x7f3038079960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 shutdown_connections 2024-04-20T11:21:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.806+0000 7f303d400700 1 -- 172.21.15.44:0/351268674 wait complete. 2024-04-20T11:21:14.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.913+0000 7f5faca00700 1 -- 172.21.15.44:0/1455190896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 msgr2=0x7f5fa8108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.913+0000 7f5faca00700 1 --2- 172.21.15.44:0/1455190896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa8108e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009990 tx=0x7f5f98009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 -- 172.21.15.44:0/1455190896 shutdown_connections 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 --2- 172.21.15.44:0/1455190896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806b100 0x7f5fa810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 --2- 172.21.15.44:0/1455190896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa8108e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 --2- 172.21.15.44:0/1455190896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa806a760 0x7f5fa806abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 -- 172.21.15.44:0/1455190896 >> 172.21.15.44:0/1455190896 conn(0x7f5fa8078900 msgr2=0x7f5fa8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 -- 172.21.15.44:0/1455190896 shutdown_connections 2024-04-20T11:21:14.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.914+0000 7f5faca00700 1 -- 172.21.15.44:0/1455190896 wait complete. 2024-04-20T11:21:14.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.915+0000 7f5faca00700 1 Processor -- start 2024-04-20T11:21:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.915+0000 7f5faca00700 1 -- start start 2024-04-20T11:21:14.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5faca00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 0x7f5fa819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5faca00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa806b100 0x7f5fa819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5faca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5faca00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8112f40 con 0x7f5fa806b100 2024-04-20T11:21:14.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5faca00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8112dc0 con 0x7f5fa806a760 2024-04-20T11:21:14.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5faca00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81130c0 con 0x7f5fa8108a40 2024-04-20T11:21:14.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5fa5e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 0x7f5fa819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5fa5e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 0x7f5fa819b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:47946/0 (socket says 172.21.15.44:47946) 2024-04-20T11:21:14.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5fa5e00700 1 -- 172.21.15.44:0/2205274991 learned_addr learned my addr 172.21.15.44:0/2205274991 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:14.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5fa6800700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.916+0000 7f5fa5400700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa806b100 0x7f5fa819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa5e00700 1 -- 172.21.15.44:0/2205274991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 msgr2=0x7f5fa819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa5e00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa5e00700 1 -- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa806b100 msgr2=0x7f5fa819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:14.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa5e00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa806b100 0x7f5fa819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:14.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa5e00700 1 -- 172.21.15.44:0/2205274991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa81a0510 con 0x7f5fa806a760 2024-04-20T11:21:14.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa6800700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:14.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5fa5e00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 0x7f5fa819b550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f98005910 tx=0x7f5f980047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:14.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98027070 con 0x7f5fa806a760 2024-04-20T11:21:14.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa81a07a0 con 0x7f5fa806a760 2024-04-20T11:21:14.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.917+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f9800fe00 con 0x7f5fa806a760 2024-04-20T11:21:14.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.918+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa81a8080 con 0x7f5fa806a760 2024-04-20T11:21:14.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.918+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98017650 con 0x7f5fa806a760 2024-04-20T11:21:14.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.919+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f88005330 con 0x7f5fa806a760 2024-04-20T11:21:14.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.921+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f9802a070 con 0x7f5fa806a760 2024-04-20T11:21:14.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.923+0000 7f5f9ea00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5f8c0a3dd0 0x7f5f8c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:14.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.923+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f5f980d13b0 con 0x7f5fa806a760 2024-04-20T11:21:14.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.923+0000 7f5fa5400700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5f8c0a3dd0 0x7f5f8c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:14.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.924+0000 7f5fa5400700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5f8c0a3dd0 0x7f5f8c0a6290 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8103a40 tx=0x7f5f94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:14.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.925+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5f98013070 con 0x7f5fa806a760 2024-04-20T11:21:15.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.188+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5f880058f0 con 0x7f5fa806a760 2024-04-20T11:21:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.189+0000 7f5f9ea00700 1 -- 172.21.15.44:0/2205274991 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f9809dcc0 con 0x7f5fa806a760 2024-04-20T11:21:15.188 INFO:teuthology.orchestra.run.smithi044.stdout:77309411411 2024-04-20T11:21:15.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5f8c0a3dd0 msgr2=0x7f5f8c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5f8c0a3dd0 0x7f5f8c0a6290 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8103a40 tx=0x7f5f94009260 comp rx=0 tx=0).stop 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 msgr2=0x7f5fa819b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 0x7f5fa819b550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f98005910 tx=0x7f5f980047b0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 shutdown_connections 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5f8c0a3dd0 0x7f5f8c0a6290 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fa8108a40 0x7f5fa819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fa806a760 0x7f5fa819b550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 --2- 172.21.15.44:0/2205274991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fa806b100 0x7f5fa819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 >> 172.21.15.44:0/2205274991 conn(0x7f5fa8078900 msgr2=0x7f5fa81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 shutdown_connections 2024-04-20T11:21:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.192+0000 7f5faca00700 1 -- 172.21.15.44:0/2205274991 wait complete. 2024-04-20T11:21:15.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:15 smithi078 ceph-mon[21464]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:15.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:15 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/351268674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T11:21:15.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:15 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3843702141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T11:21:15.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:15 smithi044 ceph-mon[16855]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:15.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:15 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/351268674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T11:21:15.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:15 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3843702141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T11:21:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:15 smithi165 ceph-mon[21975]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:15 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/351268674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T11:21:15.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:15 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3843702141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T11:21:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.769+0000 7f6c7b000700 1 -- 172.21.15.44:0/1529017833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74103460 msgr2=0x7f6c7410f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.769+0000 7f6c7b000700 1 --2- 172.21.15.44:0/1529017833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74103460 0x7f6c7410f8c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009f90 tx=0x7f6c6c00b330 comp rx=0 tx=0).stop 2024-04-20T11:21:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 -- 172.21.15.44:0/1529017833 shutdown_connections 2024-04-20T11:21:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 --2- 172.21.15.44:0/1529017833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74103460 0x7f6c7410f8c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 --2- 172.21.15.44:0/1529017833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74108a60 0x7f6c74108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 --2- 172.21.15.44:0/1529017833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74102aa0 0x7f6c74102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 -- 172.21.15.44:0/1529017833 >> 172.21.15.44:0/1529017833 conn(0x7f6c74079bb0 msgr2=0x7f6c74079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 -- 172.21.15.44:0/1529017833 shutdown_connections 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.770+0000 7f6c7b000700 1 -- 172.21.15.44:0/1529017833 wait complete. 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.771+0000 7f6c7b000700 1 Processor -- start 2024-04-20T11:21:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.771+0000 7f6c7b000700 1 -- start start 2024-04-20T11:21:15.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c7b000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 0x7f6c741992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:15.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c7b000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74103460 0x7f6c741997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:15.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c7b000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74108a60 0x7f6c7419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:15.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c7b000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74112ef0 con 0x7f6c74108a60 2024-04-20T11:21:15.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c7b000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74112d70 con 0x7f6c74102aa0 2024-04-20T11:21:15.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c7b000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74113070 con 0x7f6c74103460 2024-04-20T11:21:15.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c73e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 0x7f6c741992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:15.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c73e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 0x7f6c741992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:47926/0 (socket says 172.21.15.44:47926) 2024-04-20T11:21:15.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c73e00700 1 -- 172.21.15.44:0/3843702141 learned_addr learned my addr 172.21.15.44:0/3843702141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:15.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c79000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74108a60 0x7f6c7419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:15.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.772+0000 7f6c73400700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74103460 0x7f6c741997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:15.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c73e00700 1 -- 172.21.15.44:0/3843702141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74103460 msgr2=0x7f6c741997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c73e00700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74103460 0x7f6c741997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c73e00700 1 -- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74108a60 msgr2=0x7f6c7419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c73e00700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74108a60 0x7f6c7419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c73e00700 1 -- 172.21.15.44:0/3843702141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7419e2b0 con 0x7f6c74102aa0 2024-04-20T11:21:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c73e00700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 0x7f6c741992a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e9c0 tx=0x7f6c6400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004a30 con 0x7f6c74102aa0 2024-04-20T11:21:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6c6400ccc0 con 0x7f6c74102aa0 2024-04-20T11:21:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7419e5a0 con 0x7f6c74102aa0 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.773+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64005650 con 0x7f6c74102aa0 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.774+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a5e20 con 0x7f6c74102aa0 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.775+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74069740 con 0x7f6c74102aa0 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.777+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6c64004d60 con 0x7f6c74102aa0 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.778+0000 7f6c70c00700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6c580a3cd0 0x7f6c580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.778+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f6c640d0740 con 0x7f6c74102aa0 2024-04-20T11:21:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.779+0000 7f6c73400700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6c580a3cd0 0x7f6c580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.780+0000 7f6c73400700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6c580a3cd0 0x7f6c580a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c7407a140 tx=0x7f6c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:14.780+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6c6409cfc0 con 0x7f6c74102aa0 2024-04-20T11:21:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.056+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6c7402df90 con 0x7f6c74102aa0 2024-04-20T11:21:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.057+0000 7f6c70c00700 1 -- 172.21.15.44:0/3843702141 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6c6409c8c0 con 0x7f6c74102aa0 2024-04-20T11:21:15.644 INFO:teuthology.orchestra.run.smithi044.stdout:55834574935 2024-04-20T11:21:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6c580a3cd0 msgr2=0x7f6c580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6c580a3cd0 0x7f6c580a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c7407a140 tx=0x7f6c60009260 comp rx=0 tx=0).stop 2024-04-20T11:21:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 msgr2=0x7f6c741992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 0x7f6c741992a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e9c0 tx=0x7f6c6400ecd0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 shutdown_connections 2024-04-20T11:21:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6c580a3cd0 0x7f6c580a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c74108a60 0x7f6c7419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c74102aa0 0x7f6c741992a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 --2- 172.21.15.44:0/3843702141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74103460 0x7f6c741997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:15.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.061+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 >> 172.21.15.44:0/3843702141 conn(0x7f6c74079bb0 msgr2=0x7f6c74101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:15.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.062+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 shutdown_connections 2024-04-20T11:21:15.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:15.062+0000 7f6c7b000700 1 -- 172.21.15.44:0/3843702141 wait complete. 2024-04-20T11:21:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 -- 172.21.15.44:0/2322866630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c103b90 msgr2=0x7f796c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 --2- 172.21.15.44:0/2322866630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c103b90 0x7f796c103f70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7960009990 tx=0x7f7960009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 -- 172.21.15.44:0/2322866630 shutdown_connections 2024-04-20T11:21:16.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 --2- 172.21.15.44:0/2322866630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c10cfc0 0x7f796c10f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 --2- 172.21.15.44:0/2322866630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c103b90 0x7f796c103f70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 --2- 172.21.15.44:0/2322866630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c1044b0 0x7f796c10c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.161+0000 7f7972c00700 1 -- 172.21.15.44:0/2322866630 >> 172.21.15.44:0/2322866630 conn(0x7f796c0fd3e0 msgr2=0x7f796c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.162+0000 7f7972c00700 1 -- 172.21.15.44:0/2322866630 shutdown_connections 2024-04-20T11:21:16.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.162+0000 7f7972c00700 1 -- 172.21.15.44:0/2322866630 wait complete. 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 Processor -- start 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 -- start start 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 0x7f796c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c1044b0 0x7f796c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c10cfc0 0x7f796c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f796c112ed0 con 0x7f796c103b90 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f796c112d50 con 0x7f796c10cfc0 2024-04-20T11:21:16.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7972c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f796c113050 con 0x7f796c1044b0 2024-04-20T11:21:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.163+0000 7f7971800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 0x7f796c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7971800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 0x7f796c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42268/0 (socket says 172.21.15.44:42268) 2024-04-20T11:21:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7971800700 1 -- 172.21.15.44:0/4007269753 learned_addr learned my addr 172.21.15.44:0/4007269753 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c1044b0 0x7f796c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:16.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7972200700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c10cfc0 0x7f796c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 -- 172.21.15.44:0/4007269753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c10cfc0 msgr2=0x7f796c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c10cfc0 0x7f796c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 -- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 msgr2=0x7f796c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 0x7f796c199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 -- 172.21.15.44:0/4007269753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c19e2e0 con 0x7f796c1044b0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7971800700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 0x7f796c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.164+0000 7f7970e00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c1044b0 0x7f796c199860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f795800e9c0 tx=0x7f795800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.165+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795800c9c0 con 0x7f796c1044b0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.165+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c19e5d0 con 0x7f796c1044b0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.165+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f795800cb60 con 0x7f796c1044b0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.165+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958018680 con 0x7f796c1044b0 2024-04-20T11:21:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.165+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c1a5e50 con 0x7f796c1044b0 2024-04-20T11:21:16.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.166+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c069740 con 0x7f796c1044b0 2024-04-20T11:21:16.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.168+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7958010430 con 0x7f796c1044b0 2024-04-20T11:21:16.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.169+0000 7f796a000700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:16.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.170+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f79580d0650 con 0x7f796c1044b0 2024-04-20T11:21:16.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.170+0000 7f7971800700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:16.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.170+0000 7f7971800700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7960009750 tx=0x7f796000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:16.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.174+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f795809ced0 con 0x7f796c1044b0 2024-04-20T11:21:16.340 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:16 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2205274991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T11:21:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.415+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f796c02df90 con 0x7f796c1044b0 2024-04-20T11:21:16.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.415+0000 7f796a000700 1 -- 172.21.15.44:0/4007269753 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f795809c7d0 con 0x7f796c1044b0 2024-04-20T11:21:16.415 INFO:teuthology.orchestra.run.smithi044.stdout:197568495674 2024-04-20T11:21:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.418+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 msgr2=0x7f79540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.418+0000 7f7972c00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7960009750 tx=0x7f796000b280 comp rx=0 tx=0).stop 2024-04-20T11:21:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.418+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c1044b0 msgr2=0x7f796c199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.418+0000 7f7972c00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c1044b0 0x7f796c199860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f795800e9c0 tx=0x7f795800ecd0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 shutdown_connections 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f79540a3cd0 0x7f79540a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f796c10cfc0 0x7f796c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796c103b90 0x7f796c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 --2- 172.21.15.44:0/4007269753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c1044b0 0x7f796c199860 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 >> 172.21.15.44:0/4007269753 conn(0x7f796c0fd3e0 msgr2=0x7f796c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 shutdown_connections 2024-04-20T11:21:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:16.419+0000 7f7972c00700 1 -- 172.21.15.44:0/4007269753 wait complete. 2024-04-20T11:21:16.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:16 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2205274991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T11:21:16.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:16 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2205274991' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T11:21:17.216 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:17 smithi044 ceph-mon[16855]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:17.216 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4007269753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T11:21:17.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:17 smithi078 ceph-mon[21464]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:17.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4007269753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T11:21:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:17 smithi165 ceph-mon[21975]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:17.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4007269753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T11:21:18.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332144 got 219043332148 for osd.7 2024-04-20T11:21:18.763 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:19.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:19 smithi078 ceph-mon[21464]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:19.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.210+0000 7fb3b1000700 1 -- 172.21.15.44:0/4210166863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 msgr2=0x7fb3ac1052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:19.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.210+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4210166863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac1052d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0099c0 tx=0x7fb39c009cd0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 -- 172.21.15.44:0/4210166863 shutdown_connections 2024-04-20T11:21:19.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4210166863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3ac109fd0 0x7fb3ac111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4210166863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac1052d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4210166863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 0x7fb3ac1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 -- 172.21.15.44:0/4210166863 >> 172.21.15.44:0/4210166863 conn(0x7fb3ac100780 msgr2=0x7fb3ac102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 -- 172.21.15.44:0/4210166863 shutdown_connections 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.211+0000 7fb3b1000700 1 -- 172.21.15.44:0/4210166863 wait complete. 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.212+0000 7fb3b1000700 1 Processor -- start 2024-04-20T11:21:19.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.212+0000 7fb3b1000700 1 -- start start 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.212+0000 7fb3b1000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.212+0000 7fb3b1000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 0x7fb3ac19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3b1000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3ac109fd0 0x7fb3ac19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3b1000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1150c0 con 0x7fb3ac104ef0 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3b1000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac114f40 con 0x7fb3ac1058a0 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3b1000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac115240 con 0x7fb3ac109fd0 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac19b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56768/0 (socket says 172.21.15.44:56768) 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 -- 172.21.15.44:0/4272517575 learned_addr learned my addr 172.21.15.44:0/4272517575 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9400700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 0x7fb3ac19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:19.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3aa800700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3ac109fd0 0x7fb3ac19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 -- 172.21.15.44:0/4272517575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3ac109fd0 msgr2=0x7fb3ac19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3ac109fd0 0x7fb3ac19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 -- 172.21.15.44:0/4272517575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 msgr2=0x7fb3ac19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 0x7fb3ac19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9e00700 1 -- 172.21.15.44:0/4272517575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac1a04f0 con 0x7fb3ac104ef0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.213+0000 7fb3a9400700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 0x7fb3ac19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.214+0000 7fb3a9e00700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac19b530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006010 tx=0x7fb39c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.214+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c009e60 con 0x7fb3ac104ef0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.214+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb39c00be40 con 0x7fb3ac104ef0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.214+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac1a0780 con 0x7fb3ac104ef0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.214+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c003e60 con 0x7fb3ac104ef0 2024-04-20T11:21:19.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.214+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac1a8060 con 0x7fb3ac104ef0 2024-04-20T11:21:19.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.215+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac069740 con 0x7fb3ac104ef0 2024-04-20T11:21:19.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.218+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb39c00f460 con 0x7fb3ac104ef0 2024-04-20T11:21:19.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.219+0000 7fb3a2a00700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3900a3d50 0x7fb3900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:19.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.219+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fb39c0d1260 con 0x7fb3ac104ef0 2024-04-20T11:21:19.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.219+0000 7fb3a9400700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3900a3d50 0x7fb3900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:19.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.220+0000 7fb3a9400700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3900a3d50 0x7fb3900a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac10b520 tx=0x7fb398009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:19.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.222+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb39c013070 con 0x7fb3ac104ef0 2024-04-20T11:21:19.303 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:19 smithi044 ceph-mon[16855]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:19.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574935 for osd.1 2024-04-20T11:21:19.365 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.472+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb3ac02df90 con 0x7fb3ac104ef0 2024-04-20T11:21:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.473+0000 7fb3a2a00700 1 -- 172.21.15.44:0/4272517575 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb39c09db70 con 0x7fb3ac104ef0 2024-04-20T11:21:19.473 INFO:teuthology.orchestra.run.smithi044.stdout:309237645346 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3900a3d50 msgr2=0x7fb3900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3900a3d50 0x7fb3900a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac10b520 tx=0x7fb398009260 comp rx=0 tx=0).stop 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 msgr2=0x7fb3ac19b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac19b530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006010 tx=0x7fb39c003c30 comp rx=0 tx=0).stop 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 shutdown_connections 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb3900a3d50 0x7fb3900a6210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3ac109fd0 0x7fb3ac19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac104ef0 0x7fb3ac19b530 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 --2- 172.21.15.44:0/4272517575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3ac1058a0 0x7fb3ac19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:19.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 >> 172.21.15.44:0/4272517575 conn(0x7fb3ac100780 msgr2=0x7fb3ac102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:19.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 shutdown_connections 2024-04-20T11:21:19.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:19.476+0000 7fb3b1000700 1 -- 172.21.15.44:0/4272517575 wait complete. 2024-04-20T11:21:19.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:19 smithi165 ceph-mon[21975]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:20.389 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:20 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4272517575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T11:21:20.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:20 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4272517575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T11:21:20.447 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411411 for osd.2 2024-04-20T11:21:20.447 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:20.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:20 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4272517575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T11:21:21.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 -- 172.21.15.44:0/1282565783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 msgr2=0x7f9988108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 --2- 172.21.15.44:0/1282565783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 0x7f9988108e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9978009990 tx=0x7f9978009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 -- 172.21.15.44:0/1282565783 shutdown_connections 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 --2- 172.21.15.44:0/1282565783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806b100 0x7f998810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 --2- 172.21.15.44:0/1282565783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 0x7f9988108e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 --2- 172.21.15.44:0/1282565783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998806a760 0x7f998806abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.050+0000 7f998ee00700 1 -- 172.21.15.44:0/1282565783 >> 172.21.15.44:0/1282565783 conn(0x7f9988078900 msgr2=0x7f9988078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.051+0000 7f998ee00700 1 -- 172.21.15.44:0/1282565783 shutdown_connections 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.051+0000 7f998ee00700 1 -- 172.21.15.44:0/1282565783 wait complete. 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.052+0000 7f998ee00700 1 Processor -- start 2024-04-20T11:21:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.052+0000 7f998ee00700 1 -- start start 2024-04-20T11:21:21.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ee00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 0x7f9988199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ee00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998806b100 0x7f9988199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ee00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 0x7f998819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ee00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9988112eb0 con 0x7f9988108a40 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ee00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9988112d30 con 0x7f998806a760 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ee00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9988113030 con 0x7f998806b100 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f9987e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 0x7f9988199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.053+0000 7f998ce00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 0x7f998819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 0x7f9988199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60008/0 (socket says 172.21.15.44:60008) 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 -- 172.21.15.44:0/4031193777 learned_addr learned my addr 172.21.15.44:0/4031193777 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987400700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998806b100 0x7f9988199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 -- 172.21.15.44:0/4031193777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998806b100 msgr2=0x7f9988199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998806b100 0x7f9988199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 -- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 msgr2=0x7f998819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 0x7f998819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 -- 172.21.15.44:0/4031193777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998819e310 con 0x7f998806a760 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.054+0000 7f9987e00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 0x7f9988199300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9978000c00 tx=0x7f997800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.055+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978027070 con 0x7f998806a760 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.055+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9978022470 con 0x7f998806a760 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.055+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978017630 con 0x7f998806a760 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.055+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998819e5a0 con 0x7f998806a760 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.055+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881a5e80 con 0x7f998806a760 2024-04-20T11:21:21.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.056+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9988104180 con 0x7f998806a760 2024-04-20T11:21:21.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.058+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f997802a070 con 0x7f998806a760 2024-04-20T11:21:21.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.060+0000 7f9984c00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f996c0a3cd0 0x7f996c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.060+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f99780d1af0 con 0x7f998806a760 2024-04-20T11:21:21.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.060+0000 7f9987400700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f996c0a3cd0 0x7f996c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.061+0000 7f9987400700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f996c0a3cd0 0x7f996c0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9988078f00 tx=0x7f9974009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:21.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.062+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9978031070 con 0x7f998806a760 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 -- 172.21.15.44:0/3759666845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb420108a40 msgr2=0x7fb420108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 --2- 172.21.15.44:0/3759666845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb420108a40 0x7fb420108e20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb410009990 tx=0x7fb410009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 -- 172.21.15.44:0/3759666845 shutdown_connections 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 --2- 172.21.15.44:0/3759666845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42006b100 0x7fb42010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 --2- 172.21.15.44:0/3759666845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb420108a40 0x7fb420108e20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 --2- 172.21.15.44:0/3759666845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 0x7fb42006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.135+0000 7fb424e00700 1 -- 172.21.15.44:0/3759666845 >> 172.21.15.44:0/3759666845 conn(0x7fb420078900 msgr2=0x7fb420078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.136+0000 7fb424e00700 1 -- 172.21.15.44:0/3759666845 shutdown_connections 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.136+0000 7fb424e00700 1 -- 172.21.15.44:0/3759666845 wait complete. 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.137+0000 7fb424e00700 1 Processor -- start 2024-04-20T11:21:21.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.137+0000 7fb424e00700 1 -- start start 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb424e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 0x7fb42019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb424e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb42006b100 0x7fb42019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb424e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 0x7fb42019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb424e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb420112f00 con 0x7fb420108a40 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb424e00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb420112d80 con 0x7fb42006a760 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb424e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb420113080 con 0x7fb42006b100 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb41e800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 0x7fb42019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb41de00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 0x7fb42019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb41e800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 0x7fb42019fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56806/0 (socket says 172.21.15.44:56806) 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb41e800700 1 -- 172.21.15.44:0/967549853 learned_addr learned my addr 172.21.15.44:0/967549853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.138+0000 7fb41d400700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb42006b100 0x7fb42019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41de00700 1 -- 172.21.15.44:0/967549853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb42006b100 msgr2=0x7fb42019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41de00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb42006b100 0x7fb42019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41de00700 1 -- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 msgr2=0x7fb42019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41de00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 0x7fb42019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41de00700 1 -- 172.21.15.44:0/967549853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4201a0570 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41e800700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 0x7fb42019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.139+0000 7fb41de00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 0x7fb42019b5b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb41000b2e0 tx=0x7fb4100047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.140+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410026070 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.140+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb41000fe00 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.140+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4201a0800 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.140+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100173f0 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.140+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4201a80e0 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.142+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb420104180 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.143+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb410029070 con 0x7fb42006a760 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.144+0000 7fb416a00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb4040a3cd0 0x7fb4040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:21.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.145+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fb4100d1220 con 0x7fb42006a760 2024-04-20T11:21:21.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.145+0000 7fb41d400700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb4040a3cd0 0x7fb4040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:21.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.146+0000 7fb41d400700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb4040a3cd0 0x7fb4040a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009680 tx=0x7fb40c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:21.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.148+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb41005dff0 con 0x7fb42006a760 2024-04-20T11:21:21.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.196+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb410009d10 con 0x7fb42006a760 2024-04-20T11:21:21.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.197+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9978009d10 con 0x7f998806a760 2024-04-20T11:21:21.325 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:21 smithi044 ceph-mon[16855]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:21.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.344+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f998802df90 con 0x7f998806a760 2024-04-20T11:21:21.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.345+0000 7f9984c00700 1 -- 172.21.15.44:0/4031193777 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9978013070 con 0x7f998806a760 2024-04-20T11:21:21.345 INFO:teuthology.orchestra.run.smithi044.stdout:343597383709 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f996c0a3cd0 msgr2=0x7f996c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f996c0a3cd0 0x7f996c0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9988078f00 tx=0x7f9974009260 comp rx=0 tx=0).stop 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 msgr2=0x7f9988199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 0x7f9988199300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9978000c00 tx=0x7f997800b610 comp rx=0 tx=0).stop 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 shutdown_connections 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f996c0a3cd0 0x7f996c0a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9988108a40 0x7f998819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f998806a760 0x7f9988199300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 --2- 172.21.15.44:0/4031193777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998806b100 0x7f9988199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 >> 172.21.15.44:0/4031193777 conn(0x7f9988078900 msgr2=0x7f9988100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 shutdown_connections 2024-04-20T11:21:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.348+0000 7f998ee00700 1 -- 172.21.15.44:0/4031193777 wait complete. 2024-04-20T11:21:21.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.418+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb42019c8c0 con 0x7fb42006a760 2024-04-20T11:21:21.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.418+0000 7fb416a00700 1 -- 172.21.15.44:0/967549853 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb410013070 con 0x7fb42006a760 2024-04-20T11:21:21.418 INFO:teuthology.orchestra.run.smithi044.stdout:107374182476 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb4040a3cd0 msgr2=0x7fb4040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb4040a3cd0 0x7fb4040a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009680 tx=0x7fb40c009260 comp rx=0 tx=0).stop 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 msgr2=0x7fb42019b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 0x7fb42019b5b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb41000b2e0 tx=0x7fb4100047b0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 shutdown_connections 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fb4040a3cd0 0x7fb4040a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420108a40 0x7fb42019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42006a760 0x7fb42019b5b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 --2- 172.21.15.44:0/967549853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb42006b100 0x7fb42019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:21.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.421+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 >> 172.21.15.44:0/967549853 conn(0x7fb420078900 msgr2=0x7fb4200fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:21.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.422+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 shutdown_connections 2024-04-20T11:21:21.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.422+0000 7fb424e00700 1 -- 172.21.15.44:0/967549853 wait complete. 2024-04-20T11:21:21.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:21 smithi078 ceph-mon[21464]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:21.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:21 smithi165 ceph-mon[21975]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:22.243 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:22 smithi044 ceph-mon[16855]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T11:21:22.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:22 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4031193777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T11:21:22.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:22 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/967549853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T11:21:22.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:22 smithi165 ceph-mon[21975]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T11:21:22.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:22 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4031193777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T11:21:22.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:22 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/967549853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T11:21:22.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:22 smithi078 ceph-mon[21464]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T11:21:22.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:22 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4031193777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T11:21:22.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:22 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/967549853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T11:21:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 -- 172.21.15.44:0/2945684110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 msgr2=0x7fe54c077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 --2- 172.21.15.44:0/2945684110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c077360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009990 tx=0x7fe53c009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 -- 172.21.15.44:0/2945684110 shutdown_connections 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 --2- 172.21.15.44:0/2945684110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c113800 0x7fe54c115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 --2- 172.21.15.44:0/2945684110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c077360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 --2- 172.21.15.44:0/2945684110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c0778a0 0x7fe54c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 -- 172.21.15.44:0/2945684110 >> 172.21.15.44:0/2945684110 conn(0x7fe54c100630 msgr2=0x7fe54c102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.986+0000 7fe550a00700 1 -- 172.21.15.44:0/2945684110 shutdown_connections 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.987+0000 7fe550a00700 1 -- 172.21.15.44:0/2945684110 wait complete. 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.988+0000 7fe550a00700 1 Processor -- start 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.988+0000 7fe550a00700 1 -- start start 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe550a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:22.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe550a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 0x7fe54c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe550a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c113800 0x7fe54c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe550a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c119480 con 0x7fe54c0778a0 2024-04-20T11:21:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe550a00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c119300 con 0x7fe54c076f80 2024-04-20T11:21:22.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe550a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c119600 con 0x7fe54c113800 2024-04-20T11:21:22.869 INFO:teuthology.orchestra.run.smithi044.stdout:163208757313 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe541e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 0x7fe54c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe541e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 0x7fe54c1a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56816/0 (socket says 172.21.15.44:56816) 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe541e00700 1 -- 172.21.15.44:0/4196345897 learned_addr learned my addr 172.21.15.44:0/4196345897 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54a800700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c1a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54b200700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c113800 0x7fe54c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54a800700 1 -- 172.21.15.44:0/4196345897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c113800 msgr2=0x7fe54c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54a800700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c113800 0x7fe54c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54a800700 1 -- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 msgr2=0x7fe54c1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54a800700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 0x7fe54c1a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54a800700 1 -- 172.21.15.44:0/4196345897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19ffa0 con 0x7fe54c076f80 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe541e00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 0x7fe54c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:22.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.989+0000 7fe54b200700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c113800 0x7fe54c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.990+0000 7fe54a800700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c1a55f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0098e0 tx=0x7fe53c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.990+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c027070 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.990+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe53c00fe00 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.990+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c1a0230 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.990+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c017650 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.990+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1ac3a0 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.991+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c0786c0 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.993+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe53c02a070 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.995+0000 7fe543e00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5300a3c40 0x7fe5300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.995+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fe53c0d13c0 con 0x7fe54c076f80 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.995+0000 7fe541e00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5300a3c40 0x7fe5300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.996+0000 7fe541e00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5300a3c40 0x7fe5300a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe54c1a0e80 tx=0x7fe534009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:21.999+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe53c05eff0 con 0x7fe54c076f80 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.243+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe54c02df90 con 0x7fe54c076f80 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.244+0000 7fe543e00700 1 -- 172.21.15.44:0/4196345897 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe53c013070 con 0x7fe54c076f80 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5300a3c40 msgr2=0x7fe5300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5300a3c40 0x7fe5300a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe54c1a0e80 tx=0x7fe534009260 comp rx=0 tx=0).stop 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 msgr2=0x7fe54c1a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c1a55f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0098e0 tx=0x7fe53c00b840 comp rx=0 tx=0).stop 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 shutdown_connections 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5300a3c40 0x7fe5300a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe54c113800 0x7fe54c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54c076f80 0x7fe54c1a55f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 --2- 172.21.15.44:0/4196345897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe54c0778a0 0x7fe54c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 >> 172.21.15.44:0/4196345897 conn(0x7fe54c100630 msgr2=0x7fe54c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 shutdown_connections 2024-04-20T11:21:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:22.247+0000 7fe550a00700 1 -- 172.21.15.44:0/4196345897 wait complete. 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.137+0000 7f38d8a00700 1 -- 172.21.15.44:0/1312331581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4104dc0 msgr2=0x7f38d41051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.137+0000 7f38d8a00700 1 --2- 172.21.15.44:0/1312331581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4104dc0 0x7f38d41051a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f38c40099c0 tx=0x7f38c4009cd0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.137+0000 7f38d8a00700 1 -- 172.21.15.44:0/1312331581 shutdown_connections 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.137+0000 7f38d8a00700 1 --2- 172.21.15.44:0/1312331581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4109ea0 0x7f38d4111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.137+0000 7f38d8a00700 1 --2- 172.21.15.44:0/1312331581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4104dc0 0x7f38d41051a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.138+0000 7f38d8a00700 1 --2- 172.21.15.44:0/1312331581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 0x7f38d41097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.138+0000 7f38d8a00700 1 -- 172.21.15.44:0/1312331581 >> 172.21.15.44:0/1312331581 conn(0x7f38d4100630 msgr2=0x7f38d4102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.138+0000 7f38d8a00700 1 -- 172.21.15.44:0/1312331581 shutdown_connections 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.138+0000 7f38d8a00700 1 -- 172.21.15.44:0/1312331581 wait complete. 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 Processor -- start 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 -- start start 2024-04-20T11:21:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4104dc0 0x7f38d419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 0x7f38d419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4109ea0 0x7f38d419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38d4115000 con 0x7f38d4109ea0 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38d4114e80 con 0x7f38d4104dc0 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.139+0000 7f38d8a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38d4115180 con 0x7f38d4105770 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38d3400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4104dc0 0x7f38d419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38d3e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4109ea0 0x7f38d419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 0x7f38d419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38d3400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4104dc0 0x7f38d419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60030/0 (socket says 172.21.15.44:60030) 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38d3400700 1 -- 172.21.15.44:0/3386606847 learned_addr learned my addr 172.21.15.44:0/3386606847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 -- 172.21.15.44:0/3386606847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4104dc0 msgr2=0x7f38d419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4104dc0 0x7f38d419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 -- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4109ea0 msgr2=0x7f38d419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:23.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4109ea0 0x7f38d419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 -- 172.21.15.44:0/3386606847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d41a0430 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.140+0000 7f38cbe00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 0x7f38d419b9b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00e9c0 tx=0x7f38bc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.141+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc004220 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.141+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f38bc004d30 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.141+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc005040 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.141+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d41a0720 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.141+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d41a7fa0 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.142+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d4069740 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.144+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38bc004750 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.145+0000 7f38d0c00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f38b80a3c40 0x7f38b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.146+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f38bc0d04c0 con 0x7f38d4105770 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.146+0000 7f38d3400700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f38b80a3c40 0x7f38b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.146+0000 7f38d3400700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f38b80a3c40 0x7f38b80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009990 tx=0x7f38c4005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:23.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.148+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f38bc09cea0 con 0x7f38d4105770 2024-04-20T11:21:23.339 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:23 smithi044 ceph-mon[16855]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-20T11:21:23.339 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:23 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4196345897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T11:21:23.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.389+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f38d419c780 con 0x7f38d4105770 2024-04-20T11:21:23.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.390+0000 7f38d0c00700 1 -- 172.21.15.44:0/3386606847 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f38bc09c7a0 con 0x7f38d4105770 2024-04-20T11:21:23.391 INFO:teuthology.orchestra.run.smithi044.stdout:287762808873 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f38b80a3c40 msgr2=0x7f38b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f38b80a3c40 0x7f38b80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009990 tx=0x7f38c4005ab0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 msgr2=0x7f38d419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 0x7f38d419b9b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00e9c0 tx=0x7f38bc00ecd0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 shutdown_connections 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f38b80a3c40 0x7f38b80a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d4109ea0 0x7f38d419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38d4104dc0 0x7f38d419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 --2- 172.21.15.44:0/3386606847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38d4105770 0x7f38d419b9b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 >> 172.21.15.44:0/3386606847 conn(0x7f38d4100630 msgr2=0x7f38d4102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.393+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 shutdown_connections 2024-04-20T11:21:23.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:23.394+0000 7f38d8a00700 1 -- 172.21.15.44:0/3386606847 wait complete. 2024-04-20T11:21:23.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:23 smithi165 ceph-mon[21975]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-20T11:21:23.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:23 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4196345897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T11:21:23.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:23 smithi078 ceph-mon[21464]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-20T11:21:23.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:23 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4196345897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T11:21:24.415 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:24 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3386606847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T11:21:24.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:24 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3386606847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T11:21:24.564 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920832 got 141733920838 for osd.4 2024-04-20T11:21:24.564 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 -- 172.21.15.44:0/615463315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c5006a760 msgr2=0x7f7c5006abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 --2- 172.21.15.44:0/615463315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c5006a760 0x7f7c5006abc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009990 tx=0x7f7c38009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 -- 172.21.15.44:0/615463315 shutdown_connections 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 --2- 172.21.15.44:0/615463315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c5006b100 0x7f7c5010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 --2- 172.21.15.44:0/615463315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c50108a40 0x7f7c50108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 --2- 172.21.15.44:0/615463315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c5006a760 0x7f7c5006abc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 -- 172.21.15.44:0/615463315 >> 172.21.15.44:0/615463315 conn(0x7f7c50078900 msgr2=0x7f7c50078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 -- 172.21.15.44:0/615463315 shutdown_connections 2024-04-20T11:21:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.617+0000 7f7c56c00700 1 -- 172.21.15.44:0/615463315 wait complete. 2024-04-20T11:21:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.619+0000 7f7c56c00700 1 Processor -- start 2024-04-20T11:21:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.619+0000 7f7c56c00700 1 -- start start 2024-04-20T11:21:24.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.619+0000 7f7c56c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 0x7f7c50103610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:24.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.619+0000 7f7c56c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c5006b100 0x7f7c50101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:24.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c56c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c50108a40 0x7f7c50102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:24.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c56c00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50112f40 con 0x7f7c50108a40 2024-04-20T11:21:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c56c00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50112dc0 con 0x7f7c5006b100 2024-04-20T11:21:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c56c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501130c0 con 0x7f7c5006a760 2024-04-20T11:21:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 0x7f7c50103610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 0x7f7c50103610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:52308/0 (socket says 172.21.15.44:52308) 2024-04-20T11:21:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 -- 172.21.15.44:0/3975687413 learned_addr learned my addr 172.21.15.44:0/3975687413 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c47400700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c5006b100 0x7f7c50101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:24.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c54c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c50108a40 0x7f7c50102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 -- 172.21.15.44:0/3975687413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c5006b100 msgr2=0x7f7c50101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c5006b100 0x7f7c50101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 -- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c50108a40 msgr2=0x7f7c50102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c50108a40 0x7f7c50102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c4fe00700 1 -- 172.21.15.44:0/3975687413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c501abd00 con 0x7f7c5006a760 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.620+0000 7f7c54c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c50108a40 0x7f7c50102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.621+0000 7f7c4fe00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 0x7f7c50103610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c370 tx=0x7f7c4000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.621+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000cea0 con 0x7f7c5006a760 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.621+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7c400090d0 con 0x7f7c5006a760 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.621+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c501abf30 con 0x7f7c5006a760 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.621+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40007500 con 0x7f7c5006a760 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.621+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501ac390 con 0x7f7c5006a760 2024-04-20T11:21:24.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.622+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c50104180 con 0x7f7c5006a760 2024-04-20T11:21:24.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.624+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c4001a020 con 0x7f7c5006a760 2024-04-20T11:21:24.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.625+0000 7f7c4d600700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7c340a3c40 0x7f7c340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:24.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.626+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f7c400cfe60 con 0x7f7c5006a760 2024-04-20T11:21:24.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.626+0000 7f7c47400700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7c340a3c40 0x7f7c340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:24.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.626+0000 7f7c47400700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7c340a3c40 0x7f7c340a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c50103220 tx=0x7f7c3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:24.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.628+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7c40010040 con 0x7f7c5006a760 2024-04-20T11:21:24.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:24 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3386606847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T11:21:24.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.869+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7c5002df90 con 0x7f7c5006a760 2024-04-20T11:21:24.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.870+0000 7f7c4d600700 1 -- 172.21.15.44:0/3975687413 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c4009c840 con 0x7f7c5006a760 2024-04-20T11:21:24.871 INFO:teuthology.orchestra.run.smithi044.stdout:253403070512 2024-04-20T11:21:24.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7c340a3c40 msgr2=0x7f7c340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:24.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7c340a3c40 0x7f7c340a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c50103220 tx=0x7f7c3801a040 comp rx=0 tx=0).stop 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 msgr2=0x7f7c50103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 0x7f7c50103610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c370 tx=0x7f7c4000c680 comp rx=0 tx=0).stop 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 shutdown_connections 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f7c340a3c40 0x7f7c340a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c50108a40 0x7f7c50102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c5006a760 0x7f7c50103610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 --2- 172.21.15.44:0/3975687413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c5006b100 0x7f7c50101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 >> 172.21.15.44:0/3975687413 conn(0x7f7c50078900 msgr2=0x7f7c500ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 shutdown_connections 2024-04-20T11:21:24.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:24.873+0000 7f7c56c00700 1 -- 172.21.15.44:0/3975687413 wait complete. 2024-04-20T11:21:25.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:25 smithi044 ceph-mon[16855]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-20T11:21:25.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:25 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3975687413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T11:21:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:25 smithi165 ceph-mon[21975]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-20T11:21:25.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:25 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3975687413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T11:21:25.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:25 smithi078 ceph-mon[21464]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-20T11:21:25.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:25 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3975687413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T11:21:25.736 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757313 for osd.5 2024-04-20T11:21:25.736 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:25.967 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495669 got 197568495674 for osd.6 2024-04-20T11:21:25.968 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:26.470 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182476 for osd.3 2024-04-20T11:21:26.470 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:26.762 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645346 for osd.10 2024-04-20T11:21:26.762 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:26.939 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:27.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:27 smithi044 ceph-mon[16855]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-20T11:21:27.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:27 smithi165 ceph-mon[21975]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-20T11:21:27.618 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808873 for osd.9 2024-04-20T11:21:27.618 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:27.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:27 smithi078 ceph-mon[21464]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-20T11:21:27.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383704 got 343597383709 for osd.11 2024-04-20T11:21:27.986 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:29.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070512 for osd.8 2024-04-20T11:21:29.059 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:29.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:29 smithi044 ceph-mon[16855]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-20T11:21:29.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:29 smithi165 ceph-mon[21975]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-20T11:21:29.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:29 smithi078 ceph-mon[21464]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-20T11:21:31.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 -- 172.21.15.44:0/3299350242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06a760 msgr2=0x7f924c06abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 --2- 172.21.15.44:0/3299350242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06a760 0x7f924c06abc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 -- 172.21.15.44:0/3299350242 shutdown_connections 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 --2- 172.21.15.44:0/3299350242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924c06b100 0x7f924c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 --2- 172.21.15.44:0/3299350242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f924c108a40 0x7f924c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 --2- 172.21.15.44:0/3299350242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06a760 0x7f924c06abc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 -- 172.21.15.44:0/3299350242 >> 172.21.15.44:0/3299350242 conn(0x7f924c078900 msgr2=0x7f924c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 -- 172.21.15.44:0/3299350242 shutdown_connections 2024-04-20T11:21:31.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.275+0000 7f9252800700 1 -- 172.21.15.44:0/3299350242 wait complete. 2024-04-20T11:21:31.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.276+0000 7f9252800700 1 Processor -- start 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 -- start start 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924c06a760 0x7f924c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 0x7f924c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f924c108a40 0x7f924c19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f924c112ee0 con 0x7f924c06b100 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f924c112d60 con 0x7f924c108a40 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f9252800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f924c113060 con 0x7f924c06a760 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f924aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 0x7f924c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f924aa00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 0x7f924c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44774/0 (socket says 172.21.15.44:44774) 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f924aa00700 1 -- 172.21.15.44:0/2950330896 learned_addr learned my addr 172.21.15.44:0/2950330896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f924b400700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924c06a760 0x7f924c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.277+0000 7f924be00700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f924c108a40 0x7f924c19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.278+0000 7f924aa00700 1 -- 172.21.15.44:0/2950330896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924c06a760 msgr2=0x7f924c19b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.278+0000 7f924aa00700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924c06a760 0x7f924c19b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.278+0000 7f924aa00700 1 -- 172.21.15.44:0/2950330896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f924c108a40 msgr2=0x7f924c19fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.278+0000 7f924aa00700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f924c108a40 0x7f924c19fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.278+0000 7f924aa00700 1 -- 172.21.15.44:0/2950330896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c1a05e0 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.278+0000 7f924aa00700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 0x7f924c19bb10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9238000c00 tx=0x7f9238005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.279+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238027070 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.279+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f923800fdc0 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.279+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c1a0870 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.279+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92380173f0 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.279+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c1a8150 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.280+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c069740 con 0x7f924c06b100 2024-04-20T11:21:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.282+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f923802a070 con 0x7f924c06b100 2024-04-20T11:21:31.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.284+0000 7f9243e00700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f92300a3c40 0x7f92300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:31.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.284+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f92380d0ff0 con 0x7f924c06b100 2024-04-20T11:21:31.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.284+0000 7f924b400700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f92300a3c40 0x7f92300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:31.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.285+0000 7f924b400700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f92300a3c40 0x7f92300a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f923c009680 tx=0x7f923c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:31.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.289+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9238013070 con 0x7f924c06b100 2024-04-20T11:21:31.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:31 smithi044 ceph-mon[16855]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-20T11:21:31.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:31 smithi165 ceph-mon[21975]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-20T11:21:31.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.531+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f924c02df90 con 0x7f924c06b100 2024-04-20T11:21:31.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.532+0000 7f9243e00700 1 -- 172.21.15.44:0/2950330896 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f923809daa0 con 0x7f924c06b100 2024-04-20T11:21:31.533 INFO:teuthology.orchestra.run.smithi044.stdout:34359738462 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f92300a3c40 msgr2=0x7f92300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f92300a3c40 0x7f92300a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f923c009680 tx=0x7f923c009260 comp rx=0 tx=0).stop 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 msgr2=0x7f924c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 0x7f924c19bb10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9238000c00 tx=0x7f9238005590 comp rx=0 tx=0).stop 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 shutdown_connections 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f92300a3c40 0x7f92300a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f924c108a40 0x7f924c19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924c06a760 0x7f924c19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 --2- 172.21.15.44:0/2950330896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924c06b100 0x7f924c19bb10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 >> 172.21.15.44:0/2950330896 conn(0x7f924c078900 msgr2=0x7f924c10deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 shutdown_connections 2024-04-20T11:21:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:31.535+0000 7f9252800700 1 -- 172.21.15.44:0/2950330896 wait complete. 2024-04-20T11:21:31.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:31 smithi078 ceph-mon[21464]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-20T11:21:32.429 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738462 for osd.0 2024-04-20T11:21:32.430 DEBUG:teuthology.parallel:result is None 2024-04-20T11:21:32.430 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-20T11:21:32.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph pg dump --format=json 2024-04-20T11:21:32.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:32 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2950330896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T11:21:32.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:32.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:32 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:32.499 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:32 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2950330896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T11:21:32.499 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:32.499 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:32 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:32.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:32 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2950330896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T11:21:32.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:32.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:32 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:32.912 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:33.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:33 smithi044 ceph-mon[16855]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-20T11:21:33.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:21:33.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:21:33.365 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:33 smithi044 ceph-mon[16855]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:33 smithi165 ceph-mon[21975]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-20T11:21:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:21:33.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:21:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:33 smithi165 ceph-mon[21975]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:33.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:33 smithi078 ceph-mon[21464]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-20T11:21:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:21:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:21:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:33 smithi078 ceph-mon[21464]: from='mgr.14152 172.21.15.44:0/4130593222' entity='mgr.x' 2024-04-20T11:21:35.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 -- 172.21.15.44:0/2846223057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 msgr2=0x7f4510115df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:35.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 --2- 172.21.15.44:0/2846223057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f4510115df0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4508009f90 tx=0x7f450800b330 comp rx=0 tx=0).stop 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 -- 172.21.15.44:0/2846223057 shutdown_connections 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 --2- 172.21.15.44:0/2846223057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f4510115df0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 --2- 172.21.15.44:0/2846223057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f451010f040 0x7f451010f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 --2- 172.21.15.44:0/2846223057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f4510109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 -- 172.21.15.44:0/2846223057 >> 172.21.15.44:0/2846223057 conn(0x7f45100787e0 msgr2=0x7f4510107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.012+0000 7f4516a00700 1 -- 172.21.15.44:0/2846223057 shutdown_connections 2024-04-20T11:21:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.013+0000 7f4516a00700 1 -- 172.21.15.44:0/2846223057 wait complete. 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 Processor -- start 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 -- start start 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f45101a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f45101a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f451010f040 0x7f451019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f45101193b0 con 0x7f4510108f90 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4510119230 con 0x7f4510109950 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4516a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4510119530 con 0x7f451010f040 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.014+0000 7f4515600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f45101a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f45101a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44794/0 (socket says 172.21.15.44:44794) 2024-04-20T11:21:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 -- 172.21.15.44:0/3199289978 learned_addr learned my addr 172.21.15.44:0/3199289978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4514c00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f45101a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4516000700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f451010f040 0x7f451019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 -- 172.21.15.44:0/3199289978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f451010f040 msgr2=0x7f451019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f451010f040 0x7f451019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 -- 172.21.15.44:0/3199289978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 msgr2=0x7f45101a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f45101a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 -- 172.21.15.44:0/3199289978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451019fcf0 con 0x7f4510108f90 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4514c00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f45101a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.015+0000 7f4515600700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f45101a53d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f450400e9f0 tx=0x7f450400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.016+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504004a30 con 0x7f4510108f90 2024-04-20T11:21:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.016+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451019ffe0 con 0x7f4510108f90 2024-04-20T11:21:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.016+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f450400ccc0 con 0x7f4510108f90 2024-04-20T11:21:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.016+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040056c0 con 0x7f4510108f90 2024-04-20T11:21:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.016+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45101ac2a0 con 0x7f4510108f90 2024-04-20T11:21:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.017+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4510069740 con 0x7f4510108f90 2024-04-20T11:21:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.019+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f450400c7c0 con 0x7f4510108f90 2024-04-20T11:21:35.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.020+0000 7f450e000700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f44f80a3c40 0x7f44f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:35.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.021+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f45040d04e0 con 0x7f4510108f90 2024-04-20T11:21:35.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.021+0000 7f4514c00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f44f80a3c40 0x7f44f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:35.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.022+0000 7f4514c00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f44f80a3c40 0x7f44f80a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4510078e90 tx=0x7f44fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:35.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.025+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f450409cec0 con 0x7f4510108f90 2024-04-20T11:21:35.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.264+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45101a0ac0 con 0x7f44f80a3c40 2024-04-20T11:21:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.274+0000 7f450e000700 1 -- 172.21.15.44:0/3199289978 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259098 (secure 0 0 0) 0x7f45101a0ac0 con 0x7f44f80a3c40 2024-04-20T11:21:35.279 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f44f80a3c40 msgr2=0x7f44f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f44f80a3c40 0x7f44f80a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4510078e90 tx=0x7f44fc009260 comp rx=0 tx=0).stop 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 msgr2=0x7f45101a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f45101a53d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f450400e9f0 tx=0x7f450400ed00 comp rx=0 tx=0).stop 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 shutdown_connections 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f44f80a3c40 0x7f44f80a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f451010f040 0x7f451019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4510108f90 0x7f45101a53d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 --2- 172.21.15.44:0/3199289978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4510109950 0x7f45101a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.280+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 >> 172.21.15.44:0/3199289978 conn(0x7f45100787e0 msgr2=0x7f4510079610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.281+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 shutdown_connections 2024-04-20T11:21:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:35.281+0000 7f4516a00700 1 -- 172.21.15.44:0/3199289978 wait complete. 2024-04-20T11:21:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:21:35.634 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:35 smithi044 ceph-mon[16855]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-20T11:21:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:35 smithi078 ceph-mon[21464]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-20T11:21:35.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:35 smithi165 ceph-mon[21975]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-20T11:21:36.274 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-04-20T11:21:34.084658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657178,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1013,"num_read_kb":629,"num_write":427,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360852,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580972,"statfs":{"total":1151940427776,"available":1151570915328,"internally_reserved":0,"allocated":11980800,"data_stored":7436922,"data_compressed":113148,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005097"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.058482+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:21:20.058482+0000","last_peered":"2024-04-20T11:21:20.058482+0000","last_clean":"2024-04-20T11:21:20.058482+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:21:20.058482+0000","last_undegraded":"2024-04-20T11:21:20.058482+0000","last_fullsized":"2024-04-20T11:21:20.058482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060485+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:21:20.060485+0000","last_peered":"2024-04-20T11:21:20.060485+0000","last_clean":"2024-04-20T11:21:20.060485+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:21:20.060485+0000","last_undegraded":"2024-04-20T11:21:20.060485+0000","last_fullsized":"2024-04-20T11:21:20.060485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.056533+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:21:20.056533+0000","last_peered":"2024-04-20T11:21:20.056533+0000","last_clean":"2024-04-20T11:21:20.056533+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:21:20.056533+0000","last_undegraded":"2024-04-20T11:21:20.056533+0000","last_fullsized":"2024-04-20T11:21:20.056533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.030067+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:21:20.030067+0000","last_peered":"2024-04-20T11:21:20.030067+0000","last_clean":"2024-04-20T11:21:20.030067+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:21:20.030067+0000","last_undegraded":"2024-04-20T11:21:20.030067+0000","last_fullsized":"2024-04-20T11:21:20.030067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.040818+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:21:20.040818+0000","last_peered":"2024-04-20T11:21:20.040818+0000","last_clean":"2024-04-20T11:21:20.040818+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:21:20.040818+0000","last_undegraded":"2024-04-20T11:21:20.040818+0000","last_fullsized":"2024-04-20T11:21:20.040818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.044206+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:21:20.044206+0000","last_peered":"2024-04-20T11:21:20.044206+0000","last_clean":"2024-04-20T11:21:20.044206+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:21:20.044206+0000","last_undegraded":"2024-04-20T11:21:20.044206+0000","last_fullsized":"2024-04-20T11:21:20.044206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049655+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:21:20.049655+0000","last_peered":"2024-04-20T11:21:20.049655+0000","last_clean":"2024-04-20T11:21:20.049655+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:21:20.049655+0000","last_undegraded":"2024-04-20T11:21:20.049655+0000","last_fullsized":"2024-04-20T11:21:20.049655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.052811+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:21:20.052811+0000","last_peered":"2024-04-20T11:21:20.052811+0000","last_clean":"2024-04-20T11:21:20.052811+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:21:20.052811+0000","last_undegraded":"2024-04-20T11:21:20.052811+0000","last_fullsized":"2024-04-20T11:21:20.052811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.050966+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:21:20.050966+0000","last_peered":"2024-04-20T11:21:20.050966+0000","last_clean":"2024-04-20T11:21:20.050966+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:21:20.050966+0000","last_undegraded":"2024-04-20T11:21:20.050966+0000","last_fullsized":"2024-04-20T11:21:20.050966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.045175+0000","last_change":"2024-04-20T11:20:51.223729+0000","last_active":"2024-04-20T11:21:20.045175+0000","last_peered":"2024-04-20T11:21:20.045175+0000","last_clean":"2024-04-20T11:21:20.045175+0000","last_became_active":"2024-04-20T11:20:51.223340+0000","last_became_peered":"2024-04-20T11:20:51.223340+0000","last_unstale":"2024-04-20T11:21:20.045175+0000","last_undegraded":"2024-04-20T11:21:20.045175+0000","last_fullsized":"2024-04-20T11:21:20.045175+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:13.728111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059671+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:21:20.059671+0000","last_peered":"2024-04-20T11:21:20.059671+0000","last_clean":"2024-04-20T11:21:20.059671+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:21:20.059671+0000","last_undegraded":"2024-04-20T11:21:20.059671+0000","last_fullsized":"2024-04-20T11:21:20.059671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.029319+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:21:20.029319+0000","last_peered":"2024-04-20T11:21:20.029319+0000","last_clean":"2024-04-20T11:21:20.029319+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:21:20.029319+0000","last_undegraded":"2024-04-20T11:21:20.029319+0000","last_fullsized":"2024-04-20T11:21:20.029319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065850+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:21:20.065850+0000","last_peered":"2024-04-20T11:21:20.065850+0000","last_clean":"2024-04-20T11:21:20.065850+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:21:20.065850+0000","last_undegraded":"2024-04-20T11:21:20.065850+0000","last_fullsized":"2024-04-20T11:21:20.065850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060783+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:21:20.060783+0000","last_peered":"2024-04-20T11:21:20.060783+0000","last_clean":"2024-04-20T11:21:20.060783+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:21:20.060783+0000","last_undegraded":"2024-04-20T11:21:20.060783+0000","last_fullsized":"2024-04-20T11:21:20.060783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.057099+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:21:20.057099+0000","last_peered":"2024-04-20T11:21:20.057099+0000","last_clean":"2024-04-20T11:21:20.057099+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:21:20.057099+0000","last_undegraded":"2024-04-20T11:21:20.057099+0000","last_fullsized":"2024-04-20T11:21:20.057099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.057574+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:21:20.057574+0000","last_peered":"2024-04-20T11:21:20.057574+0000","last_clean":"2024-04-20T11:21:20.057574+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:21:20.057574+0000","last_undegraded":"2024-04-20T11:21:20.057574+0000","last_fullsized":"2024-04-20T11:21:20.057574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059400+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:21:20.059400+0000","last_peered":"2024-04-20T11:21:20.059400+0000","last_clean":"2024-04-20T11:21:20.059400+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:21:20.059400+0000","last_undegraded":"2024-04-20T11:21:20.059400+0000","last_fullsized":"2024-04-20T11:21:20.059400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.063948+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:21:20.063948+0000","last_peered":"2024-04-20T11:21:20.063948+0000","last_clean":"2024-04-20T11:21:20.063948+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:21:20.063948+0000","last_undegraded":"2024-04-20T11:21:20.063948+0000","last_fullsized":"2024-04-20T11:21:20.063948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.046338+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:21:20.046338+0000","last_peered":"2024-04-20T11:21:20.046338+0000","last_clean":"2024-04-20T11:21:20.046338+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:21:20.046338+0000","last_undegraded":"2024-04-20T11:21:20.046338+0000","last_fullsized":"2024-04-20T11:21:20.046338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:31.994960+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:21:31.994960+0000","last_peered":"2024-04-20T11:21:31.994960+0000","last_clean":"2024-04-20T11:21:31.994960+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:21:31.994960+0000","last_undegraded":"2024-04-20T11:21:31.994960+0000","last_fullsized":"2024-04-20T11:21:31.994960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.054186+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:21:20.054186+0000","last_peered":"2024-04-20T11:21:20.054186+0000","last_clean":"2024-04-20T11:21:20.054186+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:21:20.054186+0000","last_undegraded":"2024-04-20T11:21:20.054186+0000","last_fullsized":"2024-04-20T11:21:20.054186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.504159+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:20:51.504159+0000","last_peered":"2024-04-20T11:20:51.504159+0000","last_clean":"2024-04-20T11:20:51.504159+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:20:51.504159+0000","last_undegraded":"2024-04-20T11:20:51.504159+0000","last_fullsized":"2024-04-20T11:20:51.504159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060159+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:21:20.060159+0000","last_peered":"2024-04-20T11:21:20.060159+0000","last_clean":"2024-04-20T11:21:20.060159+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:21:20.060159+0000","last_undegraded":"2024-04-20T11:21:20.060159+0000","last_fullsized":"2024-04-20T11:21:20.060159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217428+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:20:51.217428+0000","last_peered":"2024-04-20T11:20:51.217428+0000","last_clean":"2024-04-20T11:20:51.217428+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:20:51.217428+0000","last_undegraded":"2024-04-20T11:20:51.217428+0000","last_fullsized":"2024-04-20T11:20:51.217428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.067681+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:21:20.067681+0000","last_peered":"2024-04-20T11:21:20.067681+0000","last_clean":"2024-04-20T11:21:20.067681+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:21:20.067681+0000","last_undegraded":"2024-04-20T11:21:20.067681+0000","last_fullsized":"2024-04-20T11:21:20.067681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217078+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:20:51.217078+0000","last_peered":"2024-04-20T11:20:51.217078+0000","last_clean":"2024-04-20T11:20:51.217078+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:20:51.217078+0000","last_undegraded":"2024-04-20T11:20:51.217078+0000","last_fullsized":"2024-04-20T11:20:51.217078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.048847+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:21:20.048847+0000","last_peered":"2024-04-20T11:21:20.048847+0000","last_clean":"2024-04-20T11:21:20.048847+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:21:20.048847+0000","last_undegraded":"2024-04-20T11:21:20.048847+0000","last_fullsized":"2024-04-20T11:21:20.048847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:32.924201+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:21:32.924201+0000","last_peered":"2024-04-20T11:21:32.924201+0000","last_clean":"2024-04-20T11:21:32.924201+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:21:32.924201+0000","last_undegraded":"2024-04-20T11:21:32.924201+0000","last_fullsized":"2024-04-20T11:21:32.924201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.043159+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:21:20.043159+0000","last_peered":"2024-04-20T11:21:20.043159+0000","last_clean":"2024-04-20T11:21:20.043159+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:21:20.043159+0000","last_undegraded":"2024-04-20T11:21:20.043159+0000","last_fullsized":"2024-04-20T11:21:20.043159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.219792+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:20:51.219792+0000","last_peered":"2024-04-20T11:20:51.219792+0000","last_clean":"2024-04-20T11:20:51.219792+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:20:51.219792+0000","last_undegraded":"2024-04-20T11:20:51.219792+0000","last_fullsized":"2024-04-20T11:20:51.219792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060237+0000","last_change":"2024-04-20T11:20:51.505789+0000","last_active":"2024-04-20T11:21:20.060237+0000","last_peered":"2024-04-20T11:21:20.060237+0000","last_clean":"2024-04-20T11:21:20.060237+0000","last_became_active":"2024-04-20T11:20:51.505449+0000","last_became_peered":"2024-04-20T11:20:51.505449+0000","last_unstale":"2024-04-20T11:21:20.060237+0000","last_undegraded":"2024-04-20T11:21:20.060237+0000","last_fullsized":"2024-04-20T11:21:20.060237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:30.003670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220510+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:20:51.220510+0000","last_peered":"2024-04-20T11:20:51.220510+0000","last_clean":"2024-04-20T11:20:51.220510+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:20:51.220510+0000","last_undegraded":"2024-04-20T11:20:51.220510+0000","last_fullsized":"2024-04-20T11:20:51.220510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.062113+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:21:20.062113+0000","last_peered":"2024-04-20T11:21:20.062113+0000","last_clean":"2024-04-20T11:21:20.062113+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:21:20.062113+0000","last_undegraded":"2024-04-20T11:21:20.062113+0000","last_fullsized":"2024-04-20T11:21:20.062113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220110+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:20:51.220110+0000","last_peered":"2024-04-20T11:20:51.220110+0000","last_clean":"2024-04-20T11:20:51.220110+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:20:51.220110+0000","last_undegraded":"2024-04-20T11:20:51.220110+0000","last_fullsized":"2024-04-20T11:20:51.220110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.052665+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:21:20.052665+0000","last_peered":"2024-04-20T11:21:20.052665+0000","last_clean":"2024-04-20T11:21:20.052665+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:21:20.052665+0000","last_undegraded":"2024-04-20T11:21:20.052665+0000","last_fullsized":"2024-04-20T11:21:20.052665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":190,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:30.920891+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:21:30.920891+0000","last_peered":"2024-04-20T11:21:30.920891+0000","last_clean":"2024-04-20T11:21:30.920891+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:21:30.920891+0000","last_undegraded":"2024-04-20T11:21:30.920891+0000","last_fullsized":"2024-04-20T11:21:30.920891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060821+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:21:20.060821+0000","last_peered":"2024-04-20T11:21:20.060821+0000","last_clean":"2024-04-20T11:21:20.060821+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:21:20.060821+0000","last_undegraded":"2024-04-20T11:21:20.060821+0000","last_fullsized":"2024-04-20T11:21:20.060821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220783+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:20:51.220783+0000","last_peered":"2024-04-20T11:20:51.220783+0000","last_clean":"2024-04-20T11:20:51.220783+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:20:51.220783+0000","last_undegraded":"2024-04-20T11:20:51.220783+0000","last_fullsized":"2024-04-20T11:20:51.220783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060543+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:21:20.060543+0000","last_peered":"2024-04-20T11:21:20.060543+0000","last_clean":"2024-04-20T11:21:20.060543+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:21:20.060543+0000","last_undegraded":"2024-04-20T11:21:20.060543+0000","last_fullsized":"2024-04-20T11:21:20.060543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220871+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:20:51.220871+0000","last_peered":"2024-04-20T11:20:51.220871+0000","last_clean":"2024-04-20T11:20:51.220871+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:20:51.220871+0000","last_undegraded":"2024-04-20T11:20:51.220871+0000","last_fullsized":"2024-04-20T11:20:51.220871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059751+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:21:20.059751+0000","last_peered":"2024-04-20T11:21:20.059751+0000","last_clean":"2024-04-20T11:21:20.059751+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:21:20.059751+0000","last_undegraded":"2024-04-20T11:21:20.059751+0000","last_fullsized":"2024-04-20T11:21:20.059751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":226,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:30.920709+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:21:30.920709+0000","last_peered":"2024-04-20T11:21:30.920709+0000","last_clean":"2024-04-20T11:21:30.920709+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:21:30.920709+0000","last_undegraded":"2024-04-20T11:21:30.920709+0000","last_fullsized":"2024-04-20T11:21:30.920709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59090,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":145,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.044828+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:21:20.044828+0000","last_peered":"2024-04-20T11:21:20.044828+0000","last_clean":"2024-04-20T11:21:20.044828+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:21:20.044828+0000","last_undegraded":"2024-04-20T11:21:20.044828+0000","last_fullsized":"2024-04-20T11:21:20.044828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216620+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:20:51.216620+0000","last_peered":"2024-04-20T11:20:51.216620+0000","last_clean":"2024-04-20T11:20:51.216620+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:20:51.216620+0000","last_undegraded":"2024-04-20T11:20:51.216620+0000","last_fullsized":"2024-04-20T11:20:51.216620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.064746+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:21:20.064746+0000","last_peered":"2024-04-20T11:21:20.064746+0000","last_clean":"2024-04-20T11:21:20.064746+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:21:20.064746+0000","last_undegraded":"2024-04-20T11:21:20.064746+0000","last_fullsized":"2024-04-20T11:21:20.064746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220250+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:20:51.220250+0000","last_peered":"2024-04-20T11:20:51.220250+0000","last_clean":"2024-04-20T11:20:51.220250+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:20:51.220250+0000","last_undegraded":"2024-04-20T11:20:51.220250+0000","last_fullsized":"2024-04-20T11:20:51.220250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.064019+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:21:20.064019+0000","last_peered":"2024-04-20T11:21:20.064019+0000","last_clean":"2024-04-20T11:21:20.064019+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:21:20.064019+0000","last_undegraded":"2024-04-20T11:21:20.064019+0000","last_fullsized":"2024-04-20T11:21:20.064019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:27.985252+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:21:27.985252+0000","last_peered":"2024-04-20T11:21:27.985252+0000","last_clean":"2024-04-20T11:21:27.985252+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:21:27.985252+0000","last_undegraded":"2024-04-20T11:21:27.985252+0000","last_fullsized":"2024-04-20T11:21:27.985252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.042935+0000","last_change":"2024-04-20T11:20:51.223843+0000","last_active":"2024-04-20T11:21:20.042935+0000","last_peered":"2024-04-20T11:21:20.042935+0000","last_clean":"2024-04-20T11:21:20.042935+0000","last_became_active":"2024-04-20T11:20:51.223311+0000","last_became_peered":"2024-04-20T11:20:51.223311+0000","last_unstale":"2024-04-20T11:21:20.042935+0000","last_undegraded":"2024-04-20T11:21:20.042935+0000","last_fullsized":"2024-04-20T11:21:20.042935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:38.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.236718+0000","last_change":"2024-04-20T11:20:51.236281+0000","last_active":"2024-04-20T11:20:51.236718+0000","last_peered":"2024-04-20T11:20:51.236718+0000","last_clean":"2024-04-20T11:20:51.236718+0000","last_became_active":"2024-04-20T11:20:51.222316+0000","last_became_peered":"2024-04-20T11:20:51.222316+0000","last_unstale":"2024-04-20T11:20:51.236718+0000","last_undegraded":"2024-04-20T11:20:51.236718+0000","last_fullsized":"2024-04-20T11:20:51.236718+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.048905+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:21:20.048905+0000","last_peered":"2024-04-20T11:21:20.048905+0000","last_clean":"2024-04-20T11:21:20.048905+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:21:20.048905+0000","last_undegraded":"2024-04-20T11:21:20.048905+0000","last_fullsized":"2024-04-20T11:21:20.048905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217311+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:20:51.217311+0000","last_peered":"2024-04-20T11:20:51.217311+0000","last_clean":"2024-04-20T11:20:51.217311+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:20:51.217311+0000","last_undegraded":"2024-04-20T11:20:51.217311+0000","last_fullsized":"2024-04-20T11:20:51.217311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.051338+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:21:20.051338+0000","last_peered":"2024-04-20T11:21:20.051338+0000","last_clean":"2024-04-20T11:21:20.051338+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:21:20.051338+0000","last_undegraded":"2024-04-20T11:21:20.051338+0000","last_fullsized":"2024-04-20T11:21:20.051338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216252+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:20:51.216252+0000","last_peered":"2024-04-20T11:20:51.216252+0000","last_clean":"2024-04-20T11:20:51.216252+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:20:51.216252+0000","last_undegraded":"2024-04-20T11:20:51.216252+0000","last_fullsized":"2024-04-20T11:20:51.216252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.052860+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:21:20.052860+0000","last_peered":"2024-04-20T11:21:20.052860+0000","last_clean":"2024-04-20T11:21:20.052860+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:21:20.052860+0000","last_undegraded":"2024-04-20T11:21:20.052860+0000","last_fullsized":"2024-04-20T11:21:20.052860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:30.994209+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:21:30.994209+0000","last_peered":"2024-04-20T11:21:30.994209+0000","last_clean":"2024-04-20T11:21:30.994209+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:21:30.994209+0000","last_undegraded":"2024-04-20T11:21:30.994209+0000","last_fullsized":"2024-04-20T11:21:30.994209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060392+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:21:20.060392+0000","last_peered":"2024-04-20T11:21:20.060392+0000","last_clean":"2024-04-20T11:21:20.060392+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:21:20.060392+0000","last_undegraded":"2024-04-20T11:21:20.060392+0000","last_fullsized":"2024-04-20T11:21:20.060392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049127+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:21:20.049127+0000","last_peered":"2024-04-20T11:21:20.049127+0000","last_clean":"2024-04-20T11:21:20.049127+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:21:20.049127+0000","last_undegraded":"2024-04-20T11:21:20.049127+0000","last_fullsized":"2024-04-20T11:21:20.049127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:28.986614+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:21:28.986614+0000","last_peered":"2024-04-20T11:21:28.986614+0000","last_clean":"2024-04-20T11:21:28.986614+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:21:28.986614+0000","last_undegraded":"2024-04-20T11:21:28.986614+0000","last_fullsized":"2024-04-20T11:21:28.986614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.053052+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:21:20.053052+0000","last_peered":"2024-04-20T11:21:20.053052+0000","last_clean":"2024-04-20T11:21:20.053052+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:21:20.053052+0000","last_undegraded":"2024-04-20T11:21:20.053052+0000","last_fullsized":"2024-04-20T11:21:20.053052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.501625+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:20:51.501625+0000","last_peered":"2024-04-20T11:20:51.501625+0000","last_clean":"2024-04-20T11:20:51.501625+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:20:51.501625+0000","last_undegraded":"2024-04-20T11:20:51.501625+0000","last_fullsized":"2024-04-20T11:20:51.501625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220166+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:20:51.220166+0000","last_peered":"2024-04-20T11:20:51.220166+0000","last_clean":"2024-04-20T11:20:51.220166+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:20:51.220166+0000","last_undegraded":"2024-04-20T11:20:51.220166+0000","last_fullsized":"2024-04-20T11:20:51.220166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.056508+0000","last_change":"2024-04-20T11:20:51.224226+0000","last_active":"2024-04-20T11:21:20.056508+0000","last_peered":"2024-04-20T11:21:20.056508+0000","last_clean":"2024-04-20T11:21:20.056508+0000","last_became_active":"2024-04-20T11:20:51.223728+0000","last_became_peered":"2024-04-20T11:20:51.223728+0000","last_unstale":"2024-04-20T11:21:20.056508+0000","last_undegraded":"2024-04-20T11:21:20.056508+0000","last_fullsized":"2024-04-20T11:21:20.056508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:37.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220249+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:20:51.220249+0000","last_peered":"2024-04-20T11:20:51.220249+0000","last_clean":"2024-04-20T11:20:51.220249+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:20:51.220249+0000","last_undegraded":"2024-04-20T11:20:51.220249+0000","last_fullsized":"2024-04-20T11:20:51.220249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217323+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:20:51.217323+0000","last_peered":"2024-04-20T11:20:51.217323+0000","last_clean":"2024-04-20T11:20:51.217323+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:20:51.217323+0000","last_undegraded":"2024-04-20T11:20:51.217323+0000","last_fullsized":"2024-04-20T11:20:51.217323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.062405+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:21:20.062405+0000","last_peered":"2024-04-20T11:21:20.062405+0000","last_clean":"2024-04-20T11:21:20.062405+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:21:20.062405+0000","last_undegraded":"2024-04-20T11:21:20.062405+0000","last_fullsized":"2024-04-20T11:21:20.062405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216530+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:20:51.216530+0000","last_peered":"2024-04-20T11:20:51.216530+0000","last_clean":"2024-04-20T11:20:51.216530+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:20:51.216530+0000","last_undegraded":"2024-04-20T11:20:51.216530+0000","last_fullsized":"2024-04-20T11:20:51.216530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.051272+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:21:20.051272+0000","last_peered":"2024-04-20T11:21:20.051272+0000","last_clean":"2024-04-20T11:21:20.051272+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:21:20.051272+0000","last_undegraded":"2024-04-20T11:21:20.051272+0000","last_fullsized":"2024-04-20T11:21:20.051272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.501424+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:20:51.501424+0000","last_peered":"2024-04-20T11:20:51.501424+0000","last_clean":"2024-04-20T11:20:51.501424+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:20:51.501424+0000","last_undegraded":"2024-04-20T11:20:51.501424+0000","last_fullsized":"2024-04-20T11:20:51.501424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.063788+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:21:20.063788+0000","last_peered":"2024-04-20T11:21:20.063788+0000","last_clean":"2024-04-20T11:21:20.063788+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:21:20.063788+0000","last_undegraded":"2024-04-20T11:21:20.063788+0000","last_fullsized":"2024-04-20T11:21:20.063788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.225236+0000","last_change":"2024-04-20T11:20:51.225236+0000","last_active":"2024-04-20T11:20:51.225236+0000","last_peered":"2024-04-20T11:20:51.225236+0000","last_clean":"2024-04-20T11:20:51.225236+0000","last_became_active":"2024-04-20T11:20:51.221635+0000","last_became_peered":"2024-04-20T11:20:51.221635+0000","last_unstale":"2024-04-20T11:20:51.225236+0000","last_undegraded":"2024-04-20T11:20:51.225236+0000","last_fullsized":"2024-04-20T11:20:51.225236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:03.894361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.011173+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:21:20.011173+0000","last_peered":"2024-04-20T11:21:20.011173+0000","last_clean":"2024-04-20T11:21:20.011173+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:21:20.011173+0000","last_undegraded":"2024-04-20T11:21:20.011173+0000","last_fullsized":"2024-04-20T11:21:20.011173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.503397+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:20:51.503397+0000","last_peered":"2024-04-20T11:20:51.503397+0000","last_clean":"2024-04-20T11:20:51.503397+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:20:51.503397+0000","last_undegraded":"2024-04-20T11:20:51.503397+0000","last_fullsized":"2024-04-20T11:20:51.503397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.058199+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:21:20.058199+0000","last_peered":"2024-04-20T11:21:20.058199+0000","last_clean":"2024-04-20T11:21:20.058199+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:21:20.058199+0000","last_undegraded":"2024-04-20T11:21:20.058199+0000","last_fullsized":"2024-04-20T11:21:20.058199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216437+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:20:51.216437+0000","last_peered":"2024-04-20T11:20:51.216437+0000","last_clean":"2024-04-20T11:20:51.216437+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:20:51.216437+0000","last_undegraded":"2024-04-20T11:20:51.216437+0000","last_fullsized":"2024-04-20T11:20:51.216437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.063602+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:21:20.063602+0000","last_peered":"2024-04-20T11:21:20.063602+0000","last_clean":"2024-04-20T11:21:20.063602+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:21:20.063602+0000","last_undegraded":"2024-04-20T11:21:20.063602+0000","last_fullsized":"2024-04-20T11:21:20.063602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216712+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:20:51.216712+0000","last_peered":"2024-04-20T11:20:51.216712+0000","last_clean":"2024-04-20T11:20:51.216712+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:20:51.216712+0000","last_undegraded":"2024-04-20T11:20:51.216712+0000","last_fullsized":"2024-04-20T11:20:51.216712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.058425+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:21:20.058425+0000","last_peered":"2024-04-20T11:21:20.058425+0000","last_clean":"2024-04-20T11:21:20.058425+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:21:20.058425+0000","last_undegraded":"2024-04-20T11:21:20.058425+0000","last_fullsized":"2024-04-20T11:21:20.058425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:32.022528+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:21:32.022528+0000","last_peered":"2024-04-20T11:21:32.022528+0000","last_clean":"2024-04-20T11:21:32.022528+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:21:32.022528+0000","last_undegraded":"2024-04-20T11:21:32.022528+0000","last_fullsized":"2024-04-20T11:21:32.022528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059931+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:21:20.059931+0000","last_peered":"2024-04-20T11:21:20.059931+0000","last_clean":"2024-04-20T11:21:20.059931+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:21:20.059931+0000","last_undegraded":"2024-04-20T11:21:20.059931+0000","last_fullsized":"2024-04-20T11:21:20.059931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220696+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:20:51.220696+0000","last_peered":"2024-04-20T11:20:51.220696+0000","last_clean":"2024-04-20T11:20:51.220696+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:20:51.220696+0000","last_undegraded":"2024-04-20T11:20:51.220696+0000","last_fullsized":"2024-04-20T11:20:51.220696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049226+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:21:20.049226+0000","last_peered":"2024-04-20T11:21:20.049226+0000","last_clean":"2024-04-20T11:21:20.049226+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:21:20.049226+0000","last_undegraded":"2024-04-20T11:21:20.049226+0000","last_fullsized":"2024-04-20T11:21:20.049226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220808+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:20:51.220808+0000","last_peered":"2024-04-20T11:20:51.220808+0000","last_clean":"2024-04-20T11:20:51.220808+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:20:51.220808+0000","last_undegraded":"2024-04-20T11:20:51.220808+0000","last_fullsized":"2024-04-20T11:20:51.220808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.045396+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:21:20.045396+0000","last_peered":"2024-04-20T11:21:20.045396+0000","last_clean":"2024-04-20T11:21:20.045396+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:21:20.045396+0000","last_undegraded":"2024-04-20T11:21:20.045396+0000","last_fullsized":"2024-04-20T11:21:20.045396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.051534+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:21:20.051534+0000","last_peered":"2024-04-20T11:21:20.051534+0000","last_clean":"2024-04-20T11:21:20.051534+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:21:20.051534+0000","last_undegraded":"2024-04-20T11:21:20.051534+0000","last_fullsized":"2024-04-20T11:21:20.051534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049866+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:21:20.049866+0000","last_peered":"2024-04-20T11:21:20.049866+0000","last_clean":"2024-04-20T11:21:20.049866+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:21:20.049866+0000","last_undegraded":"2024-04-20T11:21:20.049866+0000","last_fullsized":"2024-04-20T11:21:20.049866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059978+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:21:20.059978+0000","last_peered":"2024-04-20T11:21:20.059978+0000","last_clean":"2024-04-20T11:21:20.059978+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:21:20.059978+0000","last_undegraded":"2024-04-20T11:21:20.059978+0000","last_fullsized":"2024-04-20T11:21:20.059978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065930+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:21:20.065930+0000","last_peered":"2024-04-20T11:21:20.065930+0000","last_clean":"2024-04-20T11:21:20.065930+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:21:20.065930+0000","last_undegraded":"2024-04-20T11:21:20.065930+0000","last_fullsized":"2024-04-20T11:21:20.065930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065500+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:21:20.065500+0000","last_peered":"2024-04-20T11:21:20.065500+0000","last_clean":"2024-04-20T11:21:20.065500+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:21:20.065500+0000","last_undegraded":"2024-04-20T11:21:20.065500+0000","last_fullsized":"2024-04-20T11:21:20.065500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.050496+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:21:20.050496+0000","last_peered":"2024-04-20T11:21:20.050496+0000","last_clean":"2024-04-20T11:21:20.050496+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:21:20.050496+0000","last_undegraded":"2024-04-20T11:21:20.050496+0000","last_fullsized":"2024-04-20T11:21:20.050496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.042323+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:21:20.042323+0000","last_peered":"2024-04-20T11:21:20.042323+0000","last_clean":"2024-04-20T11:21:20.042323+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:21:20.042323+0000","last_undegraded":"2024-04-20T11:21:20.042323+0000","last_fullsized":"2024-04-20T11:21:20.042323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.040554+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:21:20.040554+0000","last_peered":"2024-04-20T11:21:20.040554+0000","last_clean":"2024-04-20T11:21:20.040554+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:21:20.040554+0000","last_undegraded":"2024-04-20T11:21:20.040554+0000","last_fullsized":"2024-04-20T11:21:20.040554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.055847+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:21:20.055847+0000","last_peered":"2024-04-20T11:21:20.055847+0000","last_clean":"2024-04-20T11:21:20.055847+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:21:20.055847+0000","last_undegraded":"2024-04-20T11:21:20.055847+0000","last_fullsized":"2024-04-20T11:21:20.055847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.055171+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:21:20.055171+0000","last_peered":"2024-04-20T11:21:20.055171+0000","last_clean":"2024-04-20T11:21:20.055171+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:21:20.055171+0000","last_undegraded":"2024-04-20T11:21:20.055171+0000","last_fullsized":"2024-04-20T11:21:20.055171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.057265+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:21:20.057265+0000","last_peered":"2024-04-20T11:21:20.057265+0000","last_clean":"2024-04-20T11:21:20.057265+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:21:20.057265+0000","last_undegraded":"2024-04-20T11:21:20.057265+0000","last_fullsized":"2024-04-20T11:21:20.057265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065639+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:21:20.065639+0000","last_peered":"2024-04-20T11:21:20.065639+0000","last_clean":"2024-04-20T11:21:20.065639+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:21:20.065639+0000","last_undegraded":"2024-04-20T11:21:20.065639+0000","last_fullsized":"2024-04-20T11:21:20.065639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.046927+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:21:20.046927+0000","last_peered":"2024-04-20T11:21:20.046927+0000","last_clean":"2024-04-20T11:21:20.046927+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:21:20.046927+0000","last_undegraded":"2024-04-20T11:21:20.046927+0000","last_fullsized":"2024-04-20T11:21:20.046927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282095,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66576,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":877,"num_read_kb":510,"num_write":244,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199728,"data_compressed":5514,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":457285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1107292,"data_compressed":37716,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1049608,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":516291,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":458159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":517043,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1048754,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60842,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61594,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61358,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:21:36.279 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph pg dump --format=json 2024-04-20T11:21:36.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:36 smithi078 ceph-mon[21464]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:21:36.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:36 smithi044 ceph-mon[16855]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:21:36.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:36 smithi165 ceph-mon[21975]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:21:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:37 smithi078 ceph-mon[21464]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-20T11:21:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:37 smithi044 ceph-mon[16855]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-20T11:21:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:37 smithi165 ceph-mon[21975]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-20T11:21:39.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 -- 172.21.15.44:0/4062997199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe564109c90 msgr2=0x7fe56410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:39.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 --2- 172.21.15.44:0/4062997199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe564109c90 0x7fe56410dce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098f0 tx=0x7fe54c009c00 comp rx=0 tx=0).stop 2024-04-20T11:21:39.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 -- 172.21.15.44:0/4062997199 shutdown_connections 2024-04-20T11:21:39.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 --2- 172.21.15.44:0/4062997199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe56410e3c0 0x7fe564115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 --2- 172.21.15.44:0/4062997199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 0x7fe5641096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 --2- 172.21.15.44:0/4062997199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe564109c90 0x7fe56410dce0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 -- 172.21.15.44:0/4062997199 >> 172.21.15.44:0/4062997199 conn(0x7fe564076220 msgr2=0x7fe564107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:39.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 -- 172.21.15.44:0/4062997199 shutdown_connections 2024-04-20T11:21:39.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.108+0000 7fe569a00700 1 -- 172.21.15.44:0/4062997199 wait complete. 2024-04-20T11:21:39.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.109+0000 7fe569a00700 1 Processor -- start 2024-04-20T11:21:39.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.109+0000 7fe569a00700 1 -- start start 2024-04-20T11:21:39.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe569a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 0x7fe5641a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:39.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe569a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe564109c90 0x7fe5641a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:39.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe569a00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe56410e3c0 0x7fe56419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe569a00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641194c0 con 0x7fe5641092e0 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe569a00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe564119340 con 0x7fe56410e3c0 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe569a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe564119640 con 0x7fe564109c90 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe562a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 0x7fe5641a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe563400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe56410e3c0 0x7fe56419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe562a00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 0x7fe5641a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52840/0 (socket says 172.21.15.44:52840) 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe562a00700 1 -- 172.21.15.44:0/631000925 learned_addr learned my addr 172.21.15.44:0/631000925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.110+0000 7fe55a000700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe564109c90 0x7fe5641a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe563400700 1 -- 172.21.15.44:0/631000925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe564109c90 msgr2=0x7fe5641a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe563400700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe564109c90 0x7fe5641a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe563400700 1 -- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 msgr2=0x7fe5641a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe563400700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 0x7fe5641a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe563400700 1 -- 172.21.15.44:0/631000925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5641a0020 con 0x7fe56410e3c0 2024-04-20T11:21:39.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe563400700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe56410e3c0 0x7fe56419f840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00efc0 tx=0x7fe55c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:39.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0092e0 con 0x7fe56410e3c0 2024-04-20T11:21:39.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5641a02b0 con 0x7fe56410e3c0 2024-04-20T11:21:39.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe55c00ce40 con 0x7fe56410e3c0 2024-04-20T11:21:39.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.111+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c007cd0 con 0x7fe56410e3c0 2024-04-20T11:21:39.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.112+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5641ac4d0 con 0x7fe56410e3c0 2024-04-20T11:21:39.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.113+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56410a7c0 con 0x7fe56410e3c0 2024-04-20T11:21:39.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.115+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe55c01b020 con 0x7fe56410e3c0 2024-04-20T11:21:39.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.116+0000 7fe55be00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5480a3cc0 0x7fe5480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:39.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.116+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fe55c0d0980 con 0x7fe56410e3c0 2024-04-20T11:21:39.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.116+0000 7fe562a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5480a3cc0 0x7fe5480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:39.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.117+0000 7fe562a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5480a3cc0 0x7fe5480a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe554005d10 tx=0x7fe554009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:39.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.119+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe55c010040 con 0x7fe56410e3c0 2024-04-20T11:21:39.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.348+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe564062e90 con 0x7fe5480a3cc0 2024-04-20T11:21:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.358+0000 7fe55be00700 1 -- 172.21.15.44:0/631000925 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259096 (secure 0 0 0) 0x7fe564062e90 con 0x7fe5480a3cc0 2024-04-20T11:21:39.363 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5480a3cc0 msgr2=0x7fe5480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5480a3cc0 0x7fe5480a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe554005d10 tx=0x7fe554009310 comp rx=0 tx=0).stop 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe56410e3c0 msgr2=0x7fe56419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe56410e3c0 0x7fe56419f840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00efc0 tx=0x7fe55c00c3a0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 shutdown_connections 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe5480a3cc0 0x7fe5480a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe56410e3c0 0x7fe56419f840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5641092e0 0x7fe5641a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 --2- 172.21.15.44:0/631000925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe564109c90 0x7fe5641a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:39.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.363+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 >> 172.21.15.44:0/631000925 conn(0x7fe564076220 msgr2=0x7fe564076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:39.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.364+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 shutdown_connections 2024-04-20T11:21:39.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:39.364+0000 7fe569a00700 1 -- 172.21.15.44:0/631000925 wait complete. 2024-04-20T11:21:39.367 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:21:39.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:39 smithi078 ceph-mon[21464]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:39.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:39 smithi044 ceph-mon[16855]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:39 smithi165 ceph-mon[21975]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:40.356 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-20T11:21:38.086439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657178,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1048,"num_read_kb":649,"num_write":427,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360852,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580972,"statfs":{"total":1151940427776,"available":1151570915328,"internally_reserved":0,"allocated":11980800,"data_stored":7436922,"data_compressed":113148,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005189"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.058482+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:21:20.058482+0000","last_peered":"2024-04-20T11:21:20.058482+0000","last_clean":"2024-04-20T11:21:20.058482+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:21:20.058482+0000","last_undegraded":"2024-04-20T11:21:20.058482+0000","last_fullsized":"2024-04-20T11:21:20.058482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060485+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:21:20.060485+0000","last_peered":"2024-04-20T11:21:20.060485+0000","last_clean":"2024-04-20T11:21:20.060485+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:21:20.060485+0000","last_undegraded":"2024-04-20T11:21:20.060485+0000","last_fullsized":"2024-04-20T11:21:20.060485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.056533+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:21:20.056533+0000","last_peered":"2024-04-20T11:21:20.056533+0000","last_clean":"2024-04-20T11:21:20.056533+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:21:20.056533+0000","last_undegraded":"2024-04-20T11:21:20.056533+0000","last_fullsized":"2024-04-20T11:21:20.056533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.030067+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:21:20.030067+0000","last_peered":"2024-04-20T11:21:20.030067+0000","last_clean":"2024-04-20T11:21:20.030067+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:21:20.030067+0000","last_undegraded":"2024-04-20T11:21:20.030067+0000","last_fullsized":"2024-04-20T11:21:20.030067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.040818+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:21:20.040818+0000","last_peered":"2024-04-20T11:21:20.040818+0000","last_clean":"2024-04-20T11:21:20.040818+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:21:20.040818+0000","last_undegraded":"2024-04-20T11:21:20.040818+0000","last_fullsized":"2024-04-20T11:21:20.040818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.044206+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:21:20.044206+0000","last_peered":"2024-04-20T11:21:20.044206+0000","last_clean":"2024-04-20T11:21:20.044206+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:21:20.044206+0000","last_undegraded":"2024-04-20T11:21:20.044206+0000","last_fullsized":"2024-04-20T11:21:20.044206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049655+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:21:20.049655+0000","last_peered":"2024-04-20T11:21:20.049655+0000","last_clean":"2024-04-20T11:21:20.049655+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:21:20.049655+0000","last_undegraded":"2024-04-20T11:21:20.049655+0000","last_fullsized":"2024-04-20T11:21:20.049655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.052811+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:21:20.052811+0000","last_peered":"2024-04-20T11:21:20.052811+0000","last_clean":"2024-04-20T11:21:20.052811+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:21:20.052811+0000","last_undegraded":"2024-04-20T11:21:20.052811+0000","last_fullsized":"2024-04-20T11:21:20.052811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.050966+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:21:20.050966+0000","last_peered":"2024-04-20T11:21:20.050966+0000","last_clean":"2024-04-20T11:21:20.050966+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:21:20.050966+0000","last_undegraded":"2024-04-20T11:21:20.050966+0000","last_fullsized":"2024-04-20T11:21:20.050966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.045175+0000","last_change":"2024-04-20T11:20:51.223729+0000","last_active":"2024-04-20T11:21:20.045175+0000","last_peered":"2024-04-20T11:21:20.045175+0000","last_clean":"2024-04-20T11:21:20.045175+0000","last_became_active":"2024-04-20T11:20:51.223340+0000","last_became_peered":"2024-04-20T11:20:51.223340+0000","last_unstale":"2024-04-20T11:21:20.045175+0000","last_undegraded":"2024-04-20T11:21:20.045175+0000","last_fullsized":"2024-04-20T11:21:20.045175+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:13.728111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059671+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:21:20.059671+0000","last_peered":"2024-04-20T11:21:20.059671+0000","last_clean":"2024-04-20T11:21:20.059671+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:21:20.059671+0000","last_undegraded":"2024-04-20T11:21:20.059671+0000","last_fullsized":"2024-04-20T11:21:20.059671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.029319+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:21:20.029319+0000","last_peered":"2024-04-20T11:21:20.029319+0000","last_clean":"2024-04-20T11:21:20.029319+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:21:20.029319+0000","last_undegraded":"2024-04-20T11:21:20.029319+0000","last_fullsized":"2024-04-20T11:21:20.029319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065850+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:21:20.065850+0000","last_peered":"2024-04-20T11:21:20.065850+0000","last_clean":"2024-04-20T11:21:20.065850+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:21:20.065850+0000","last_undegraded":"2024-04-20T11:21:20.065850+0000","last_fullsized":"2024-04-20T11:21:20.065850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060783+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:21:20.060783+0000","last_peered":"2024-04-20T11:21:20.060783+0000","last_clean":"2024-04-20T11:21:20.060783+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:21:20.060783+0000","last_undegraded":"2024-04-20T11:21:20.060783+0000","last_fullsized":"2024-04-20T11:21:20.060783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.057099+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:21:20.057099+0000","last_peered":"2024-04-20T11:21:20.057099+0000","last_clean":"2024-04-20T11:21:20.057099+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:21:20.057099+0000","last_undegraded":"2024-04-20T11:21:20.057099+0000","last_fullsized":"2024-04-20T11:21:20.057099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.057574+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:21:20.057574+0000","last_peered":"2024-04-20T11:21:20.057574+0000","last_clean":"2024-04-20T11:21:20.057574+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:21:20.057574+0000","last_undegraded":"2024-04-20T11:21:20.057574+0000","last_fullsized":"2024-04-20T11:21:20.057574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059400+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:21:20.059400+0000","last_peered":"2024-04-20T11:21:20.059400+0000","last_clean":"2024-04-20T11:21:20.059400+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:21:20.059400+0000","last_undegraded":"2024-04-20T11:21:20.059400+0000","last_fullsized":"2024-04-20T11:21:20.059400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.063948+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:21:20.063948+0000","last_peered":"2024-04-20T11:21:20.063948+0000","last_clean":"2024-04-20T11:21:20.063948+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:21:20.063948+0000","last_undegraded":"2024-04-20T11:21:20.063948+0000","last_fullsized":"2024-04-20T11:21:20.063948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.046338+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:21:20.046338+0000","last_peered":"2024-04-20T11:21:20.046338+0000","last_clean":"2024-04-20T11:21:20.046338+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:21:20.046338+0000","last_undegraded":"2024-04-20T11:21:20.046338+0000","last_fullsized":"2024-04-20T11:21:20.046338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:31.994960+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:21:31.994960+0000","last_peered":"2024-04-20T11:21:31.994960+0000","last_clean":"2024-04-20T11:21:31.994960+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:21:31.994960+0000","last_undegraded":"2024-04-20T11:21:31.994960+0000","last_fullsized":"2024-04-20T11:21:31.994960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.054186+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:21:20.054186+0000","last_peered":"2024-04-20T11:21:20.054186+0000","last_clean":"2024-04-20T11:21:20.054186+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:21:20.054186+0000","last_undegraded":"2024-04-20T11:21:20.054186+0000","last_fullsized":"2024-04-20T11:21:20.054186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.504159+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:20:51.504159+0000","last_peered":"2024-04-20T11:20:51.504159+0000","last_clean":"2024-04-20T11:20:51.504159+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:20:51.504159+0000","last_undegraded":"2024-04-20T11:20:51.504159+0000","last_fullsized":"2024-04-20T11:20:51.504159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060159+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:21:20.060159+0000","last_peered":"2024-04-20T11:21:20.060159+0000","last_clean":"2024-04-20T11:21:20.060159+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:21:20.060159+0000","last_undegraded":"2024-04-20T11:21:20.060159+0000","last_fullsized":"2024-04-20T11:21:20.060159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217428+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:20:51.217428+0000","last_peered":"2024-04-20T11:20:51.217428+0000","last_clean":"2024-04-20T11:20:51.217428+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:20:51.217428+0000","last_undegraded":"2024-04-20T11:20:51.217428+0000","last_fullsized":"2024-04-20T11:20:51.217428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.067681+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:21:20.067681+0000","last_peered":"2024-04-20T11:21:20.067681+0000","last_clean":"2024-04-20T11:21:20.067681+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:21:20.067681+0000","last_undegraded":"2024-04-20T11:21:20.067681+0000","last_fullsized":"2024-04-20T11:21:20.067681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217078+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:20:51.217078+0000","last_peered":"2024-04-20T11:20:51.217078+0000","last_clean":"2024-04-20T11:20:51.217078+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:20:51.217078+0000","last_undegraded":"2024-04-20T11:20:51.217078+0000","last_fullsized":"2024-04-20T11:20:51.217078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.048847+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:21:20.048847+0000","last_peered":"2024-04-20T11:21:20.048847+0000","last_clean":"2024-04-20T11:21:20.048847+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:21:20.048847+0000","last_undegraded":"2024-04-20T11:21:20.048847+0000","last_fullsized":"2024-04-20T11:21:20.048847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:32.924201+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:21:32.924201+0000","last_peered":"2024-04-20T11:21:32.924201+0000","last_clean":"2024-04-20T11:21:32.924201+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:21:32.924201+0000","last_undegraded":"2024-04-20T11:21:32.924201+0000","last_fullsized":"2024-04-20T11:21:32.924201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.043159+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:21:20.043159+0000","last_peered":"2024-04-20T11:21:20.043159+0000","last_clean":"2024-04-20T11:21:20.043159+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:21:20.043159+0000","last_undegraded":"2024-04-20T11:21:20.043159+0000","last_fullsized":"2024-04-20T11:21:20.043159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.219792+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:20:51.219792+0000","last_peered":"2024-04-20T11:20:51.219792+0000","last_clean":"2024-04-20T11:20:51.219792+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:20:51.219792+0000","last_undegraded":"2024-04-20T11:20:51.219792+0000","last_fullsized":"2024-04-20T11:20:51.219792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060237+0000","last_change":"2024-04-20T11:20:51.505789+0000","last_active":"2024-04-20T11:21:20.060237+0000","last_peered":"2024-04-20T11:21:20.060237+0000","last_clean":"2024-04-20T11:21:20.060237+0000","last_became_active":"2024-04-20T11:20:51.505449+0000","last_became_peered":"2024-04-20T11:20:51.505449+0000","last_unstale":"2024-04-20T11:21:20.060237+0000","last_undegraded":"2024-04-20T11:21:20.060237+0000","last_fullsized":"2024-04-20T11:21:20.060237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:30.003670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220510+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:20:51.220510+0000","last_peered":"2024-04-20T11:20:51.220510+0000","last_clean":"2024-04-20T11:20:51.220510+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:20:51.220510+0000","last_undegraded":"2024-04-20T11:20:51.220510+0000","last_fullsized":"2024-04-20T11:20:51.220510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.062113+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:21:20.062113+0000","last_peered":"2024-04-20T11:21:20.062113+0000","last_clean":"2024-04-20T11:21:20.062113+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:21:20.062113+0000","last_undegraded":"2024-04-20T11:21:20.062113+0000","last_fullsized":"2024-04-20T11:21:20.062113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220110+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:20:51.220110+0000","last_peered":"2024-04-20T11:20:51.220110+0000","last_clean":"2024-04-20T11:20:51.220110+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:20:51.220110+0000","last_undegraded":"2024-04-20T11:20:51.220110+0000","last_fullsized":"2024-04-20T11:20:51.220110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.052665+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:21:20.052665+0000","last_peered":"2024-04-20T11:21:20.052665+0000","last_clean":"2024-04-20T11:21:20.052665+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:21:20.052665+0000","last_undegraded":"2024-04-20T11:21:20.052665+0000","last_fullsized":"2024-04-20T11:21:20.052665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":200,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:35.927755+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:21:35.927755+0000","last_peered":"2024-04-20T11:21:35.927755+0000","last_clean":"2024-04-20T11:21:35.927755+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:21:35.927755+0000","last_undegraded":"2024-04-20T11:21:35.927755+0000","last_fullsized":"2024-04-20T11:21:35.927755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060821+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:21:20.060821+0000","last_peered":"2024-04-20T11:21:20.060821+0000","last_clean":"2024-04-20T11:21:20.060821+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:21:20.060821+0000","last_undegraded":"2024-04-20T11:21:20.060821+0000","last_fullsized":"2024-04-20T11:21:20.060821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220783+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:20:51.220783+0000","last_peered":"2024-04-20T11:20:51.220783+0000","last_clean":"2024-04-20T11:20:51.220783+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:20:51.220783+0000","last_undegraded":"2024-04-20T11:20:51.220783+0000","last_fullsized":"2024-04-20T11:20:51.220783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060543+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:21:20.060543+0000","last_peered":"2024-04-20T11:21:20.060543+0000","last_clean":"2024-04-20T11:21:20.060543+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:21:20.060543+0000","last_undegraded":"2024-04-20T11:21:20.060543+0000","last_fullsized":"2024-04-20T11:21:20.060543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220871+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:20:51.220871+0000","last_peered":"2024-04-20T11:20:51.220871+0000","last_clean":"2024-04-20T11:20:51.220871+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:20:51.220871+0000","last_undegraded":"2024-04-20T11:20:51.220871+0000","last_fullsized":"2024-04-20T11:20:51.220871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059751+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:21:20.059751+0000","last_peered":"2024-04-20T11:21:20.059751+0000","last_clean":"2024-04-20T11:21:20.059751+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:21:20.059751+0000","last_undegraded":"2024-04-20T11:21:20.059751+0000","last_fullsized":"2024-04-20T11:21:20.059751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":236,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:35.927497+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:21:35.927497+0000","last_peered":"2024-04-20T11:21:35.927497+0000","last_clean":"2024-04-20T11:21:35.927497+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:21:35.927497+0000","last_undegraded":"2024-04-20T11:21:35.927497+0000","last_fullsized":"2024-04-20T11:21:35.927497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59090,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":150,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.044828+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:21:20.044828+0000","last_peered":"2024-04-20T11:21:20.044828+0000","last_clean":"2024-04-20T11:21:20.044828+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:21:20.044828+0000","last_undegraded":"2024-04-20T11:21:20.044828+0000","last_fullsized":"2024-04-20T11:21:20.044828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216620+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:20:51.216620+0000","last_peered":"2024-04-20T11:20:51.216620+0000","last_clean":"2024-04-20T11:20:51.216620+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:20:51.216620+0000","last_undegraded":"2024-04-20T11:20:51.216620+0000","last_fullsized":"2024-04-20T11:20:51.216620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.064746+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:21:20.064746+0000","last_peered":"2024-04-20T11:21:20.064746+0000","last_clean":"2024-04-20T11:21:20.064746+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:21:20.064746+0000","last_undegraded":"2024-04-20T11:21:20.064746+0000","last_fullsized":"2024-04-20T11:21:20.064746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220250+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:20:51.220250+0000","last_peered":"2024-04-20T11:20:51.220250+0000","last_clean":"2024-04-20T11:20:51.220250+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:20:51.220250+0000","last_undegraded":"2024-04-20T11:20:51.220250+0000","last_fullsized":"2024-04-20T11:20:51.220250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.064019+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:21:20.064019+0000","last_peered":"2024-04-20T11:21:20.064019+0000","last_clean":"2024-04-20T11:21:20.064019+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:21:20.064019+0000","last_undegraded":"2024-04-20T11:21:20.064019+0000","last_fullsized":"2024-04-20T11:21:20.064019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":117,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:32.994989+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:21:32.994989+0000","last_peered":"2024-04-20T11:21:32.994989+0000","last_clean":"2024-04-20T11:21:32.994989+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:21:32.994989+0000","last_undegraded":"2024-04-20T11:21:32.994989+0000","last_fullsized":"2024-04-20T11:21:32.994989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.042935+0000","last_change":"2024-04-20T11:20:51.223843+0000","last_active":"2024-04-20T11:21:20.042935+0000","last_peered":"2024-04-20T11:21:20.042935+0000","last_clean":"2024-04-20T11:21:20.042935+0000","last_became_active":"2024-04-20T11:20:51.223311+0000","last_became_peered":"2024-04-20T11:20:51.223311+0000","last_unstale":"2024-04-20T11:21:20.042935+0000","last_undegraded":"2024-04-20T11:21:20.042935+0000","last_fullsized":"2024-04-20T11:21:20.042935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:38.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.236718+0000","last_change":"2024-04-20T11:20:51.236281+0000","last_active":"2024-04-20T11:20:51.236718+0000","last_peered":"2024-04-20T11:20:51.236718+0000","last_clean":"2024-04-20T11:20:51.236718+0000","last_became_active":"2024-04-20T11:20:51.222316+0000","last_became_peered":"2024-04-20T11:20:51.222316+0000","last_unstale":"2024-04-20T11:20:51.236718+0000","last_undegraded":"2024-04-20T11:20:51.236718+0000","last_fullsized":"2024-04-20T11:20:51.236718+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.048905+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:21:20.048905+0000","last_peered":"2024-04-20T11:21:20.048905+0000","last_clean":"2024-04-20T11:21:20.048905+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:21:20.048905+0000","last_undegraded":"2024-04-20T11:21:20.048905+0000","last_fullsized":"2024-04-20T11:21:20.048905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217311+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:20:51.217311+0000","last_peered":"2024-04-20T11:20:51.217311+0000","last_clean":"2024-04-20T11:20:51.217311+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:20:51.217311+0000","last_undegraded":"2024-04-20T11:20:51.217311+0000","last_fullsized":"2024-04-20T11:20:51.217311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.051338+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:21:20.051338+0000","last_peered":"2024-04-20T11:21:20.051338+0000","last_clean":"2024-04-20T11:21:20.051338+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:21:20.051338+0000","last_undegraded":"2024-04-20T11:21:20.051338+0000","last_fullsized":"2024-04-20T11:21:20.051338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216252+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:20:51.216252+0000","last_peered":"2024-04-20T11:20:51.216252+0000","last_clean":"2024-04-20T11:20:51.216252+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:20:51.216252+0000","last_undegraded":"2024-04-20T11:20:51.216252+0000","last_fullsized":"2024-04-20T11:20:51.216252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.052860+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:21:20.052860+0000","last_peered":"2024-04-20T11:21:20.052860+0000","last_clean":"2024-04-20T11:21:20.052860+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:21:20.052860+0000","last_undegraded":"2024-04-20T11:21:20.052860+0000","last_fullsized":"2024-04-20T11:21:20.052860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:36.002583+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:21:36.002583+0000","last_peered":"2024-04-20T11:21:36.002583+0000","last_clean":"2024-04-20T11:21:36.002583+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:21:36.002583+0000","last_undegraded":"2024-04-20T11:21:36.002583+0000","last_fullsized":"2024-04-20T11:21:36.002583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.060392+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:21:20.060392+0000","last_peered":"2024-04-20T11:21:20.060392+0000","last_clean":"2024-04-20T11:21:20.060392+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:21:20.060392+0000","last_undegraded":"2024-04-20T11:21:20.060392+0000","last_fullsized":"2024-04-20T11:21:20.060392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049127+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:21:20.049127+0000","last_peered":"2024-04-20T11:21:20.049127+0000","last_clean":"2024-04-20T11:21:20.049127+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:21:20.049127+0000","last_undegraded":"2024-04-20T11:21:20.049127+0000","last_fullsized":"2024-04-20T11:21:20.049127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:33.996250+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:21:33.996250+0000","last_peered":"2024-04-20T11:21:33.996250+0000","last_clean":"2024-04-20T11:21:33.996250+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:21:33.996250+0000","last_undegraded":"2024-04-20T11:21:33.996250+0000","last_fullsized":"2024-04-20T11:21:33.996250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.053052+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:21:20.053052+0000","last_peered":"2024-04-20T11:21:20.053052+0000","last_clean":"2024-04-20T11:21:20.053052+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:21:20.053052+0000","last_undegraded":"2024-04-20T11:21:20.053052+0000","last_fullsized":"2024-04-20T11:21:20.053052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.501625+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:20:51.501625+0000","last_peered":"2024-04-20T11:20:51.501625+0000","last_clean":"2024-04-20T11:20:51.501625+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:20:51.501625+0000","last_undegraded":"2024-04-20T11:20:51.501625+0000","last_fullsized":"2024-04-20T11:20:51.501625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220166+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:20:51.220166+0000","last_peered":"2024-04-20T11:20:51.220166+0000","last_clean":"2024-04-20T11:20:51.220166+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:20:51.220166+0000","last_undegraded":"2024-04-20T11:20:51.220166+0000","last_fullsized":"2024-04-20T11:20:51.220166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.056508+0000","last_change":"2024-04-20T11:20:51.224226+0000","last_active":"2024-04-20T11:21:20.056508+0000","last_peered":"2024-04-20T11:21:20.056508+0000","last_clean":"2024-04-20T11:21:20.056508+0000","last_became_active":"2024-04-20T11:20:51.223728+0000","last_became_peered":"2024-04-20T11:20:51.223728+0000","last_unstale":"2024-04-20T11:21:20.056508+0000","last_undegraded":"2024-04-20T11:21:20.056508+0000","last_fullsized":"2024-04-20T11:21:20.056508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:37.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220249+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:20:51.220249+0000","last_peered":"2024-04-20T11:20:51.220249+0000","last_clean":"2024-04-20T11:20:51.220249+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:20:51.220249+0000","last_undegraded":"2024-04-20T11:20:51.220249+0000","last_fullsized":"2024-04-20T11:20:51.220249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.217323+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:20:51.217323+0000","last_peered":"2024-04-20T11:20:51.217323+0000","last_clean":"2024-04-20T11:20:51.217323+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:20:51.217323+0000","last_undegraded":"2024-04-20T11:20:51.217323+0000","last_fullsized":"2024-04-20T11:20:51.217323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.062405+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:21:20.062405+0000","last_peered":"2024-04-20T11:21:20.062405+0000","last_clean":"2024-04-20T11:21:20.062405+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:21:20.062405+0000","last_undegraded":"2024-04-20T11:21:20.062405+0000","last_fullsized":"2024-04-20T11:21:20.062405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216530+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:20:51.216530+0000","last_peered":"2024-04-20T11:20:51.216530+0000","last_clean":"2024-04-20T11:20:51.216530+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:20:51.216530+0000","last_undegraded":"2024-04-20T11:20:51.216530+0000","last_fullsized":"2024-04-20T11:20:51.216530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.051272+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:21:20.051272+0000","last_peered":"2024-04-20T11:21:20.051272+0000","last_clean":"2024-04-20T11:21:20.051272+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:21:20.051272+0000","last_undegraded":"2024-04-20T11:21:20.051272+0000","last_fullsized":"2024-04-20T11:21:20.051272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.501424+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:20:51.501424+0000","last_peered":"2024-04-20T11:20:51.501424+0000","last_clean":"2024-04-20T11:20:51.501424+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:20:51.501424+0000","last_undegraded":"2024-04-20T11:20:51.501424+0000","last_fullsized":"2024-04-20T11:20:51.501424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.063788+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:21:20.063788+0000","last_peered":"2024-04-20T11:21:20.063788+0000","last_clean":"2024-04-20T11:21:20.063788+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:21:20.063788+0000","last_undegraded":"2024-04-20T11:21:20.063788+0000","last_fullsized":"2024-04-20T11:21:20.063788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.225236+0000","last_change":"2024-04-20T11:20:51.225236+0000","last_active":"2024-04-20T11:20:51.225236+0000","last_peered":"2024-04-20T11:20:51.225236+0000","last_clean":"2024-04-20T11:20:51.225236+0000","last_became_active":"2024-04-20T11:20:51.221635+0000","last_became_peered":"2024-04-20T11:20:51.221635+0000","last_unstale":"2024-04-20T11:20:51.225236+0000","last_undegraded":"2024-04-20T11:20:51.225236+0000","last_fullsized":"2024-04-20T11:20:51.225236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:03.894361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.011173+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:21:20.011173+0000","last_peered":"2024-04-20T11:21:20.011173+0000","last_clean":"2024-04-20T11:21:20.011173+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:21:20.011173+0000","last_undegraded":"2024-04-20T11:21:20.011173+0000","last_fullsized":"2024-04-20T11:21:20.011173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.503397+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:20:51.503397+0000","last_peered":"2024-04-20T11:20:51.503397+0000","last_clean":"2024-04-20T11:20:51.503397+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:20:51.503397+0000","last_undegraded":"2024-04-20T11:20:51.503397+0000","last_fullsized":"2024-04-20T11:20:51.503397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.058199+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:21:20.058199+0000","last_peered":"2024-04-20T11:21:20.058199+0000","last_clean":"2024-04-20T11:21:20.058199+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:21:20.058199+0000","last_undegraded":"2024-04-20T11:21:20.058199+0000","last_fullsized":"2024-04-20T11:21:20.058199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216437+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:20:51.216437+0000","last_peered":"2024-04-20T11:20:51.216437+0000","last_clean":"2024-04-20T11:20:51.216437+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:20:51.216437+0000","last_undegraded":"2024-04-20T11:20:51.216437+0000","last_fullsized":"2024-04-20T11:20:51.216437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.063602+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:21:20.063602+0000","last_peered":"2024-04-20T11:21:20.063602+0000","last_clean":"2024-04-20T11:21:20.063602+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:21:20.063602+0000","last_undegraded":"2024-04-20T11:21:20.063602+0000","last_fullsized":"2024-04-20T11:21:20.063602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.216712+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:20:51.216712+0000","last_peered":"2024-04-20T11:20:51.216712+0000","last_clean":"2024-04-20T11:20:51.216712+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:20:51.216712+0000","last_undegraded":"2024-04-20T11:20:51.216712+0000","last_fullsized":"2024-04-20T11:20:51.216712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.058425+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:21:20.058425+0000","last_peered":"2024-04-20T11:21:20.058425+0000","last_clean":"2024-04-20T11:21:20.058425+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:21:20.058425+0000","last_undegraded":"2024-04-20T11:21:20.058425+0000","last_fullsized":"2024-04-20T11:21:20.058425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:32.022528+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:21:32.022528+0000","last_peered":"2024-04-20T11:21:32.022528+0000","last_clean":"2024-04-20T11:21:32.022528+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:21:32.022528+0000","last_undegraded":"2024-04-20T11:21:32.022528+0000","last_fullsized":"2024-04-20T11:21:32.022528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059931+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:21:20.059931+0000","last_peered":"2024-04-20T11:21:20.059931+0000","last_clean":"2024-04-20T11:21:20.059931+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:21:20.059931+0000","last_undegraded":"2024-04-20T11:21:20.059931+0000","last_fullsized":"2024-04-20T11:21:20.059931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220696+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:20:51.220696+0000","last_peered":"2024-04-20T11:20:51.220696+0000","last_clean":"2024-04-20T11:20:51.220696+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:20:51.220696+0000","last_undegraded":"2024-04-20T11:20:51.220696+0000","last_fullsized":"2024-04-20T11:20:51.220696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049226+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:21:20.049226+0000","last_peered":"2024-04-20T11:21:20.049226+0000","last_clean":"2024-04-20T11:21:20.049226+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:21:20.049226+0000","last_undegraded":"2024-04-20T11:21:20.049226+0000","last_fullsized":"2024-04-20T11:21:20.049226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:20:51.220808+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:20:51.220808+0000","last_peered":"2024-04-20T11:20:51.220808+0000","last_clean":"2024-04-20T11:20:51.220808+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:20:51.220808+0000","last_undegraded":"2024-04-20T11:20:51.220808+0000","last_fullsized":"2024-04-20T11:20:51.220808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.045396+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:21:20.045396+0000","last_peered":"2024-04-20T11:21:20.045396+0000","last_clean":"2024-04-20T11:21:20.045396+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:21:20.045396+0000","last_undegraded":"2024-04-20T11:21:20.045396+0000","last_fullsized":"2024-04-20T11:21:20.045396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.051534+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:21:20.051534+0000","last_peered":"2024-04-20T11:21:20.051534+0000","last_clean":"2024-04-20T11:21:20.051534+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:21:20.051534+0000","last_undegraded":"2024-04-20T11:21:20.051534+0000","last_fullsized":"2024-04-20T11:21:20.051534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.049866+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:21:20.049866+0000","last_peered":"2024-04-20T11:21:20.049866+0000","last_clean":"2024-04-20T11:21:20.049866+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:21:20.049866+0000","last_undegraded":"2024-04-20T11:21:20.049866+0000","last_fullsized":"2024-04-20T11:21:20.049866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.059978+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:21:20.059978+0000","last_peered":"2024-04-20T11:21:20.059978+0000","last_clean":"2024-04-20T11:21:20.059978+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:21:20.059978+0000","last_undegraded":"2024-04-20T11:21:20.059978+0000","last_fullsized":"2024-04-20T11:21:20.059978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065930+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:21:20.065930+0000","last_peered":"2024-04-20T11:21:20.065930+0000","last_clean":"2024-04-20T11:21:20.065930+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:21:20.065930+0000","last_undegraded":"2024-04-20T11:21:20.065930+0000","last_fullsized":"2024-04-20T11:21:20.065930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065500+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:21:20.065500+0000","last_peered":"2024-04-20T11:21:20.065500+0000","last_clean":"2024-04-20T11:21:20.065500+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:21:20.065500+0000","last_undegraded":"2024-04-20T11:21:20.065500+0000","last_fullsized":"2024-04-20T11:21:20.065500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.050496+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:21:20.050496+0000","last_peered":"2024-04-20T11:21:20.050496+0000","last_clean":"2024-04-20T11:21:20.050496+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:21:20.050496+0000","last_undegraded":"2024-04-20T11:21:20.050496+0000","last_fullsized":"2024-04-20T11:21:20.050496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.042323+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:21:20.042323+0000","last_peered":"2024-04-20T11:21:20.042323+0000","last_clean":"2024-04-20T11:21:20.042323+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:21:20.042323+0000","last_undegraded":"2024-04-20T11:21:20.042323+0000","last_fullsized":"2024-04-20T11:21:20.042323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.040554+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:21:20.040554+0000","last_peered":"2024-04-20T11:21:20.040554+0000","last_clean":"2024-04-20T11:21:20.040554+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:21:20.040554+0000","last_undegraded":"2024-04-20T11:21:20.040554+0000","last_fullsized":"2024-04-20T11:21:20.040554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.055847+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:21:20.055847+0000","last_peered":"2024-04-20T11:21:20.055847+0000","last_clean":"2024-04-20T11:21:20.055847+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:21:20.055847+0000","last_undegraded":"2024-04-20T11:21:20.055847+0000","last_fullsized":"2024-04-20T11:21:20.055847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.055171+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:21:20.055171+0000","last_peered":"2024-04-20T11:21:20.055171+0000","last_clean":"2024-04-20T11:21:20.055171+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:21:20.055171+0000","last_undegraded":"2024-04-20T11:21:20.055171+0000","last_fullsized":"2024-04-20T11:21:20.055171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.057265+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:21:20.057265+0000","last_peered":"2024-04-20T11:21:20.057265+0000","last_clean":"2024-04-20T11:21:20.057265+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:21:20.057265+0000","last_undegraded":"2024-04-20T11:21:20.057265+0000","last_fullsized":"2024-04-20T11:21:20.057265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.065639+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:21:20.065639+0000","last_peered":"2024-04-20T11:21:20.065639+0000","last_clean":"2024-04-20T11:21:20.065639+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:21:20.065639+0000","last_undegraded":"2024-04-20T11:21:20.065639+0000","last_fullsized":"2024-04-20T11:21:20.065639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T11:21:20.046927+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:21:20.046927+0000","last_peered":"2024-04-20T11:21:20.046927+0000","last_clean":"2024-04-20T11:21:20.046927+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:21:20.046927+0000","last_undegraded":"2024-04-20T11:21:20.046927+0000","last_fullsized":"2024-04-20T11:21:20.046927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282095,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66576,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":912,"num_read_kb":530,"num_write":244,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199728,"data_compressed":5514,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":457285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1107292,"data_compressed":37716,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070515,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1049608,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":516291,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":458159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":517043,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1048754,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34235,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60842,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61594,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61358,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:21:40.360 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-20T11:21:40.361 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T11:21:40.361 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-20T11:21:40.361 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph health --format=json 2024-04-20T11:21:40.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:40 smithi078 ceph-mon[21464]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:21:40.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:40 smithi044 ceph-mon[16855]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:21:40.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:40 smithi165 ceph-mon[21975]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:21:40.876 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:41.353 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:41 smithi044 ceph-mon[16855]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:41 smithi078 ceph-mon[21464]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:41 smithi165 ceph-mon[21975]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:43.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 -- 172.21.15.44:0/2331284825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 msgr2=0x7f9864111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 --2- 172.21.15.44:0/2331284825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 0x7f9864111b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b0a0 tx=0x7f985c00b3b0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 -- 172.21.15.44:0/2331284825 shutdown_connections 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 --2- 172.21.15.44:0/2331284825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 0x7f9864111b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 --2- 172.21.15.44:0/2331284825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864104ed0 0x7f98641052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 --2- 172.21.15.44:0/2331284825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864105880 0x7f98641098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.209+0000 7f986c600700 1 -- 172.21.15.44:0/2331284825 >> 172.21.15.44:0/2331284825 conn(0x7f9864100760 msgr2=0x7f9864102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.210+0000 7f986c600700 1 -- 172.21.15.44:0/2331284825 shutdown_connections 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.210+0000 7f986c600700 1 -- 172.21.15.44:0/2331284825 wait complete. 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.211+0000 7f986c600700 1 Processor -- start 2024-04-20T11:21:43.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.211+0000 7f986c600700 1 -- start start 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f986c600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 0x7f986419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f986c600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864105880 0x7f986419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f986c600700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 0x7f986419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f986c600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9864115130 con 0x7f9864104ed0 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f986c600700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9864114fb0 con 0x7f9864109fb0 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f986c600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98641152b0 con 0x7f9864105880 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f9869c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 0x7f986419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f9869c00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 0x7f986419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52850/0 (socket says 172.21.15.44:52850) 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f9869c00700 1 -- 172.21.15.44:0/71466964 learned_addr learned my addr 172.21.15.44:0/71466964 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:43.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.212+0000 7f9869200700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864105880 0x7f986419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f986a600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 0x7f986419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9869c00700 1 -- 172.21.15.44:0/71466964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864105880 msgr2=0x7f986419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9869c00700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864105880 0x7f986419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9869c00700 1 -- 172.21.15.44:0/71466964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 msgr2=0x7f986419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9869c00700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 0x7f986419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9869c00700 1 -- 172.21.15.44:0/71466964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98641a04d0 con 0x7f9864104ed0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9869c00700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 0x7f986419b4c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f985800e9a0 tx=0x7f985800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985800cbd0 con 0x7f9864104ed0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f985800cd70 con 0x7f9864104ed0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98641a07c0 con 0x7f9864104ed0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.213+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580187c0 con 0x7f9864104ed0 2024-04-20T11:21:43.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.214+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a8040 con 0x7f9864104ed0 2024-04-20T11:21:43.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.215+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9864069740 con 0x7f9864104ed0 2024-04-20T11:21:43.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.217+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9858010430 con 0x7f9864104ed0 2024-04-20T11:21:43.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.218+0000 7f9862000700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f984c0a3c40 0x7f984c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:43.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.219+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f98580d0650 con 0x7f9864104ed0 2024-04-20T11:21:43.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.219+0000 7f9869200700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f984c0a3c40 0x7f984c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:43.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.219+0000 7f9869200700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f984c0a3c40 0x7f984c0a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f986410b440 tx=0x7f98500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:43.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.223+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f985809d030 con 0x7f9864104ed0 2024-04-20T11:21:43.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:43 smithi044 ceph-mon[16855]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:43.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.504+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f98641a8270 con 0x7f9864104ed0 2024-04-20T11:21:43.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.505+0000 7f9862000700 1 -- 172.21.15.44:0/71466964 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f985809c930 con 0x7f9864104ed0 2024-04-20T11:21:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:43.506 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f984c0a3c40 msgr2=0x7f984c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f984c0a3c40 0x7f984c0a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f986410b440 tx=0x7f98500073d0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 msgr2=0x7f986419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 0x7f986419b4c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f985800e9a0 tx=0x7f985800ecb0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 shutdown_connections 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f984c0a3c40 0x7f984c0a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9864109fb0 0x7f986419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864104ed0 0x7f986419b4c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 --2- 172.21.15.44:0/71466964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864105880 0x7f986419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 >> 172.21.15.44:0/71466964 conn(0x7f9864100760 msgr2=0x7f9864101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 shutdown_connections 2024-04-20T11:21:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:43.508+0000 7f986c600700 1 -- 172.21.15.44:0/71466964 wait complete. 2024-04-20T11:21:43.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:43 smithi078 ceph-mon[21464]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:43 smithi165 ceph-mon[21975]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:44.482 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-20T11:21:44.482 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:21:44.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/71466964' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T11:21:44.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/71466964' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T11:21:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 -- 172.21.15.44:0/3876630311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908105970 msgr2=0x7f1908105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 --2- 172.21.15.44:0/3876630311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908105970 0x7f1908105df0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1900009ca0 tx=0x7f190002c0f0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 -- 172.21.15.44:0/3876630311 shutdown_connections 2024-04-20T11:21:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 --2- 172.21.15.44:0/3876630311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908105970 0x7f1908105df0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 --2- 172.21.15.44:0/3876630311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908102fc0 0x7f1908105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 --2- 172.21.15.44:0/3876630311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1908100650 0x7f1908102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 -- 172.21.15.44:0/3876630311 >> 172.21.15.44:0/3876630311 conn(0x7f19080fc270 msgr2=0x7f19080fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 -- 172.21.15.44:0/3876630311 shutdown_connections 2024-04-20T11:21:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.709+0000 7f190e800640 1 -- 172.21.15.44:0/3876630311 wait complete. 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.710+0000 7f190e800640 1 Processor -- start 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.710+0000 7f190e800640 1 -- start start 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 0x7f190819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1908102fc0 0x7f190819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908105970 0x7f19081a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f19081141a0 con 0x7f1908102fc0 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190e800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1908114020 con 0x7f1908100650 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1908114320 con 0x7f1908105970 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 0x7f190819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1908102fc0 0x7f190819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 0x7f190819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48870/0 (socket says 172.21.15.44:48870) 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 -- 172.21.15.44:0/4047898468 learned_addr learned my addr 172.21.15.44:0/4047898468 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190de00640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908105970 0x7f19081a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 -- 172.21.15.44:0/4047898468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908105970 msgr2=0x7f19081a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908105970 0x7f19081a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 -- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1908102fc0 msgr2=0x7f190819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1908102fc0 0x7f190819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.711+0000 7f190d400640 1 -- 172.21.15.44:0/4047898468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19081a6360 con 0x7f1908100650 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.712+0000 7f190d400640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 0x7f190819e740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00e9f0 tx=0x7f18fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.712+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00cd80 con 0x7f1908100650 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.712+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19081a6650 con 0x7f1908100650 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.712+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18fc010930 con 0x7f1908100650 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.712+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc018b00 con 0x7f1908100650 2024-04-20T11:21:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.712+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19081a6ba0 con 0x7f1908100650 2024-04-20T11:21:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.713+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19080fdd60 con 0x7f1908100650 2024-04-20T11:21:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.715+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f18fc018ca0 con 0x7f1908100650 2024-04-20T11:21:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.717+0000 7f1906000640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f18f00a3bb0 0x7f18f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.717+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f18fc0d0770 con 0x7f1908100650 2024-04-20T11:21:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.717+0000 7f190ca00640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f18f00a3bb0 0x7f18f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.718+0000 7f190ca00640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f18f00a3bb0 0x7f18f00a6070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f190819fc50 tx=0x7f18f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:44.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.721+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f18fc09cb00 con 0x7f1908100650 2024-04-20T11:21:44.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/71466964' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T11:21:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.933+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1908062b60 con 0x7f1908100650 2024-04-20T11:21:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.935+0000 7f1906000640 1 -- 172.21.15.44:0/4047898468 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22511 (secure 0 0 0) 0x7f18fc0a0e00 con 0x7f1908100650 2024-04-20T11:21:44.936 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:44.936 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:51.201931+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:21:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f18f00a3bb0 msgr2=0x7f18f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f18f00a3bb0 0x7f18f00a6070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f190819fc50 tx=0x7f18f4005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 msgr2=0x7f190819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 0x7f190819e740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00e9f0 tx=0x7f18fc00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 shutdown_connections 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f18f00a3bb0 0x7f18f00a6070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908105970 0x7f19081a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1908102fc0 0x7f190819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 --2- 172.21.15.44:0/4047898468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1908100650 0x7f190819e740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 >> 172.21.15.44:0/4047898468 conn(0x7f19080fc270 msgr2=0x7f1908100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 shutdown_connections 2024-04-20T11:21:44.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:44.938+0000 7f190e800640 1 -- 172.21.15.44:0/4047898468 wait complete. 2024-04-20T11:21:44.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:21:44.961 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:21:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 -- 172.21.15.44:0/1822366832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc102fa0 msgr2=0x7f69cc103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 --2- 172.21.15.44:0/1822366832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc102fa0 0x7f69cc103420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b402bc40 comp rx=0 tx=0).stop 2024-04-20T11:21:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 -- 172.21.15.44:0/1822366832 shutdown_connections 2024-04-20T11:21:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 --2- 172.21.15.44:0/1822366832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc103960 0x7f69cc10a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 --2- 172.21.15.44:0/1822366832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc102fa0 0x7f69cc103420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 --2- 172.21.15.44:0/1822366832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69cc101db0 0x7f69cc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.177+0000 7f69d1600640 1 -- 172.21.15.44:0/1822366832 >> 172.21.15.44:0/1822366832 conn(0x7f69cc0fd320 msgr2=0x7f69cc0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.178+0000 7f69d1600640 1 -- 172.21.15.44:0/1822366832 shutdown_connections 2024-04-20T11:21:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.178+0000 7f69d1600640 1 -- 172.21.15.44:0/1822366832 wait complete. 2024-04-20T11:21:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.179+0000 7f69d1600640 1 Processor -- start 2024-04-20T11:21:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.179+0000 7f69d1600640 1 -- start start 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.179+0000 7f69d1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 0x7f69cc075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.179+0000 7f69d1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 0x7f69cc074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.179+0000 7f69d1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69cc103960 0x7f69cc074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.179+0000 7f69d1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc10db60 con 0x7f69cc102fa0 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69d1600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc10d9e0 con 0x7f69cc103960 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69d1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc10dce0 con 0x7f69cc101db0 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 0x7f69cc074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 0x7f69cc075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 0x7f69cc074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52896/0 (socket says 172.21.15.44:52896) 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 -- 172.21.15.44:0/2228884005 learned_addr learned my addr 172.21.15.44:0/2228884005 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 0x7f69cc075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:36508/0 (socket says 172.21.15.44:36508) 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69d0c00640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69cc103960 0x7f69cc074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 -- 172.21.15.44:0/2228884005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 msgr2=0x7f69cc075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 0x7f69cc075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 -- 172.21.15.44:0/2228884005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69cc103960 msgr2=0x7f69cc074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69cc103960 0x7f69cc074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69c3400640 1 -- 172.21.15.44:0/2228884005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc075120 con 0x7f69cc102fa0 2024-04-20T11:21:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.180+0000 7f69cbe00640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 0x7f69cc075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.181+0000 7f69c3400640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 0x7f69cc074350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000c00 tx=0x7f69b402c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.181+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4043020 con 0x7f69cc102fa0 2024-04-20T11:21:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.181+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f69b40042d0 con 0x7f69cc102fa0 2024-04-20T11:21:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.181+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc1a2290 con 0x7f69cc102fa0 2024-04-20T11:21:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.181+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4003c50 con 0x7f69cc102fa0 2024-04-20T11:21:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.181+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc1a27e0 con 0x7f69cc102fa0 2024-04-20T11:21:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.182+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69cc109070 con 0x7f69cc102fa0 2024-04-20T11:21:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.184+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f69b4046020 con 0x7f69cc102fa0 2024-04-20T11:21:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.186+0000 7f69c9600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69b00a3bb0 0x7f69b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.186+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f69b40f24f0 con 0x7f69cc102fa0 2024-04-20T11:21:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.186+0000 7f69cbe00640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69b00a3bb0 0x7f69b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.187+0000 7f69cbe00640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69b00a3bb0 0x7f69b00a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69cc102e00 tx=0x7f69bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:45.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.191+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f69b404d020 con 0x7f69cc102fa0 2024-04-20T11:21:45.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.404+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69cc062b60 con 0x7f69cc102fa0 2024-04-20T11:21:45.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.405+0000 7f69c9600640 1 -- 172.21.15.44:0/2228884005 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f69b40beed0 con 0x7f69cc102fa0 2024-04-20T11:21:45.406 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:21:45.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.407+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69b00a3bb0 msgr2=0x7f69b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.407+0000 7f69d1600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69b00a3bb0 0x7f69b00a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69cc102e00 tx=0x7f69bc005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:45.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.407+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 msgr2=0x7f69cc074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.407+0000 7f69d1600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 0x7f69cc074350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000c00 tx=0x7f69b402c730 comp rx=0 tx=0).stop 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 shutdown_connections 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f69b00a3bb0 0x7f69b00a6070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69cc103960 0x7f69cc074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69cc102fa0 0x7f69cc074350 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 --2- 172.21.15.44:0/2228884005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69cc101db0 0x7f69cc075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 >> 172.21.15.44:0/2228884005 conn(0x7f69cc0fd320 msgr2=0x7f69cc106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 shutdown_connections 2024-04-20T11:21:45.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.408+0000 7f69d1600640 1 -- 172.21.15.44:0/2228884005 wait complete. 2024-04-20T11:21:45.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:21:45.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 -- 172.21.15.44:0/2506140035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 msgr2=0x7f9c281044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2506140035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c281044c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009900 tx=0x7f9c1802bc40 comp rx=0 tx=0).stop 2024-04-20T11:21:45.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 -- 172.21.15.44:0/2506140035 shutdown_connections 2024-04-20T11:21:45.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2506140035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105c50 0x7f9c2810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2506140035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105290 0x7f9c28105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2506140035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c281044c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 -- 172.21.15.44:0/2506140035 >> 172.21.15.44:0/2506140035 conn(0x7f9c280ff610 msgr2=0x7f9c28101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 -- 172.21.15.44:0/2506140035 shutdown_connections 2024-04-20T11:21:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.645+0000 7f9c27e00640 1 -- 172.21.15.44:0/2506140035 wait complete. 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.646+0000 7f9c27e00640 1 Processor -- start 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.646+0000 7f9c27e00640 1 -- start start 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c2819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105290 0x7f9c2819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105c50 0x7f9c281a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810fe50 con 0x7f9c281040a0 2024-04-20T11:21:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810fcd0 con 0x7f9c28105c50 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810ffd0 con 0x7f9c28105290 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c2819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c2819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52902/0 (socket says 172.21.15.44:52902) 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c26a00640 1 -- 172.21.15.44:0/2792562622 learned_addr learned my addr 172.21.15.44:0/2792562622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27400640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105c50 0x7f9c281a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c1fe00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105290 0x7f9c2819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27400640 1 -- 172.21.15.44:0/2792562622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105290 msgr2=0x7f9c2819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27400640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105290 0x7f9c2819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27400640 1 -- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 msgr2=0x7f9c2819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27400640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c2819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c27400640 1 -- 172.21.15.44:0/2792562622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c281a2060 con 0x7f9c28105c50 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c26a00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c2819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.647+0000 7f9c1fe00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105290 0x7f9c2819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c27400640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105c50 0x7f9c281a19f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000e940 tx=0x7f9c2000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004d20 con 0x7f9c28105c50 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c2000cb20 con 0x7f9c28105c50 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c281a2350 con 0x7f9c28105c50 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200116b0 con 0x7f9c28105c50 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.648+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281a28f0 con 0x7f9c28105c50 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.649+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2810b360 con 0x7f9c28105c50 2024-04-20T11:21:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.651+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9c20005000 con 0x7f9c28105c50 2024-04-20T11:21:45.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.652+0000 7f9c1f400640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9c0c0a3c30 0x7f9c0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:45.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.653+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f9c200d0450 con 0x7f9c28105c50 2024-04-20T11:21:45.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.653+0000 7f9c26a00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9c0c0a3c30 0x7f9c0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:45.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.653+0000 7f9c26a00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9c0c0a3c30 0x7f9c0c0a60f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c1802ca40 tx=0x7f9c1802dea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:45.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.658+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9c20016030 con 0x7f9c28105c50 2024-04-20T11:21:45.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:45 smithi078 ceph-mon[21464]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:45.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:45 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4047898468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:45.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:45 smithi044 ceph-mon[16855]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:45.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:45 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4047898468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:45 smithi165 ceph-mon[21975]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:45 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4047898468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.868+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c28062b60 con 0x7f9c28105c50 2024-04-20T11:21:45.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.869+0000 7f9c1f400640 1 -- 172.21.15.44:0/2792562622 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f9c2009ce30 con 0x7f9c28105c50 2024-04-20T11:21:45.869 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:21:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9c0c0a3c30 msgr2=0x7f9c0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9c0c0a3c30 0x7f9c0c0a60f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c1802ca40 tx=0x7f9c1802dea0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105c50 msgr2=0x7f9c281a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105c50 0x7f9c281a19f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000e940 tx=0x7f9c2000ee00 comp rx=0 tx=0).stop 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 shutdown_connections 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f9c0c0a3c30 0x7f9c0c0a60f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c28105c50 0x7f9c281a19f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c28105290 0x7f9c2819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 --2- 172.21.15.44:0/2792562622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c281040a0 0x7f9c2819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 >> 172.21.15.44:0/2792562622 conn(0x7f9c280ff610 msgr2=0x7f9c28108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 shutdown_connections 2024-04-20T11:21:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:45.872+0000 7f9c27e00640 1 -- 172.21.15.44:0/2792562622 wait complete. 2024-04-20T11:21:45.894 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:21:46.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.109+0000 7fabf7e00640 1 -- 172.21.15.44:0/2983336855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 msgr2=0x7fabf8103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.109+0000 7fabf7e00640 1 --2- 172.21.15.44:0/2983336855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 0x7fabf8103410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009900 tx=0x7fabe002bc40 comp rx=0 tx=0).stop 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 -- 172.21.15.44:0/2983336855 shutdown_connections 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 --2- 172.21.15.44:0/2983336855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8103950 0x7fabf810a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 --2- 172.21.15.44:0/2983336855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 0x7fabf8103410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 --2- 172.21.15.44:0/2983336855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8101da0 0x7fabf81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 -- 172.21.15.44:0/2983336855 >> 172.21.15.44:0/2983336855 conn(0x7fabf80fd310 msgr2=0x7fabf80ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 -- 172.21.15.44:0/2983336855 shutdown_connections 2024-04-20T11:21:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.110+0000 7fabf7e00640 1 -- 172.21.15.44:0/2983336855 wait complete. 2024-04-20T11:21:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 Processor -- start 2024-04-20T11:21:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 -- start start 2024-04-20T11:21:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 0x7fabf8075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 0x7fabf8074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8103950 0x7fabf8074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fabf810db50 con 0x7fabf8101da0 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabf810d9d0 con 0x7fabf8103950 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.111+0000 7fabf7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fabf810dcd0 con 0x7fabf8102f90 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 0x7fabf8075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 0x7fabf8074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 0x7fabf8075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52916/0 (socket says 172.21.15.44:52916) 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 -- 172.21.15.44:0/106338971 learned_addr learned my addr 172.21.15.44:0/106338971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf7400640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8103950 0x7fabf8074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 -- 172.21.15.44:0/106338971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 msgr2=0x7fabf8074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 0x7fabf8074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 -- 172.21.15.44:0/106338971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8103950 msgr2=0x7fabf8074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8103950 0x7fabf8074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 -- 172.21.15.44:0/106338971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf8075120 con 0x7fabf8101da0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf7400640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8103950 0x7fabf8074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.112+0000 7fabf6a00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 0x7fabf8075d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabe800e9c0 tx=0x7fabe800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.113+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe800cd80 con 0x7fabf8101da0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.113+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf81a2230 con 0x7fabf8101da0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.113+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabe80035f0 con 0x7fabf8101da0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.113+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8010690 con 0x7fabf8101da0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.113+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf81a2750 con 0x7fabf8101da0 2024-04-20T11:21:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.114+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf8108ff0 con 0x7fabf8101da0 2024-04-20T11:21:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.116+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fabe80037a0 con 0x7fabf8101da0 2024-04-20T11:21:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.117+0000 7fabef400640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fabdc0a3bb0 0x7fabdc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.118+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fabe80d0780 con 0x7fabf8101da0 2024-04-20T11:21:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.118+0000 7fabf6000640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fabdc0a3bb0 0x7fabdc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.119+0000 7fabf6000640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fabdc0a3bb0 0x7fabdc0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fabe002c910 tx=0x7fabe0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.122+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fabe809d200 con 0x7fabf8101da0 2024-04-20T11:21:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.330+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fabf8062b60 con 0x7fabf8101da0 2024-04-20T11:21:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.331+0000 7fabef400640 1 -- 172.21.15.44:0/106338971 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fabe809cb00 con 0x7fabf8101da0 2024-04-20T11:21:46.332 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fabdc0a3bb0 msgr2=0x7fabdc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fabdc0a3bb0 0x7fabdc0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fabe002c910 tx=0x7fabe0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 msgr2=0x7fabf8075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 0x7fabf8075d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabe800e9c0 tx=0x7fabe800ee80 comp rx=0 tx=0).stop 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 shutdown_connections 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fabdc0a3bb0 0x7fabdc0a6070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabf8103950 0x7fabf8074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf8102f90 0x7fabf8074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 --2- 172.21.15.44:0/106338971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabf8101da0 0x7fabf8075d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 >> 172.21.15.44:0/106338971 conn(0x7fabf80fd310 msgr2=0x7fabf81061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.334+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 shutdown_connections 2024-04-20T11:21:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.335+0000 7fabf7e00640 1 -- 172.21.15.44:0/106338971 wait complete. 2024-04-20T11:21:46.356 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 -- 172.21.15.44:0/2308293367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c101dd0 msgr2=0x7f958c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 --2- 172.21.15.44:0/2308293367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c101dd0 0x7f958c1021f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9580009860 tx=0x7f958002bb70 comp rx=0 tx=0).stop 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 -- 172.21.15.44:0/2308293367 shutdown_connections 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 --2- 172.21.15.44:0/2308293367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f958c103980 0x7f958c10a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 --2- 172.21.15.44:0/2308293367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c102fc0 0x7f958c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 --2- 172.21.15.44:0/2308293367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c101dd0 0x7f958c1021f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 -- 172.21.15.44:0/2308293367 >> 172.21.15.44:0/2308293367 conn(0x7f958c0fd340 msgr2=0x7f958c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 -- 172.21.15.44:0/2308293367 shutdown_connections 2024-04-20T11:21:46.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.565+0000 7f9592800640 1 -- 172.21.15.44:0/2308293367 wait complete. 2024-04-20T11:21:46.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 Processor -- start 2024-04-20T11:21:46.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 -- start start 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 0x7f958c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c102fc0 0x7f958c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f958c103980 0x7f958c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f958c10db60 con 0x7f958c101dd0 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f958c10d9e0 con 0x7f958c102fc0 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9592800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f958c10dce0 con 0x7f958c103980 2024-04-20T11:21:46.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.567+0000 7f9591400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 0x7f958c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9590a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c102fc0 0x7f958c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 0x7f958c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52934/0 (socket says 172.21.15.44:52934) 2024-04-20T11:21:46.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 -- 172.21.15.44:0/2154904978 learned_addr learned my addr 172.21.15.44:0/2154904978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:46.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591e00640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f958c103980 0x7f958c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 -- 172.21.15.44:0/2154904978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f958c103980 msgr2=0x7f958c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f958c103980 0x7f958c1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 -- 172.21.15.44:0/2154904978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c102fc0 msgr2=0x7f958c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c102fc0 0x7f958c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 -- 172.21.15.44:0/2154904978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c1a2090 con 0x7f958c101dd0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.568+0000 7f9591400640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 0x7f958c19a470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9580034a80 tx=0x7f9580005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.569+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580041020 con 0x7f958c101dd0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.569+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f958002ca10 con 0x7f958c101dd0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.569+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c1a2320 con 0x7f958c101dd0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.569+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580039040 con 0x7f958c101dd0 2024-04-20T11:21:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.569+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c1a2810 con 0x7f958c101dd0 2024-04-20T11:21:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.570+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c1a2b30 con 0x7f958c101dd0 2024-04-20T11:21:46.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.572+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9580044020 con 0x7f958c101dd0 2024-04-20T11:21:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.574+0000 7f958a000640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f95740a3bb0 0x7f95740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.574+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f95800f1ff0 con 0x7f958c101dd0 2024-04-20T11:21:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.574+0000 7f9590a00640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f95740a3bb0 0x7f95740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:46.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.575+0000 7f9590a00640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f95740a3bb0 0x7f95740a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f958c19b980 tx=0x7f9578005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:46.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.576+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f958004b020 con 0x7f958c101dd0 2024-04-20T11:21:46.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:46 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2228884005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:21:46.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:46 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2792562622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:21:46.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:46 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2228884005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:21:46.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:46 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2792562622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:21:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:46 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2228884005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:21:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:46 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2792562622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:21:46.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.835+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f958c1a2d00 con 0x7f958c101dd0 2024-04-20T11:21:46.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.837+0000 7f958a000640 1 -- 172.21.15.44:0/2154904978 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f95800be9f0 con 0x7f958c101dd0 2024-04-20T11:21:46.837 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:46.838 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:20:09.840804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:21:46.838 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:21:46.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.839+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f95740a3bb0 msgr2=0x7f95740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.839+0000 7f9592800640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f95740a3bb0 0x7f95740a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f958c19b980 tx=0x7f9578005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:46.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.839+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 msgr2=0x7f958c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:46.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.839+0000 7f9592800640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 0x7f958c19a470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9580034a80 tx=0x7f9580005dc0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 shutdown_connections 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f95740a3bb0 0x7f95740a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f958c103980 0x7f958c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f958c102fc0 0x7f958c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 --2- 172.21.15.44:0/2154904978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f958c101dd0 0x7f958c19a470 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 >> 172.21.15.44:0/2154904978 conn(0x7f958c0fd340 msgr2=0x7f958c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 shutdown_connections 2024-04-20T11:21:46.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:46.840+0000 7f9592800640 1 -- 172.21.15.44:0/2154904978 wait complete. 2024-04-20T11:21:46.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:21:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 -- 172.21.15.44:0/3233598986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac105d70 msgr2=0x7ffbac10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/3233598986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac105d70 0x7ffbac10c600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009ca0 tx=0x7ffba402c100 comp rx=0 tx=0).stop 2024-04-20T11:21:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 -- 172.21.15.44:0/3233598986 shutdown_connections 2024-04-20T11:21:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/3233598986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac105d70 0x7ffbac10c600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/3233598986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbac1053b0 0x7ffbac105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/3233598986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 -- 172.21.15.44:0/3233598986 >> 172.21.15.44:0/3233598986 conn(0x7ffbac0ff750 msgr2=0x7ffbac101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 -- 172.21.15.44:0/3233598986 shutdown_connections 2024-04-20T11:21:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.080+0000 7ffbb2e00640 1 -- 172.21.15.44:0/3233598986 wait complete. 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.081+0000 7ffbb2e00640 1 Processor -- start 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.081+0000 7ffbb2e00640 1 -- start start 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb2e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac1053b0 0x7ffbac19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb2e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbac105d70 0x7ffbac1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac10ff60 con 0x7ffbac1041c0 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb2e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac10fde0 con 0x7ffbac1053b0 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb2e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1100e0 con 0x7ffbac105d70 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52960/0 (socket says 172.21.15.44:52960) 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 -- 172.21.15.44:0/761077223 learned_addr learned my addr 172.21.15.44:0/761077223 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbab400640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac1053b0 0x7ffbac19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbb0e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbac105d70 0x7ffbac1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 -- 172.21.15.44:0/761077223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbac105d70 msgr2=0x7ffbac1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbac105d70 0x7ffbac1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 -- 172.21.15.44:0/761077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac1053b0 msgr2=0x7ffbac19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac1053b0 0x7ffbac19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.082+0000 7ffbabe00640 1 -- 172.21.15.44:0/761077223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac1a2160 con 0x7ffbac1041c0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.083+0000 7ffbabe00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac19a540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9c0 tx=0x7ffb9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.083+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00cd80 con 0x7ffbac1041c0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.083+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac1a2450 con 0x7ffbac1041c0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.083+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffb9c0035f0 con 0x7ffbac1041c0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.083+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c010690 con 0x7ffbac1041c0 2024-04-20T11:21:47.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.083+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac1a29a0 con 0x7ffbac1041c0 2024-04-20T11:21:47.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.084+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbac10b410 con 0x7ffbac1041c0 2024-04-20T11:21:47.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.086+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffb9c0037a0 con 0x7ffbac1041c0 2024-04-20T11:21:47.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.088+0000 7ffba8c00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ffb900a3bb0 0x7ffb900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.088+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7ffb9c0d0810 con 0x7ffbac1041c0 2024-04-20T11:21:47.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.088+0000 7ffbab400640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ffb900a3bb0 0x7ffb900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.089+0000 7ffbab400640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ffb900a3bb0 0x7ffb900a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb980095f0 tx=0x7ffb98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:47.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.092+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffb9c09d1f0 con 0x7ffbac1041c0 2024-04-20T11:21:47.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.302+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbac062b90 con 0x7ffbac1041c0 2024-04-20T11:21:47.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.306+0000 7ffba8c00640 1 -- 172.21.15.44:0/761077223 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22511 (secure 0 0 0) 0x7ffb9c09caf0 con 0x7ffbac1041c0 2024-04-20T11:21:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:47.308 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:51.201931+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:21:47.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.309+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ffb900a3bb0 msgr2=0x7ffb900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.309+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ffb900a3bb0 0x7ffb900a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb980095f0 tx=0x7ffb98005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:47.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.309+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 msgr2=0x7ffbac19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.309+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac19a540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9c0 tx=0x7ffb9c00ee80 comp rx=0 tx=0).stop 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 shutdown_connections 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7ffb900a3bb0 0x7ffb900a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbac105d70 0x7ffbac1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbac1053b0 0x7ffbac19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 --2- 172.21.15.44:0/761077223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbac1041c0 0x7ffbac19a540 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 >> 172.21.15.44:0/761077223 conn(0x7ffbac0ff750 msgr2=0x7ffbac1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 shutdown_connections 2024-04-20T11:21:47.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.310+0000 7ffbb2e00640 1 -- 172.21.15.44:0/761077223 wait complete. 2024-04-20T11:21:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:21:47.332 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:21:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 -- 172.21.15.44:0/817581596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 msgr2=0x7fed6c1043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 --2- 172.21.15.44:0/817581596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c1043e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fed60009900 tx=0x7fed6002bc30 comp rx=0 tx=0).stop 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 -- 172.21.15.44:0/817581596 shutdown_connections 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 --2- 172.21.15.44:0/817581596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c105b70 0x7fed6c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 --2- 172.21.15.44:0/817581596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c1051b0 0x7fed6c105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 --2- 172.21.15.44:0/817581596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c1043e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 -- 172.21.15.44:0/817581596 >> 172.21.15.44:0/817581596 conn(0x7fed6c0ff530 msgr2=0x7fed6c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 -- 172.21.15.44:0/817581596 shutdown_connections 2024-04-20T11:21:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.542+0000 7fed72a00640 1 -- 172.21.15.44:0/817581596 wait complete. 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 Processor -- start 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 -- start start 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c1051b0 0x7fed6c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c105b70 0x7fed6c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c10fd20 con 0x7fed6c105b70 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c10fba0 con 0x7fed6c1051b0 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.543+0000 7fed72a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c10fea0 con 0x7fed6c103fc0 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed71600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed71600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:36594/0 (socket says 172.21.15.44:36594) 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed71600640 1 -- 172.21.15.44:0/3769046082 learned_addr learned my addr 172.21.15.44:0/3769046082 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed72000640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c105b70 0x7fed6c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed70c00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c1051b0 0x7fed6c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed72000640 1 -- 172.21.15.44:0/3769046082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 msgr2=0x7fed6c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed72000640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed72000640 1 -- 172.21.15.44:0/3769046082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c1051b0 msgr2=0x7fed6c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed72000640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c1051b0 0x7fed6c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.544+0000 7fed72000640 1 -- 172.21.15.44:0/3769046082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c1a1f80 con 0x7fed6c105b70 2024-04-20T11:21:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.545+0000 7fed72000640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c105b70 0x7fed6c1a1910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed64009fd0 tx=0x7fed6400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:47.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.545+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6400ce20 con 0x7fed6c105b70 2024-04-20T11:21:47.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.545+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c1a2270 con 0x7fed6c105b70 2024-04-20T11:21:47.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.545+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fed64004b20 con 0x7fed6c105b70 2024-04-20T11:21:47.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.545+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64010790 con 0x7fed6c105b70 2024-04-20T11:21:47.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.545+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c1a2810 con 0x7fed6c105b70 2024-04-20T11:21:47.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.546+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6c10b280 con 0x7fed6c105b70 2024-04-20T11:21:47.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.548+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fed64004cd0 con 0x7fed6c105b70 2024-04-20T11:21:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.550+0000 7fed6a000640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fed540a3bb0 0x7fed540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:47.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.550+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fed640d0b30 con 0x7fed6c105b70 2024-04-20T11:21:47.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.550+0000 7fed71600640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fed540a3bb0 0x7fed540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:47.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.551+0000 7fed71600640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fed540a3bb0 0x7fed540a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed6002ca30 tx=0x7fed60009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:47.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.554+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fed6409d510 con 0x7fed6c105b70 2024-04-20T11:21:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:47 smithi078 ceph-mon[21464]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:47 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/106338971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:21:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:47 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2154904978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:47 smithi044 ceph-mon[16855]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:47 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/106338971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:21:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:47 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2154904978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:47 smithi165 ceph-mon[21975]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:47 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/106338971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:21:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:47 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2154904978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:47.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.761+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed6c02d5d0 con 0x7fed6c105b70 2024-04-20T11:21:47.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.762+0000 7fed6a000640 1 -- 172.21.15.44:0/3769046082 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fed6409ce10 con 0x7fed6c105b70 2024-04-20T11:21:47.763 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fed540a3bb0 msgr2=0x7fed540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fed540a3bb0 0x7fed540a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed6002ca30 tx=0x7fed60009560 comp rx=0 tx=0).stop 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c105b70 msgr2=0x7fed6c1a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c105b70 0x7fed6c1a1910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed64009fd0 tx=0x7fed6400ef10 comp rx=0 tx=0).stop 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 shutdown_connections 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fed540a3bb0 0x7fed540a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed6c105b70 0x7fed6c1a1910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c1051b0 0x7fed6c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 --2- 172.21.15.44:0/3769046082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed6c103fc0 0x7fed6c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 >> 172.21.15.44:0/3769046082 conn(0x7fed6c0ff530 msgr2=0x7fed6c108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 shutdown_connections 2024-04-20T11:21:47.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:47.765+0000 7fed72a00640 1 -- 172.21.15.44:0/3769046082 wait complete. 2024-04-20T11:21:47.785 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 -- 172.21.15.44:0/4146155013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 msgr2=0x7f12701022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 --2- 172.21.15.44:0/4146155013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f12701022b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1260009900 tx=0x7f126002bc30 comp rx=0 tx=0).stop 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 -- 172.21.15.44:0/4146155013 shutdown_connections 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 --2- 172.21.15.44:0/4146155013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 0x7f127010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 --2- 172.21.15.44:0/4146155013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1270102fe0 0x7f1270103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 --2- 172.21.15.44:0/4146155013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f12701022b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 -- 172.21.15.44:0/4146155013 >> 172.21.15.44:0/4146155013 conn(0x7f12700fd400 msgr2=0x7f12700ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:48.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 -- 172.21.15.44:0/4146155013 shutdown_connections 2024-04-20T11:21:48.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.003+0000 7f1275c00640 1 -- 172.21.15.44:0/4146155013 wait complete. 2024-04-20T11:21:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.004+0000 7f1275c00640 1 Processor -- start 2024-04-20T11:21:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.004+0000 7f1275c00640 1 -- start start 2024-04-20T11:21:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.004+0000 7f1275c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f127019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1270102fe0 0x7f127019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 0x7f12701a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f127010dc00 con 0x7f12701039a0 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f127010da80 con 0x7f1270101e90 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f127010dd80 con 0x7f1270102fe0 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f127019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 0x7f12701a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f127019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:49014/0 (socket says 172.21.15.44:49014) 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 -- 172.21.15.44:0/288487266 learned_addr learned my addr 172.21.15.44:0/288487266 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126f400640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1270102fe0 0x7f127019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 -- 172.21.15.44:0/288487266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1270102fe0 msgr2=0x7f127019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1270102fe0 0x7f127019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 -- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 msgr2=0x7f12701a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 0x7f12701a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f126fe00640 1 -- 172.21.15.44:0/288487266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12701a2100 con 0x7f1270101e90 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.005+0000 7f1275200640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 0x7f12701a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.006+0000 7f126fe00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f127019a580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1260009230 tx=0x7f1260004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.006+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260043020 con 0x7f1270101e90 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.006+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12701a2390 con 0x7f1270101e90 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.006+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1260004250 con 0x7f1270101e90 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.006+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260003be0 con 0x7f1270101e90 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.006+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12701a28e0 con 0x7f1270101e90 2024-04-20T11:21:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.007+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12701a2ae0 con 0x7f1270101e90 2024-04-20T11:21:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.010+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1260046020 con 0x7f1270101e90 2024-04-20T11:21:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.011+0000 7f126cc00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f12540a3bb0 0x7f12540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.011+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f12600f2cf0 con 0x7f1270101e90 2024-04-20T11:21:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.011+0000 7f126f400640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f12540a3bb0 0x7f12540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.012+0000 7f126f400640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f12540a3bb0 0x7f12540a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f127019b9f0 tx=0x7f125c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.013+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f126004d020 con 0x7f1270101e90 2024-04-20T11:21:48.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.218+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12701a2cb0 con 0x7f1270101e90 2024-04-20T11:21:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.219+0000 7f126cc00640 1 -- 172.21.15.44:0/288487266 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f12600bf6d0 con 0x7f1270101e90 2024-04-20T11:21:48.219 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:21:48.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.221+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f12540a3bb0 msgr2=0x7f12540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.221+0000 7f1275c00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f12540a3bb0 0x7f12540a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f127019b9f0 tx=0x7f125c009500 comp rx=0 tx=0).stop 2024-04-20T11:21:48.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 msgr2=0x7f127019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f127019a580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1260009230 tx=0x7f1260004ab0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 shutdown_connections 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f12540a3bb0 0x7f12540a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12701039a0 0x7f12701a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1270102fe0 0x7f127019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 --2- 172.21.15.44:0/288487266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1270101e90 0x7f127019a580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 >> 172.21.15.44:0/288487266 conn(0x7f12700fd400 msgr2=0x7f1270074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 shutdown_connections 2024-04-20T11:21:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.222+0000 7f1275c00640 1 -- 172.21.15.44:0/288487266 wait complete. 2024-04-20T11:21:48.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:21:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 -- 172.21.15.44:0/1819121406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 msgr2=0x7f6ea81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 --2- 172.21.15.44:0/1819121406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea81087e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009900 tx=0x7f6e9802bc30 comp rx=0 tx=0).stop 2024-04-20T11:21:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 -- 172.21.15.44:0/1819121406 shutdown_connections 2024-04-20T11:21:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 --2- 172.21.15.44:0/1819121406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea8109f70 0x7f6ea8110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 --2- 172.21.15.44:0/1819121406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea81095b0 0x7f6ea8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 --2- 172.21.15.44:0/1819121406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea81087e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.450+0000 7f6ead000640 1 -- 172.21.15.44:0/1819121406 >> 172.21.15.44:0/1819121406 conn(0x7f6ea8077360 msgr2=0x7f6ea80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.451+0000 7f6ead000640 1 -- 172.21.15.44:0/1819121406 shutdown_connections 2024-04-20T11:21:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.451+0000 7f6ead000640 1 -- 172.21.15.44:0/1819121406 wait complete. 2024-04-20T11:21:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 Processor -- start 2024-04-20T11:21:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 -- start start 2024-04-20T11:21:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea81095b0 0x7f6ea819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea8109f70 0x7f6ea81a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8114110 con 0x7f6ea81083c0 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8113f90 con 0x7f6ea8109f70 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ead000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8114290 con 0x7f6ea81095b0 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.452+0000 7f6ea7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea8109f70 0x7f6ea81a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea81095b0 0x7f6ea819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6ea7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54532/0 (socket says 172.21.15.44:54532) 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea8109f70 0x7f6ea81a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53816/0 (socket says 172.21.15.44:53816) 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6ea7400640 1 -- 172.21.15.44:0/3251370436 learned_addr learned my addr 172.21.15.44:0/3251370436 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 -- 172.21.15.44:0/3251370436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea8109f70 msgr2=0x7f6ea81a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea8109f70 0x7f6ea81a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 -- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 msgr2=0x7f6ea819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 -- 172.21.15.44:0/3251370436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81a63d0 con 0x7f6ea81095b0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6ea7400640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.453+0000 7f6e9ea00640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea81095b0 0x7f6ea819ecf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e9f0 tx=0x7f6e9000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.454+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000ccc0 con 0x7f6ea81095b0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.454+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6e90010930 con 0x7f6ea81095b0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.454+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90018920 con 0x7f6ea81095b0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.454+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81a66c0 con 0x7f6ea81095b0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.454+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a6c60 con 0x7f6ea81095b0 2024-04-20T11:21:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.455+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea810f760 con 0x7f6ea81095b0 2024-04-20T11:21:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.457+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6e90010ae0 con 0x7f6ea81095b0 2024-04-20T11:21:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.458+0000 7f6ea4c00640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.458+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f6e900d05d0 con 0x7f6ea81095b0 2024-04-20T11:21:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.459+0000 7f6ea7400640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.459+0000 7f6ea7400640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e9802ca30 tx=0x7f6e9802de90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:48.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.464+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6e9009cfb0 con 0x7f6ea81095b0 2024-04-20T11:21:48.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.669+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ea8062b60 con 0x7f6ea81095b0 2024-04-20T11:21:48.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.670+0000 7f6ea4c00640 1 -- 172.21.15.44:0/3251370436 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f6e9009c8b0 con 0x7f6ea81095b0 2024-04-20T11:21:48.671 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:21:48.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6e8c0a3c30 msgr2=0x7f6e8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e9802ca30 tx=0x7f6e9802de90 comp rx=0 tx=0).stop 2024-04-20T11:21:48.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea81095b0 msgr2=0x7f6ea819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea81095b0 0x7f6ea819ecf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e9f0 tx=0x7f6e9000edf0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 shutdown_connections 2024-04-20T11:21:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f6e8c0a3c30 0x7f6e8c0a60f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ea8109f70 0x7f6ea81a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ea81095b0 0x7f6ea819ecf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 --2- 172.21.15.44:0/3251370436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ea81083c0 0x7f6ea819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 >> 172.21.15.44:0/3251370436 conn(0x7f6ea8077360 msgr2=0x7f6ea810c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:48.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 shutdown_connections 2024-04-20T11:21:48.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.673+0000 7f6ead000640 1 -- 172.21.15.44:0/3251370436 wait complete. 2024-04-20T11:21:48.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:48 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/761077223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:48.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:48 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3769046082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:21:48.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:48 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/288487266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:21:48.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:48 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/761077223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:48.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:48 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3769046082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:21:48.690 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:48 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/288487266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:21:48.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:21:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:48 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/761077223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:48 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3769046082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:21:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:48 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/288487266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 -- 172.21.15.44:0/1035260182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 msgr2=0x7fcdb0109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 --2- 172.21.15.44:0/1035260182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 0x7fcdb0109f80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009ca0 tx=0x7fcda802c100 comp rx=0 tx=0).stop 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 -- 172.21.15.44:0/1035260182 shutdown_connections 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 --2- 172.21.15.44:0/1035260182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 0x7fcdb0109f80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 --2- 172.21.15.44:0/1035260182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb0105200 0x7fcdb01075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 --2- 172.21.15.44:0/1035260182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb01028d0 0x7fcdb0104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 -- 172.21.15.44:0/1035260182 >> 172.21.15.44:0/1035260182 conn(0x7fcdb00fc300 msgr2=0x7fcdb00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 -- 172.21.15.44:0/1035260182 shutdown_connections 2024-04-20T11:21:48.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.888+0000 7fcdafe00640 1 -- 172.21.15.44:0/1035260182 wait complete. 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.889+0000 7fcdafe00640 1 Processor -- start 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.889+0000 7fcdafe00640 1 -- start start 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 0x7fcdb01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb0105200 0x7fcdb0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 0x7fcdb019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdafe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb010dbb0 con 0x7fcdb0105200 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdafe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb010da30 con 0x7fcdb0107b30 2024-04-20T11:21:48.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdafe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb010dd30 con 0x7fcdb01028d0 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdae000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb0105200 0x7fcdb0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdaea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 0x7fcdb01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdaea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 0x7fcdb01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42774/0 (socket says 172.21.15.44:42774) 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdaea00640 1 -- 172.21.15.44:0/2381522377 learned_addr learned my addr 172.21.15.44:0/2381522377 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.890+0000 7fcdaf400640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 0x7fcdb019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcdaea00640 1 -- 172.21.15.44:0/2381522377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 msgr2=0x7fcdb019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcdaea00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 0x7fcdb019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcdaea00640 1 -- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb0105200 msgr2=0x7fcdb0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcdaea00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb0105200 0x7fcdb0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:48.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcdaea00640 1 -- 172.21.15.44:0/2381522377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb019fdf0 con 0x7fcdb01028d0 2024-04-20T11:21:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcdaea00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 0x7fcdb01981d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcda000e9f0 tx=0x7fcda000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.891+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda000cd80 con 0x7fcdb01028d0 2024-04-20T11:21:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.892+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb01a00e0 con 0x7fcdb01028d0 2024-04-20T11:21:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.892+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcda0010930 con 0x7fcdb01028d0 2024-04-20T11:21:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.892+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0018b00 con 0x7fcdb01028d0 2024-04-20T11:21:48.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.892+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb01a0680 con 0x7fcdb01028d0 2024-04-20T11:21:48.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.894+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb00fddf0 con 0x7fcdb01028d0 2024-04-20T11:21:48.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.895+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcda0018ca0 con 0x7fcdb01028d0 2024-04-20T11:21:48.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.897+0000 7fcda7400640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcd940a3c30 0x7fcd940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:48.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.897+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fcda00d0720 con 0x7fcdb01028d0 2024-04-20T11:21:48.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.897+0000 7fcdae000640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcd940a3c30 0x7fcd940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:48.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.897+0000 7fcdae000640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcd940a3c30 0x7fcd940a60f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcdb01996e0 tx=0x7fcd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:48.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:48.901+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcda009cab0 con 0x7fcdb01028d0 2024-04-20T11:21:49.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.148+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdb002d5d0 con 0x7fcdb01028d0 2024-04-20T11:21:49.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.150+0000 7fcda7400640 1 -- 172.21.15.44:0/2381522377 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7fcda00a0db0 con 0x7fcdb01028d0 2024-04-20T11:21:49.150 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:49.151 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:20:09.840804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:21:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:21:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.152+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcd940a3c30 msgr2=0x7fcd940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.152+0000 7fcdafe00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcd940a3c30 0x7fcd940a60f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcdb01996e0 tx=0x7fcd98005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.152+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 msgr2=0x7fcdb01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.152+0000 7fcdafe00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 0x7fcdb01981d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcda000e9f0 tx=0x7fcda000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 shutdown_connections 2024-04-20T11:21:49.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fcd940a3c30 0x7fcd940a60f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb0107b30 0x7fcdb019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb0105200 0x7fcdb0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 --2- 172.21.15.44:0/2381522377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdb01028d0 0x7fcdb01981d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 >> 172.21.15.44:0/2381522377 conn(0x7fcdb00fc300 msgr2=0x7fcdb01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 shutdown_connections 2024-04-20T11:21:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.153+0000 7fcdafe00640 1 -- 172.21.15.44:0/2381522377 wait complete. 2024-04-20T11:21:49.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:21:49.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.385+0000 7f0967200640 1 -- 172.21.15.44:0/1192356910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968105b90 msgr2=0x7f096810c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 --2- 172.21.15.44:0/1192356910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968105b90 0x7f096810c420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0960009ca0 tx=0x7f096002c100 comp rx=0 tx=0).stop 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 -- 172.21.15.44:0/1192356910 shutdown_connections 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 --2- 172.21.15.44:0/1192356910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968105b90 0x7f096810c420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 --2- 172.21.15.44:0/1192356910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09681051d0 0x7f0968105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 --2- 172.21.15.44:0/1192356910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0968103fe0 0x7f0968104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 -- 172.21.15.44:0/1192356910 >> 172.21.15.44:0/1192356910 conn(0x7f09680ff570 msgr2=0x7f0968101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 -- 172.21.15.44:0/1192356910 shutdown_connections 2024-04-20T11:21:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.386+0000 7f0967200640 1 -- 172.21.15.44:0/1192356910 wait complete. 2024-04-20T11:21:49.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 Processor -- start 2024-04-20T11:21:49.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 -- start start 2024-04-20T11:21:49.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 0x7f096819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09681051d0 0x7f096819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0968105b90 0x7f09681a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f096810fd90 con 0x7f0968103fe0 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f096810fc10 con 0x7f09681051d0 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0967200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f096810ff10 con 0x7f0968105b90 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.387+0000 7f0965e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 0x7f096819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 0x7f096819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54572/0 (socket says 172.21.15.44:54572) 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 -- 172.21.15.44:0/1756522236 learned_addr learned my addr 172.21.15.44:0/1756522236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965400640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09681051d0 0x7f096819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0966800640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0968105b90 0x7f09681a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 -- 172.21.15.44:0/1756522236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0968105b90 msgr2=0x7f09681a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0968105b90 0x7f09681a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 -- 172.21.15.44:0/1756522236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09681051d0 msgr2=0x7f096819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09681051d0 0x7f096819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 -- 172.21.15.44:0/1756522236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681a1f40 con 0x7f0968103fe0 2024-04-20T11:21:49.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.388+0000 7f0965e00640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 0x7f096819a320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f095800e9c0 tx=0x7f095800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.389+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800cd80 con 0x7f0968103fe0 2024-04-20T11:21:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.389+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09580035f0 con 0x7f0968103fe0 2024-04-20T11:21:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.389+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958010690 con 0x7f0968103fe0 2024-04-20T11:21:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.389+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681a2230 con 0x7f0968103fe0 2024-04-20T11:21:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.389+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681a27d0 con 0x7f0968103fe0 2024-04-20T11:21:49.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.390+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096810b300 con 0x7f0968103fe0 2024-04-20T11:21:49.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.392+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f09580037a0 con 0x7f0968103fe0 2024-04-20T11:21:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.394+0000 7f095ea00640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f094c0a3c30 0x7f094c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.395+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f09580d0790 con 0x7f0968103fe0 2024-04-20T11:21:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.395+0000 7f0965400640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f094c0a3c30 0x7f094c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.396+0000 7f0965400640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f094c0a3c30 0x7f094c0a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f096819b830 tx=0x7f0950005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.400+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f095809d170 con 0x7f0968103fe0 2024-04-20T11:21:49.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:49 smithi044 ceph-mon[16855]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:49.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:49 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3251370436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:21:49.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:49 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2381522377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:49.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.606+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0968062b60 con 0x7f0968103fe0 2024-04-20T11:21:49.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.609+0000 7f095ea00640 1 -- 172.21.15.44:0/1756522236 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22511 (secure 0 0 0) 0x7f095809ca70 con 0x7f0968103fe0 2024-04-20T11:21:49.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:49.611 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:51.201931+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:21:49.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f094c0a3c30 msgr2=0x7f094c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f094c0a3c30 0x7f094c0a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f096819b830 tx=0x7f0950005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:49.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 msgr2=0x7f096819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 0x7f096819a320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f095800e9c0 tx=0x7f095800ee80 comp rx=0 tx=0).stop 2024-04-20T11:21:49.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 shutdown_connections 2024-04-20T11:21:49.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f094c0a3c30 0x7f094c0a60f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0968105b90 0x7f09681a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09681051d0 0x7f096819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 --2- 172.21.15.44:0/1756522236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0968103fe0 0x7f096819a320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 >> 172.21.15.44:0/1756522236 conn(0x7f09680ff570 msgr2=0x7f09681084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 shutdown_connections 2024-04-20T11:21:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.612+0000 7f0967200640 1 -- 172.21.15.44:0/1756522236 wait complete. 2024-04-20T11:21:49.634 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:21:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:49 smithi078 ceph-mon[21464]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:49.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:49 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3251370436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:21:49.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:49 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2381522377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:49 smithi165 ceph-mon[21975]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:49 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3251370436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:21:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:49 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2381522377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:49.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 -- 172.21.15.44:0/3128096576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 msgr2=0x7f4dd0104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/3128096576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd0104400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002bc30 comp rx=0 tx=0).stop 2024-04-20T11:21:49.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 -- 172.21.15.44:0/3128096576 shutdown_connections 2024-04-20T11:21:49.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/3128096576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd0105b90 0x7f4dd010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/3128096576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd01051d0 0x7f4dd0105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/3128096576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd0104400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 -- 172.21.15.44:0/3128096576 >> 172.21.15.44:0/3128096576 conn(0x7f4dd00ff570 msgr2=0x7f4dd0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 -- 172.21.15.44:0/3128096576 shutdown_connections 2024-04-20T11:21:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.836+0000 7f4dd4a00640 1 -- 172.21.15.44:0/3128096576 wait complete. 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.837+0000 7f4dd4a00640 1 Processor -- start 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.837+0000 7f4dd4a00640 1 -- start start 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dd4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dd4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd01051d0 0x7f4dd019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dd4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd0105b90 0x7f4dd01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dd4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd010fd40 con 0x7f4dd0103fe0 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dd4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd010fbc0 con 0x7f4dd0105b90 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dd4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd010fec0 con 0x7f4dd01051d0 2024-04-20T11:21:49.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54586/0 (socket says 172.21.15.44:54586) 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 -- 172.21.15.44:0/1785805343 learned_addr learned my addr 172.21.15.44:0/1785805343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dc6a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd01051d0 0x7f4dd019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcfe00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd0105b90 0x7f4dd01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 -- 172.21.15.44:0/1785805343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd01051d0 msgr2=0x7f4dd019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd01051d0 0x7f4dd019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 -- 172.21.15.44:0/1785805343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd0105b90 msgr2=0x7f4dd01a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd0105b90 0x7f4dd01a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcf400640 1 -- 172.21.15.44:0/1785805343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd01a1fe0 con 0x7f4dd0103fe0 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.838+0000 7f4dcfe00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd0105b90 0x7f4dd01a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.839+0000 7f4dcf400640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd019a3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0000c00 tx=0x7f4dc0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.839+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0043020 con 0x7f4dd0103fe0 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.839+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4dc0004250 con 0x7f4dd0103fe0 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.839+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0003c00 con 0x7f4dd0103fe0 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.839+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd01a2270 con 0x7f4dd0103fe0 2024-04-20T11:21:49.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.839+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a2770 con 0x7f4dd0103fe0 2024-04-20T11:21:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.840+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd010b300 con 0x7f4dd0103fe0 2024-04-20T11:21:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.844+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4dc0046020 con 0x7f4dd0103fe0 2024-04-20T11:21:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.845+0000 7f4dccc00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4db40a3c30 0x7f4db40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.845+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f4dc00f2780 con 0x7f4dd0103fe0 2024-04-20T11:21:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.845+0000 7f4dc6a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4db40a3c30 0x7f4db40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.845+0000 7f4dc6a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4db40a3c30 0x7f4db40a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4db80095f0 tx=0x7f4db8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:49.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:49.846+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4dc004d020 con 0x7f4dd0103fe0 2024-04-20T11:21:50.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.092+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dd002d860 con 0x7f4dd0103fe0 2024-04-20T11:21:50.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.094+0000 7f4dccc00640 1 -- 172.21.15.44:0/1785805343 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f4dc00bf200 con 0x7f4dd0103fe0 2024-04-20T11:21:50.095 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:21:50.095 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:20:09.840804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:21:50.095 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4db40a3c30 msgr2=0x7f4db40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4db40a3c30 0x7f4db40a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4db80095f0 tx=0x7f4db8005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 msgr2=0x7f4dd019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd019a3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0000c00 tx=0x7f4dc0004a30 comp rx=0 tx=0).stop 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 shutdown_connections 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f4db40a3c30 0x7f4db40a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd0105b90 0x7f4dd01a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd01051d0 0x7f4dd019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 --2- 172.21.15.44:0/1785805343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0103fe0 0x7f4dd019a3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.096+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 >> 172.21.15.44:0/1785805343 conn(0x7f4dd00ff570 msgr2=0x7f4dd01084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.097+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 shutdown_connections 2024-04-20T11:21:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.097+0000 7f4dd4a00640 1 -- 172.21.15.44:0/1785805343 wait complete. 2024-04-20T11:21:50.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T11:20:02.149413+0000', 'modified': '2024-04-20T11:20:09.840804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24596, 'mds_1': 24562, 'mds_2': 24550, 'mds_3': 24506, 'mds_4': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6833/715928335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 715928335}, {'type': 'v1', 'addr': '172.21.15.78:6833', 'nonce': 715928335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/858649103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 858649103}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 858649103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'd', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.44:6837/2623237731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 2623237731}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 2623237731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.78:6835/3770087097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 3770087097}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 3770087097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/1721181859', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 1721181859}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 1721181859}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2414137592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2414137592}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2414137592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/2547716992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2547716992}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2547716992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/4179547084', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 4179547084}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 4179547084}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'g', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.44:6839/2240418551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2240418551}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2240418551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6839/315239720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 315239720}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 315239720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24550, 24494, 24596, 24506, 24562]} 2024-04-20T11:21:50.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-20T11:21:50.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 -- 172.21.15.44:0/2481978516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 msgr2=0x7fe99c109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2481978516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 0x7fe99c109b10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe988009900 tx=0x7fe98802bc30 comp rx=0 tx=0).stop 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 -- 172.21.15.44:0/2481978516 shutdown_connections 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2481978516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c10a050 0x7fe99c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2481978516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 0x7fe99c109b10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2481978516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe99c1084a0 0x7fe99c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 -- 172.21.15.44:0/2481978516 >> 172.21.15.44:0/2481978516 conn(0x7fe99c077430 msgr2=0x7fe99c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 -- 172.21.15.44:0/2481978516 shutdown_connections 2024-04-20T11:21:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.328+0000 7fe9a3000640 1 -- 172.21.15.44:0/2481978516 wait complete. 2024-04-20T11:21:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 Processor -- start 2024-04-20T11:21:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 -- start start 2024-04-20T11:21:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 0x7fe99c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 0x7fe99c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe99c10a050 0x7fe99c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c114200 con 0x7fe99c109690 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.329+0000 7fe9a3000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c114080 con 0x7fe99c1084a0 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c114380 con 0x7fe99c10a050 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 0x7fe99c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 0x7fe99c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53874/0 (socket says 172.21.15.44:53874) 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2139998532 learned_addr learned my addr 172.21.15.44:0/2139998532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1200640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 0x7fe99c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a2600640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe99c10a050 0x7fe99c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2139998532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe99c10a050 msgr2=0x7fe99c1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe99c10a050 0x7fe99c1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 msgr2=0x7fe99c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 0x7fe99c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.330+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2139998532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c1a6440 con 0x7fe99c1084a0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.331+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 0x7fe99c19e820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe99000e9c0 tx=0x7fe99000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.331+0000 7fe99a000640 1 -- 172.21.15.44:0/2139998532 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99000cd80 con 0x7fe99c1084a0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.331+0000 7fe99a000640 1 -- 172.21.15.44:0/2139998532 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe9900035f0 con 0x7fe99c1084a0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.331+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c1a6730 con 0x7fe99c1084a0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.331+0000 7fe99a000640 1 -- 172.21.15.44:0/2139998532 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990010690 con 0x7fe99c1084a0 2024-04-20T11:21:50.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.331+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1a6c50 con 0x7fe99c1084a0 2024-04-20T11:21:50.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.334+0000 7fe99a000640 1 -- 172.21.15.44:0/2139998532 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe9900037a0 con 0x7fe99c1084a0 2024-04-20T11:21:50.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.334+0000 7fe9a71f8740 1 Processor -- start 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.334+0000 7fe9a71f8740 1 -- start start 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04afb40 0x55f2c04aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b0530 0x55f2c04b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04169d0 con 0x55f2c04b56a0 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c03bd270 con 0x55f2c04afb40 2024-04-20T11:21:50.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c0412990 con 0x55f2c04b0530 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04b7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54612/0 (socket says 172.21.15.44:54612) 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 -- 172.21.15.44:0/3675815909 learned_addr learned my addr 172.21.15.44:0/3675815909 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a2600640 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04afb40 0x55f2c04aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b0530 0x55f2c04b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 -- 172.21.15.44:0/3675815909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b0530 msgr2=0x55f2c04b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b0530 0x55f2c04b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 -- 172.21.15.44:0/3675815909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04afb40 msgr2=0x55f2c04aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04afb40 0x55f2c04aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1200640 1 -- 172.21.15.44:0/3675815909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c01292d0 con 0x55f2c04b56a0 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.335+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b0530 0x55f2c04b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe99a000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe9840a3bb0 0x7fe9840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe99a000640 1 -- 172.21.15.44:0/2139998532 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fe9900d0810 con 0x7fe99c1084a0 2024-04-20T11:21:50.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe9a1200640 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04b7b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe988005ab0 tx=0x7fe988005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141e389747de542 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe9a2600640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe9840a3bb0 0x7fe9840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe98ea00640 1 -- 172.21.15.44:0/3675815909 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98802c650 con 0x55f2c04b56a0 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe98ea00640 1 -- 172.21.15.44:0/3675815909 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe988004040 con 0x55f2c04b56a0 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.336+0000 7fe98ea00640 1 -- 172.21.15.44:0/3675815909 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9880046c0 con 0x55f2c04b56a0 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 -- 172.21.15.44:0/3675815909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 msgr2=0x55f2c04b7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a2600640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe9840a3bb0 0x7fe9840a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55f2c04b0d90 tx=0x7fe994009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04b7b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe988005ab0 tx=0x7fe988005be0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 -- 172.21.15.44:0/3675815909 shutdown_connections 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04b7b30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b0530 0x55f2c04b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/3675815909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04afb40 0x55f2c04aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 -- 172.21.15.44:0/3675815909 >> 172.21.15.44:0/3675815909 conn(0x55f2c04ab0b0 msgr2=0x55f2c04ab4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 -- 172.21.15.44:0/3675815909 shutdown_connections 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.337+0000 7fe9a71f8740 1 -- 172.21.15.44:0/3675815909 wait complete. 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 -- start start 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 Processor -- start 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04ba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04b85f0 con 0x55f2c04b56a0 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c03d05e0 con 0x55f2c04b0530 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04b8770 con 0x55f2c04afb40 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.338+0000 7fe9a71f8740 10 client.0 _ll_register_callbacks cb 0x55f2c0128ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54624/0 (socket says 172.21.15.44:54624) 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a2600640 1 -- 172.21.15.44:0/439657256 learned_addr learned my addr 172.21.15.44:0/439657256 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1200640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 -- 172.21.15.44:0/439657256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 msgr2=0x55f2c04ba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04ba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 -- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 msgr2=0x55f2c04c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 -- 172.21.15.44:0/439657256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c01f36b0 con 0x55f2c04afb40 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a2600640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04ba020 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55f2c04b0fa0 tx=0x7fe990098e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29e02505cde4f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.339+0000 7fe983400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe983400640 1 -- 172.21.15.44:0/439657256 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990003bc0 con 0x55f2c04afb40 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe983400640 1 -- 172.21.15.44:0/439657256 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fe99009bf40 con 0x55f2c04afb40 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988004040 con 0x55f2c04afb40 2024-04-20T11:21:50.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe983400640 1 -- 172.21.15.44:0/439657256 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe99009d1f0 con 0x55f2c04afb40 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe9a71f8740 10 client.24991 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2c04c4a30 con 0x55f2c04afb40 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe983400640 1 -- 172.21.15.44:0/439657256 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe988004040 con 0x55f2c04afb40 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe983400640 1 -- 172.21.15.44:0/439657256 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fe9900b8020 con 0x55f2c04afb40 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.340+0000 7fe9a71f8740 10 client.24991 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a71f8740 10 client.24991 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a71f8740 20 client.24991 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c04c6ff0 0x55f2c04c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a71f8740 4 client.24991 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2c04243f0 con 0x55f2c04c6ff0 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a1200640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c04c6ff0 0x55f2c04c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe9a1200640 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c04c6ff0 0x55f2c04c8460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff48b278fe5b59f8 server_cookie=2ff430d6519495fe in_seq=0 out_seq=0 2024-04-20T11:21:50.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.341+0000 7fe983400640 10 client.24991 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:21:50.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.342+0000 7fe983400640 1 -- 172.21.15.44:0/439657256 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2c04243f0 con 0x55f2c04c6ff0 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.342+0000 7fe983400640 10 client.24991 handle_command_reply: tid=0 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.342+0000 7fe9a71f8740 1 client.24991 shutdown 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 msgr2=0x55f2c04ba020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04ba020 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55f2c04b0fa0 tx=0x7fe990098e50 comp rx=0 tx=0).stop 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 shutdown_connections 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c04c6ff0 0x55f2c04c8460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04ba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/439657256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04ba020 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 >> 172.21.15.44:0/439657256 conn(0x55f2c04ab0b0 msgr2=0x55f2c04aa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe983400640 0 client.24991 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 shutdown_connections 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 1 -- 172.21.15.44:0/439657256 wait complete. 2024-04-20T11:21:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.343+0000 7fe9a71f8740 20 client.24991 trim_cache size 0 max 16384 2024-04-20T11:21:50.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 Processor -- start 2024-04-20T11:21:50.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 -- start start 2024-04-20T11:21:50.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04c1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04ba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04c67a0 con 0x55f2c04b56a0 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04c6e70 con 0x55f2c04b0530 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.363+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04c7540 con 0x55f2c04afb40 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53908/0 (socket says 172.21.15.44:53908) 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 -- 172.21.15.44:0/1475578197 learned_addr learned my addr 172.21.15.44:0/1475578197 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 -- 172.21.15.44:0/1475578197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 msgr2=0x55f2c04b9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04b9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 -- 172.21.15.44:0/1475578197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 msgr2=0x55f2c04ba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04ba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 -- 172.21.15.44:0/1475578197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c04b4d20 con 0x55f2c04b0530 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe9a1200640 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04c1530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe988009900 tx=0x7fe98804a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a7be9899088d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.364+0000 7fe98e000640 1 -- 172.21.15.44:0/1475578197 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98802ccd0 con 0x55f2c04b0530 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe98e000640 1 -- 172.21.15.44:0/1475578197 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe9880349e0 con 0x55f2c04b0530 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe98e000640 1 -- 172.21.15.44:0/1475578197 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98803f670 con 0x55f2c04b0530 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 -- 172.21.15.44:0/1475578197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 msgr2=0x55f2c04c1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04c1530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe988009900 tx=0x7fe98804a040 comp rx=0 tx=0).stop 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 -- 172.21.15.44:0/1475578197 shutdown_connections 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04b56a0 0x55f2c04ba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04c1530 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/1475578197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04afb40 0x55f2c04b9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 -- 172.21.15.44:0/1475578197 >> 172.21.15.44:0/1475578197 conn(0x55f2c04ab0b0 msgr2=0x55f2c04b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 -- 172.21.15.44:0/1475578197 shutdown_connections 2024-04-20T11:21:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.365+0000 7fe9a71f8740 1 -- 172.21.15.44:0/1475578197 wait complete. 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 -- start start 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 Processor -- start 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 0x55f2c04bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b56a0 0x55f2c04c0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04c82e0 con 0x55f2c04afb40 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04c7c10 con 0x55f2c04b0530 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c04c89b0 con 0x55f2c04b56a0 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.366+0000 7fe9a71f8740 10 client.0 _ll_register_callbacks cb 0x7fe990098670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 0x55f2c04bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 0x55f2c04bd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54642/0 (socket says 172.21.15.44:54642) 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2555773124 learned_addr learned my addr 172.21.15.44:0/2555773124 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a2600640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b56a0 0x55f2c04c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1200640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2555773124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b56a0 msgr2=0x55f2c04c0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b56a0 0x55f2c04c0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2555773124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 msgr2=0x55f2c04bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04bd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 -- 172.21.15.44:0/2555773124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c04c0c80 con 0x55f2c04afb40 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1200640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe9a1c00640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 0x55f2c04bd130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe990098f50 tx=0x7fe990004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb258b4ed1fc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.367+0000 7fe983400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe983400640 1 -- 172.21.15.44:0/2555773124 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99009c630 con 0x55f2c04afb40 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe983400640 1 -- 172.21.15.44:0/2555773124 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fe99009b9b0 con 0x55f2c04afb40 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9880349e0 con 0x55f2c04afb40 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe983400640 1 -- 172.21.15.44:0/2555773124 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe99007d080 con 0x55f2c04afb40 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 10 client.14889 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:50.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2c052d520 con 0x55f2c04afb40 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe983400640 1 -- 172.21.15.44:0/2555773124 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe9880349e0 con 0x55f2c04afb40 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe983400640 1 -- 172.21.15.44:0/2555773124 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fe9900b8070 con 0x55f2c04afb40 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 10 client.14889 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 10 client.14889 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 20 client.14889 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c05321a0 0x55f2c0530eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 4 client.14889 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:21:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.368+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "status"}) -- 0x55f2c041e260 con 0x55f2c05321a0 2024-04-20T11:21:50.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.369+0000 7fe9a1200640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c05321a0 0x55f2c0530eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.369+0000 7fe9a1200640 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c05321a0 0x55f2c0530eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3d82cc3fb68f881 server_cookie=1516a52baf119a01 in_seq=0 out_seq=0 2024-04-20T11:21:50.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.369+0000 7fe983400640 10 client.14889 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe983400640 1 -- 172.21.15.44:0/2555773124 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f2c041e260 con 0x55f2c05321a0 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe983400640 10 client.14889 handle_command_reply: tid=0 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 client.14889 shutdown 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 msgr2=0x55f2c04bd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 0x55f2c04bd130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe990098f50 tx=0x7fe990004000 comp rx=0 tx=0).stop 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 shutdown_connections 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c04b56a0 0x55f2c04c0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55f2c05321a0 0x55f2c0530eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c04b0530 0x55f2c04bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 --2- 172.21.15.44:0/2555773124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c04afb40 0x55f2c04bd130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 >> 172.21.15.44:0/2555773124 conn(0x55f2c04ab0b0 msgr2=0x55f2c04b9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe983400640 0 client.14889 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 shutdown_connections 2024-04-20T11:21:50.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 1 -- 172.21.15.44:0/2555773124 wait complete. 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.370+0000 7fe9a71f8740 20 client.14889 trim_cache size 0 max 16384 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24494, 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 101.53717154100001, 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:21:50.374 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:21:50.375 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:21:50.375 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 139.60677147800001 2024-04-20T11:21:50.375 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:21:50.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe9840a3bb0 msgr2=0x7fe9840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe9840a3bb0 0x7fe9840a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55f2c04b0d90 tx=0x7fe994009500 comp rx=0 tx=0).stop 2024-04-20T11:21:50.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 msgr2=0x7fe99c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 0x7fe99c19e820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe99000e9c0 tx=0x7fe99000ee80 comp rx=0 tx=0).stop 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 shutdown_connections 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fe9840a3bb0 0x7fe9840a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe99c10a050 0x7fe99c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe99c109690 0x7fe99c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 --2- 172.21.15.44:0/2139998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe99c1084a0 0x7fe99c19e820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 >> 172.21.15.44:0/2139998532 conn(0x7fe99c077430 msgr2=0x7fe99c10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 shutdown_connections 2024-04-20T11:21:50.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.371+0000 7fe9a3000640 1 -- 172.21.15.44:0/2139998532 wait complete. 2024-04-20T11:21:50.386 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:50 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1756522236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:50.386 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:50 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1785805343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:50.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 -- 172.21.15.44:0/2093011356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 msgr2=0x7f00e0078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 --2- 172.21.15.44:0/2093011356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e0078da0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009ca0 tx=0x7f00d802c100 comp rx=0 tx=0).stop 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 -- 172.21.15.44:0/2093011356 shutdown_connections 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 --2- 172.21.15.44:0/2093011356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e0078da0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 --2- 172.21.15.44:0/2093011356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0077b60 0x7f00e0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 --2- 172.21.15.44:0/2093011356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0076920 0x7f00e0076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 -- 172.21.15.44:0/2093011356 >> 172.21.15.44:0/2093011356 conn(0x7f00e00ff610 msgr2=0x7f00e0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 -- 172.21.15.44:0/2093011356 shutdown_connections 2024-04-20T11:21:50.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.604+0000 7f00e5e00640 1 -- 172.21.15.44:0/2093011356 wait complete. 2024-04-20T11:21:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 Processor -- start 2024-04-20T11:21:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 -- start start 2024-04-20T11:21:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 0x7f00e019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0077b60 0x7f00e019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e01a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0114250 con 0x7f00e0078500 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01140d0 con 0x7f00e0077b60 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.605+0000 7f00e5e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01143d0 con 0x7f00e0076920 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 0x7f00e019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0077b60 0x7f00e019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 0x7f00e019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42896/0 (socket says 172.21.15.44:42896) 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 -- 172.21.15.44:0/1019220355 learned_addr learned my addr 172.21.15.44:0/1019220355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e5400640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e01a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 -- 172.21.15.44:0/1019220355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0077b60 msgr2=0x7f00e019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0077b60 0x7f00e019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 -- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 msgr2=0x7f00e01a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e01a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 -- 172.21.15.44:0/1019220355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e01a63f0 con 0x7f00e0076920 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00dfe00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0077b60 0x7f00e019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e5400640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e01a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.606+0000 7f00e4a00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 0x7f00e019e7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e9f0 tx=0x7f00d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.607+0000 7f00dd600640 1 -- 172.21.15.44:0/1019220355 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400cd80 con 0x7f00e0076920 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.607+0000 7f00dd600640 1 -- 172.21.15.44:0/1019220355 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f00d4010930 con 0x7f00e0076920 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.607+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01a66e0 con 0x7f00e0076920 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.607+0000 7f00dd600640 1 -- 172.21.15.44:0/1019220355 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4018b00 con 0x7f00e0076920 2024-04-20T11:21:50.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.607+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01a6c30 con 0x7f00e0076920 2024-04-20T11:21:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.610+0000 7f00dd600640 1 -- 172.21.15.44:0/1019220355 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f00d4018ca0 con 0x7f00e0076920 2024-04-20T11:21:50.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.610+0000 7f00ea17e740 1 Processor -- start 2024-04-20T11:21:50.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.610+0000 7f00ea17e740 1 -- start start 2024-04-20T11:21:50.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268370cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370d530 0x562683712030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 0x562683714b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00ea17e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626836739d0 con 0x5626837126a0 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00ea17e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56268361a270 con 0x56268370d530 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00ea17e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56268366f990 con 0x56268370cb40 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 0x562683714b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268370cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268370cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42904/0 (socket says 172.21.15.44:42904) 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 0x562683714b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54664/0 (socket says 172.21.15.44:54664) 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 -- 172.21.15.44:0/1396145649 learned_addr learned my addr 172.21.15.44:0/1396145649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e4a00640 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370d530 0x562683712030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 -- 172.21.15.44:0/1396145649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370d530 msgr2=0x562683712030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370d530 0x562683712030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 -- 172.21.15.44:0/1396145649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 msgr2=0x562683714b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 0x562683714b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00e5400640 1 -- 172.21.15.44:0/1396145649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626833862d0 con 0x56268370cb40 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.611+0000 7f00dfe00640 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 0x562683714b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.612+0000 7f00e5400640 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268370cf60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009ca0 tx=0x7f00d8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b94ba99d47b7eee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.612+0000 7f00d1c00640 1 -- 172.21.15.44:0/1396145649 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8043070 con 0x56268370cb40 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.612+0000 7f00d1c00640 1 -- 172.21.15.44:0/1396145649 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f00d8032d40 con 0x56268370cb40 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.612+0000 7f00d1c00640 1 -- 172.21.15.44:0/1396145649 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8039040 con 0x56268370cb40 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.612+0000 7f00dd600640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f00c80a3bb0 0x7f00c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00e4a00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f00c80a3bb0 0x7f00c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00dd600640 1 -- 172.21.15.44:0/1019220355 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f00d40d0730 con 0x7f00e0076920 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 -- 172.21.15.44:0/1396145649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 msgr2=0x56268370cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268370cf60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009ca0 tx=0x7f00d8005ef0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 -- 172.21.15.44:0/1396145649 shutdown_connections 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626837126a0 0x562683714b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370d530 0x562683712030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1396145649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268370cf60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 -- 172.21.15.44:0/1396145649 >> 172.21.15.44:0/1396145649 conn(0x5626837080b0 msgr2=0x5626837084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 -- 172.21.15.44:0/1396145649 shutdown_connections 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00ea17e740 1 -- 172.21.15.44:0/1396145649 wait complete. 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.613+0000 7f00e4a00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f00c80a3bb0 0x7f00c80a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f00d40ce270 tx=0x7f00d40d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 -- start start 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 Processor -- start 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683717030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x562683717570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x56268371e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626837155f0 con 0x56268370d530 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56268362d5e0 con 0x56268370cb40 2024-04-20T11:21:50.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562683715770 con 0x5626837126a0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.614+0000 7f00ea17e740 10 client.0 _ll_register_callbacks cb 0x562683385ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x562683717570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x56268371e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x56268371e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42912/0 (socket says 172.21.15.44:42912) 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00e5400640 1 -- 172.21.15.44:0/784613777 learned_addr learned my addr 172.21.15.44:0/784613777 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00e5400640 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683717030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 -- 172.21.15.44:0/784613777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 msgr2=0x56268371e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x56268371e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 -- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 msgr2=0x562683717030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683717030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 -- 172.21.15.44:0/784613777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626834506b0 con 0x56268370d530 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00dfe00640 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x562683717570 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00e019fce0 tx=0x7f00cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c372bf7a38e8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.615+0000 7f00c6000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:50.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00c6000640 1 -- 172.21.15.44:0/784613777 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc012110 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00c6000640 1 -- 172.21.15.44:0/784613777 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f00cc004050 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d8032d40 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00c6000640 1 -- 172.21.15.44:0/784613777 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00cc0122b0 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00ea17e740 10 client.14901 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626837219f0 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00c6000640 1 -- 172.21.15.44:0/784613777 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f00d8032d40 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.616+0000 7f00c6000640 1 -- 172.21.15.44:0/784613777 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f00cc0104c0 con 0x56268370d530 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00ea17e740 10 client.14901 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00ea17e740 10 client.14901 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00ea17e740 20 client.14901 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:50.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00ea17e740 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562683724050 0x5626837254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00ea17e740 4 client.14901 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:21:50.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626836813f0 con 0x562683724050 2024-04-20T11:21:50.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.617+0000 7f00e5400640 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562683724050 0x5626837254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.618+0000 7f00e5400640 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562683724050 0x5626837254c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a46480b4f307b7e server_cookie=5f81dc980f6d3626 in_seq=0 out_seq=0 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.618+0000 7f00c6000640 10 client.14901 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.618+0000 7f00c6000640 1 -- 172.21.15.44:0/784613777 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626836813f0 con 0x562683724050 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.618+0000 7f00c6000640 10 client.14901 handle_command_reply: tid=0 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 client.14901 shutdown 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 msgr2=0x562683717570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x562683717570 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00e019fce0 tx=0x7f00cc00f790 comp rx=0 tx=0).stop 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 shutdown_connections 2024-04-20T11:21:50.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x56268371e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562683724050 0x5626837254c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x562683717570 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00c6000640 0 client.14901 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 --2- 172.21.15.44:0/784613777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683717030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 >> 172.21.15.44:0/784613777 conn(0x5626837080b0 msgr2=0x5626837073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 shutdown_connections 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 1 -- 172.21.15.44:0/784613777 wait complete. 2024-04-20T11:21:50.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.619+0000 7f00ea17e740 20 client.14901 trim_cache size 0 max 16384 2024-04-20T11:21:50.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.639+0000 7f00ea17e740 1 Processor -- start 2024-04-20T11:21:50.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.639+0000 7f00ea17e740 1 -- start start 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683703280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x562683708490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00ea17e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562683723800 con 0x56268370d530 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00ea17e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562683723ed0 con 0x56268370cb40 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00ea17e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5626837245a0 con 0x5626837126a0 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00e5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683703280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00e5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683703280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53986/0 (socket says 172.21.15.44:53986) 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54686/0 (socket says 172.21.15.44:54686) 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00e5400640 1 -- 172.21.15.44:0/2652389314 learned_addr learned my addr 172.21.15.44:0/2652389314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00e5400640 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x562683708490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 -- 172.21.15.44:0/2652389314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 msgr2=0x562683708490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x562683708490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 -- 172.21.15.44:0/2652389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 msgr2=0x562683703280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683703280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.640+0000 7f00dfe00640 1 -- 172.21.15.44:0/2652389314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562683711d10 con 0x56268370d530 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.641+0000 7f00dfe00640 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371e570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f00cc010af0 tx=0x7f00cc012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d152446b9af3e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.641+0000 7f00d1200640 1 -- 172.21.15.44:0/2652389314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc02b030 con 0x56268370d530 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.641+0000 7f00d1200640 1 -- 172.21.15.44:0/2652389314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f00cc005610 con 0x56268370d530 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.641+0000 7f00d1200640 1 -- 172.21.15.44:0/2652389314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0036e0 con 0x56268370d530 2024-04-20T11:21:50.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.641+0000 7f00ea17e740 1 -- 172.21.15.44:0/2652389314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 msgr2=0x56268371e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.641+0000 7f00ea17e740 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371e570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f00cc010af0 tx=0x7f00cc012990 comp rx=0 tx=0).stop 2024-04-20T11:21:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 -- 172.21.15.44:0/2652389314 shutdown_connections 2024-04-20T11:21:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5626837126a0 0x562683708490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371e570 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 --2- 172.21.15.44:0/2652389314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56268370cb40 0x562683703280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 -- 172.21.15.44:0/2652389314 >> 172.21.15.44:0/2652389314 conn(0x5626837080b0 msgr2=0x56268370dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 -- 172.21.15.44:0/2652389314 shutdown_connections 2024-04-20T11:21:50.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 -- 172.21.15.44:0/2652389314 wait complete. 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 -- start start 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.642+0000 7f00ea17e740 1 Processor -- start 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268371a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626837126a0 0x56268371dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562683725340 con 0x56268370d530 2024-04-20T11:21:50.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562683724c70 con 0x5626837126a0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562683725a10 con 0x56268370cb40 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00ea17e740 10 client.0 _ll_register_callbacks cb 0x7f00cc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54702/0 (socket says 172.21.15.44:54702) 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 -- 172.21.15.44:0/1889731095 learned_addr learned my addr 172.21.15.44:0/1889731095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00e5400640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626837126a0 0x56268371dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00e5400640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268371a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 -- 172.21.15.44:0/1889731095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 msgr2=0x56268371a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268371a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 -- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626837126a0 msgr2=0x56268371dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626837126a0 0x56268371dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.643+0000 7f00dfe00640 1 -- 172.21.15.44:0/1889731095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268371e260 con 0x56268370d530 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00dfe00640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371ace0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00fd90 tx=0x7f00cc010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fa04803e952405 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00c6000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:50.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00c6000640 1 -- 172.21.15.44:0/1889731095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc02b070 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00c6000640 1 -- 172.21.15.44:0/1889731095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f00cc003660 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00cc005610 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00c6000640 1 -- 172.21.15.44:0/1889731095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00cc005460 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00ea17e740 10 client.14907 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56268378aa40 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.644+0000 7f00c6000640 1 -- 172.21.15.44:0/1889731095 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f00cc005610 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00c6000640 1 -- 172.21.15.44:0/1889731095 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f00cc027aa0 con 0x56268370d530 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00ea17e740 10 client.14907 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00ea17e740 10 client.14907 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00ea17e740 20 client.14907 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x56268378f2f0 0x56268378e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00ea17e740 4 client.14907 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:21:50.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "status"}) -- 0x56268367b260 con 0x56268378f2f0 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.645+0000 7f00e5400640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x56268378f2f0 0x56268378e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00e5400640 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x56268378f2f0 0x56268378e000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98893b27d0f2f992 server_cookie=dfe7bf05cfc0bfe5 in_seq=0 out_seq=0 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00c6000640 10 client.14907 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00c6000640 1 -- 172.21.15.44:0/1889731095 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56268367b260 con 0x56268378f2f0 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00c6000640 10 client.14907 handle_command_reply: tid=0 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 client.14907 shutdown 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 msgr2=0x56268371ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371ace0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00fd90 tx=0x7f00cc010660 comp rx=0 tx=0).stop 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 shutdown_connections 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626837126a0 0x56268371dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x56268378f2f0 0x56268378e000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56268370d530 0x56268371ace0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 --2- 172.21.15.44:0/1889731095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56268370cb40 0x56268371a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00c6000640 0 client.14907 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 >> 172.21.15.44:0/1889731095 conn(0x5626837080b0 msgr2=0x562683703280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 shutdown_connections 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 1 -- 172.21.15.44:0/1889731095 wait complete. 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.646+0000 7f00ea17e740 20 client.14907 trim_cache size 0 max 16384 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24506, 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:21:50.651 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 102.83652118800001, 2024-04-20T11:21:50.652 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:21:50.652 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:21:50.652 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:21:50.652 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 136.934186006 2024-04-20T11:21:50.652 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:21:50.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f00c80a3bb0 msgr2=0x7f00c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f00c80a3bb0 0x7f00c80a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f00d40ce270 tx=0x7f00d40d6530 comp rx=0 tx=0).stop 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 msgr2=0x7f00e019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 0x7f00e019e7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e9f0 tx=0x7f00d400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 shutdown_connections 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f00c80a3bb0 0x7f00c80a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e0078500 0x7f00e01a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e0077b60 0x7f00e019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 --2- 172.21.15.44:0/1019220355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00e0076920 0x7f00e019e7d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 >> 172.21.15.44:0/1019220355 conn(0x7f00e00ff610 msgr2=0x7f00e0101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 shutdown_connections 2024-04-20T11:21:50.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.647+0000 7f00e5e00640 1 -- 172.21.15.44:0/1019220355 wait complete. 2024-04-20T11:21:50.663 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-20T11:21:50.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:50 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1756522236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:50 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1785805343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:50.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:50 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1756522236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:21:50.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:50 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1785805343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:21:50.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.852+0000 7fdf22200640 1 -- 172.21.15.44:0/1229201295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c1084a0 msgr2=0x7fdf1c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.852+0000 7fdf22200640 1 --2- 172.21.15.44:0/1229201295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c1088c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009900 tx=0x7fdf0402bc50 comp rx=0 tx=0).stop 2024-04-20T11:21:50.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.852+0000 7fdf22200640 1 -- 172.21.15.44:0/1229201295 shutdown_connections 2024-04-20T11:21:50.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.852+0000 7fdf22200640 1 --2- 172.21.15.44:0/1229201295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.853+0000 7fdf22200640 1 --2- 172.21.15.44:0/1229201295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c109690 0x7fdf1c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.853+0000 7fdf22200640 1 --2- 172.21.15.44:0/1229201295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c1088c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.853+0000 7fdf22200640 1 -- 172.21.15.44:0/1229201295 >> 172.21.15.44:0/1229201295 conn(0x7fdf1c077430 msgr2=0x7fdf1c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.853+0000 7fdf22200640 1 -- 172.21.15.44:0/1229201295 shutdown_connections 2024-04-20T11:21:50.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.853+0000 7fdf22200640 1 -- 172.21.15.44:0/1229201295 wait complete. 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 Processor -- start 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 -- start start 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c109690 0x7fdf1c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c114240 con 0x7fdf1c10a050 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1140c0 con 0x7fdf1c1084a0 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf22200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1143c0 con 0x7fdf1c109690 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf21800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf20e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf20e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54012/0 (socket says 172.21.15.44:54012) 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf20e00640 1 -- 172.21.15.44:0/2184839455 learned_addr learned my addr 172.21.15.44:0/2184839455 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.854+0000 7fdf21800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54716/0 (socket says 172.21.15.44:54716) 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf1be00640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c109690 0x7fdf1c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf20e00640 1 -- 172.21.15.44:0/2184839455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c109690 msgr2=0x7fdf1c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf20e00640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c109690 0x7fdf1c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf20e00640 1 -- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 msgr2=0x7fdf1c1a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf20e00640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf20e00640 1 -- 172.21.15.44:0/2184839455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1a6480 con 0x7fdf1c1084a0 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf1be00640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c109690 0x7fdf1c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf21800640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf20e00640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c19e860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005ab0 tx=0x7fdf0402c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.855+0000 7fdf19600640 1 -- 172.21.15.44:0/2184839455 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04043020 con 0x7fdf1c1084a0 2024-04-20T11:21:50.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.856+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1a6710 con 0x7fdf1c1084a0 2024-04-20T11:21:50.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.856+0000 7fdf19600640 1 -- 172.21.15.44:0/2184839455 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf040042d0 con 0x7fdf1c1084a0 2024-04-20T11:21:50.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.856+0000 7fdf19600640 1 -- 172.21.15.44:0/2184839455 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04003cc0 con 0x7fdf1c1084a0 2024-04-20T11:21:50.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.856+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a6cb0 con 0x7fdf1c1084a0 2024-04-20T11:21:50.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 Processor -- start 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 -- start start 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf19600640 1 -- 172.21.15.44:0/2184839455 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf04046020 con 0x7fdf1c1084a0 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6149d0 con 0x55dfdc6b36a0 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc5bb270 con 0x55dfdc6ae530 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.859+0000 7fdf265da740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc610990 con 0x55dfdc6adb40 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54722/0 (socket says 172.21.15.44:54722) 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 -- 172.21.15.44:0/1493103817 learned_addr learned my addr 172.21.15.44:0/1493103817 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf21800640 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf20e00640 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 -- 172.21.15.44:0/1493103817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 msgr2=0x55dfdc6adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 -- 172.21.15.44:0/1493103817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 msgr2=0x55dfdc6b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 -- 172.21.15.44:0/1493103817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdc3272d0 con 0x55dfdc6b36a0 2024-04-20T11:21:50.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.860+0000 7fdf1be00640 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6b5b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009cf0 tx=0x7fdf0c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25a72e576f3feb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf12000640 1 -- 172.21.15.44:0/1493103817 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c012450 con 0x55dfdc6b36a0 2024-04-20T11:21:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf12000640 1 -- 172.21.15.44:0/1493103817 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf0c012af0 con 0x55dfdc6b36a0 2024-04-20T11:21:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf12000640 1 -- 172.21.15.44:0/1493103817 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00fd00 con 0x55dfdc6b36a0 2024-04-20T11:21:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf19600640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fdf000a3bb0 0x7fdf000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf19600640 1 -- 172.21.15.44:0/2184839455 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fdf040f2550 con 0x7fdf1c1084a0 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf21800640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fdf000a3bb0 0x7fdf000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.861+0000 7fdf265da740 1 -- 172.21.15.44:0/1493103817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 msgr2=0x55dfdc6b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6b5b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009cf0 tx=0x7fdf0c0117b0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 -- 172.21.15.44:0/1493103817 shutdown_connections 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6b5b30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 --2- 172.21.15.44:0/1493103817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 -- 172.21.15.44:0/1493103817 >> 172.21.15.44:0/1493103817 conn(0x55dfdc6a90b0 msgr2=0x55dfdc6a94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 -- 172.21.15.44:0/1493103817 shutdown_connections 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf21800640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fdf000a3bb0 0x7fdf000a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dfdc6aed90 tx=0x7fdf140096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 -- 172.21.15.44:0/1493103817 wait complete. 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.862+0000 7fdf265da740 1 -- start start 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 Processor -- start 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6b6650 con 0x55dfdc6b36a0 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc5ce5e0 con 0x55dfdc6ae530 2024-04-20T11:21:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6b67d0 con 0x55dfdc6adb40 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf265da740 10 client.0 _ll_register_callbacks cb 0x55dfdc326ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf21800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf21800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6bf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54736/0 (socket says 172.21.15.44:54736) 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf21800640 1 -- 172.21.15.44:0/4159444585 learned_addr learned my addr 172.21.15.44:0/4159444585 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.863+0000 7fdf20e00640 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf1be00640 1 -- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 msgr2=0x55dfdc6b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf1be00640 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf1be00640 1 -- 172.21.15.44:0/4159444585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 msgr2=0x55dfdc6bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf1be00640 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf1be00640 1 -- 172.21.15.44:0/4159444585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdc3f16b0 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf1be00640 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b8550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c01c040 tx=0x7fdf0c00f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e92a9c5e345c63f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf0aa00640 1 -- 172.21.15.44:0/4159444585 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c029020 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf0aa00640 1 -- 172.21.15.44:0/4159444585 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fdf0c00fa40 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.864+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c012af0 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf0aa00640 1 -- 172.21.15.44:0/4159444585 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c00fc40 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf265da740 10 client.25040 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dfdc6c2900 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf0aa00640 1 -- 172.21.15.44:0/4159444585 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf0c012af0 con 0x55dfdc6ae530 2024-04-20T11:21:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf0aa00640 1 -- 172.21.15.44:0/4159444585 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fdf0c0035f0 con 0x55dfdc6ae530 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf265da740 10 client.25040 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf265da740 10 client.25040 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.865+0000 7fdf265da740 20 client.25040 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.866+0000 7fdf265da740 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc6c4c30 0x55dfdc6c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.866+0000 7fdf265da740 4 client.25040 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.866+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfdc6223f0 con 0x55dfdc6c4c30 2024-04-20T11:21:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.866+0000 7fdf20e00640 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc6c4c30 0x55dfdc6c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.866+0000 7fdf20e00640 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc6c4c30 0x55dfdc6c60a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35e7da0bb999e610 server_cookie=6114f8f0cb74b2d8 in_seq=0 out_seq=0 2024-04-20T11:21:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.866+0000 7fdf0aa00640 10 client.25040 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:21:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf0aa00640 1 -- 172.21.15.44:0/4159444585 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfdc6223f0 con 0x55dfdc6c4c30 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf0aa00640 10 client.25040 handle_command_reply: tid=0 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 client.25040 shutdown 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 msgr2=0x55dfdc6b8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b8550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c01c040 tx=0x7fdf0c00f660 comp rx=0 tx=0).stop 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 shutdown_connections 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc6c4c30 0x55dfdc6c60a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6ae530 0x55dfdc6b8550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 --2- 172.21.15.44:0/4159444585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf0aa00640 0 client.25040 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 >> 172.21.15.44:0/4159444585 conn(0x55dfdc6a90b0 msgr2=0x55dfdc6a83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 shutdown_connections 2024-04-20T11:21:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 1 -- 172.21.15.44:0/4159444585 wait complete. 2024-04-20T11:21:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.868+0000 7fdf265da740 20 client.25040 trim_cache size 0 max 16384 2024-04-20T11:21:50.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.888+0000 7fdf265da740 1 Processor -- start 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.888+0000 7fdf265da740 1 -- start start 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf265da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf265da740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6ae530 0x55dfdc6bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf265da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf265da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6c43e0 con 0x55dfdc6b36a0 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf265da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6c4ab0 con 0x55dfdc6adb40 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf265da740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6c5180 con 0x55dfdc6ae530 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf21800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf20e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf20e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54040/0 (socket says 172.21.15.44:54040) 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf20e00640 1 -- 172.21.15.44:0/3820137149 learned_addr learned my addr 172.21.15.44:0/3820137149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.889+0000 7fdf1be00640 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6ae530 0x55dfdc6bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf20e00640 1 -- 172.21.15.44:0/3820137149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6ae530 msgr2=0x55dfdc6bf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf20e00640 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6ae530 0x55dfdc6bf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf20e00640 1 -- 172.21.15.44:0/3820137149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 msgr2=0x55dfdc6acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf20e00640 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf20e00640 1 -- 172.21.15.44:0/3820137149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdc6b9190 con 0x55dfdc6adb40 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf1be00640 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6ae530 0x55dfdc6bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf21800640 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf20e00640 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b5a90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402c920 tx=0x7fdf040f49a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ed06d54f06b16c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf11600640 1 -- 172.21.15.44:0/3820137149 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf040be690 con 0x55dfdc6adb40 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf11600640 1 -- 172.21.15.44:0/3820137149 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf040a7e50 con 0x55dfdc6adb40 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.890+0000 7fdf11600640 1 -- 172.21.15.44:0/3820137149 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf040be220 con 0x55dfdc6adb40 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 -- 172.21.15.44:0/3820137149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 msgr2=0x55dfdc6b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b5a90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402c920 tx=0x7fdf040f49a0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 -- 172.21.15.44:0/3820137149 shutdown_connections 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6b36a0 0x55dfdc6acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6ae530 0x55dfdc6bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 --2- 172.21.15.44:0/3820137149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6adb40 0x55dfdc6b5a90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 -- 172.21.15.44:0/3820137149 >> 172.21.15.44:0/3820137149 conn(0x55dfdc6a90b0 msgr2=0x55dfdc6aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 -- 172.21.15.44:0/3820137149 shutdown_connections 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 -- 172.21.15.44:0/3820137149 wait complete. 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.891+0000 7fdf265da740 1 -- start start 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 Processor -- start 2024-04-20T11:21:50.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 0x55dfdc726bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6b36a0 0x55dfdc729110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6c5f20 con 0x55dfdc6ae530 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6c5850 con 0x55dfdc6b36a0 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfdc6c65f0 con 0x55dfdc6adb40 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf265da740 10 client.0 _ll_register_callbacks cb 0x7fdf040f5e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 0x55dfdc726bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf21800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6b36a0 0x55dfdc729110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf20e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 0x55dfdc726bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54746/0 (socket says 172.21.15.44:54746) 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.892+0000 7fdf1be00640 1 -- 172.21.15.44:0/69889187 learned_addr learned my addr 172.21.15.44:0/69889187 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf1be00640 1 -- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 msgr2=0x55dfdc6bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf1be00640 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6bd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf1be00640 1 -- 172.21.15.44:0/69889187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6b36a0 msgr2=0x55dfdc729110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf1be00640 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6b36a0 0x55dfdc729110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf1be00640 1 -- 172.21.15.44:0/69889187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdc729920 con 0x55dfdc6ae530 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf1be00640 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 0x55dfdc726bc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c000c00 tx=0x7fdf0c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98be0038610b620d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf0aa00640 1 -- 172.21.15.44:0/69889187 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c03d060 con 0x55dfdc6ae530 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf0aa00640 1 -- 172.21.15.44:0/69889187 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fdf0c0037f0 con 0x55dfdc6ae530 2024-04-20T11:21:50.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.893+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf040a7e50 con 0x55dfdc6ae530 2024-04-20T11:21:50.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf0aa00640 1 -- 172.21.15.44:0/69889187 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c029040 con 0x55dfdc6ae530 2024-04-20T11:21:50.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 10 client.14925 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:50.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dfdc72ac20 con 0x55dfdc6ae530 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf0aa00640 1 -- 172.21.15.44:0/69889187 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf040a7e50 con 0x55dfdc6ae530 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf0aa00640 1 -- 172.21.15.44:0/69889187 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fdf0c042070 con 0x55dfdc6ae530 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 10 client.14925 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 10 client.14925 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 20 client.14925 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc72f790 0x55dfdc72c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 4 client.14925 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:21:50.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.894+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "status"}) -- 0x55dfdc61c260 con 0x55dfdc72f790 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf20e00640 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc72f790 0x55dfdc72c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf20e00640 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc72f790 0x55dfdc72c2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6190b58140b7269c server_cookie=f0d7f9d6234a830a in_seq=0 out_seq=0 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf0aa00640 10 client.14925 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf0aa00640 1 -- 172.21.15.44:0/69889187 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dfdc61c260 con 0x55dfdc72f790 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf0aa00640 10 client.14925 handle_command_reply: tid=0 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf265da740 1 client.14925 shutdown 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 msgr2=0x55dfdc726bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.895+0000 7fdf265da740 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 0x55dfdc726bc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c000c00 tx=0x7fdf0c005fd0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 shutdown_connections 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55dfdc72f790 0x55dfdc72c2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfdc6b36a0 0x55dfdc729110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfdc6ae530 0x55dfdc726bc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 --2- 172.21.15.44:0/69889187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfdc6adb40 0x55dfdc6bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf0aa00640 0 client.14925 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 >> 172.21.15.44:0/69889187 conn(0x55dfdc6a90b0 msgr2=0x55dfdc6acba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 shutdown_connections 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 1 -- 172.21.15.44:0/69889187 wait complete. 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf265da740 20 client.14925 trim_cache size 0 max 16384 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24550, 2024-04-20T11:21:50.901 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:21:50.902 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:21:50.902 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 103.828831404, 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 125.27472457099999 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:21:50.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fdf000a3bb0 msgr2=0x7fdf000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf22200640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fdf000a3bb0 0x7fdf000a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dfdc6aed90 tx=0x7fdf140096a0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 msgr2=0x7fdf1c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.896+0000 7fdf22200640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c19e860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005ab0 tx=0x7fdf0402c730 comp rx=0 tx=0).stop 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 shutdown_connections 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fdf000a3bb0 0x7fdf000a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf1c10a050 0x7fdf1c1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c109690 0x7fdf1c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 --2- 172.21.15.44:0/2184839455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf1c1084a0 0x7fdf1c19e860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 >> 172.21.15.44:0/2184839455 conn(0x7fdf1c077430 msgr2=0x7fdf1c10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 shutdown_connections 2024-04-20T11:21:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:50.897+0000 7fdf22200640 1 -- 172.21.15.44:0/2184839455 wait complete. 2024-04-20T11:21:50.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-20T11:21:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 -- 172.21.15.44:0/4159670635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f335810a050 msgr2=0x7f33581108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 --2- 172.21.15.44:0/4159670635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f335810a050 0x7f33581108e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3350009be0 tx=0x7f335002bef0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 -- 172.21.15.44:0/4159670635 shutdown_connections 2024-04-20T11:21:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 --2- 172.21.15.44:0/4159670635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f335810a050 0x7f33581108e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 --2- 172.21.15.44:0/4159670635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3358109690 0x7f3358109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 --2- 172.21.15.44:0/4159670635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33581084a0 0x7f33581088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.117+0000 7f335ca00640 1 -- 172.21.15.44:0/4159670635 >> 172.21.15.44:0/4159670635 conn(0x7f3358077430 msgr2=0x7f3358079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.118+0000 7f335ca00640 1 -- 172.21.15.44:0/4159670635 shutdown_connections 2024-04-20T11:21:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.118+0000 7f335ca00640 1 -- 172.21.15.44:0/4159670635 wait complete. 2024-04-20T11:21:51.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.118+0000 7f335ca00640 1 Processor -- start 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 -- start start 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33581084a0 0x7f335819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3358109690 0x7f335819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 0x7f33581a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3358114250 con 0x7f335810a050 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33581140d0 con 0x7f3358109690 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f335ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33581143d0 con 0x7f33581084a0 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f3357e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 0x7f33581a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f3357e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 0x7f33581a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54754/0 (socket says 172.21.15.44:54754) 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f334fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3358109690 0x7f335819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f3357e00640 1 -- 172.21.15.44:0/777329324 learned_addr learned my addr 172.21.15.44:0/777329324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.119+0000 7f3357400640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33581084a0 0x7f335819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3357e00640 1 -- 172.21.15.44:0/777329324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33581084a0 msgr2=0x7f335819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3357e00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33581084a0 0x7f335819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3357e00640 1 -- 172.21.15.44:0/777329324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3358109690 msgr2=0x7f335819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3357e00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3358109690 0x7f335819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3357e00640 1 -- 172.21.15.44:0/777329324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33581a63f0 con 0x7f335810a050 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3357e00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 0x7f33581a5d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3350033550 tx=0x7f3350033820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3354c00640 1 -- 172.21.15.44:0/777329324 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350043020 con 0x7f335810a050 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33581a6680 con 0x7f335810a050 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3354c00640 1 -- 172.21.15.44:0/777329324 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3350003b80 con 0x7f335810a050 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.120+0000 7f3354c00640 1 -- 172.21.15.44:0/777329324 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335003e780 con 0x7f335810a050 2024-04-20T11:21:51.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.121+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33581a6bd0 con 0x7f335810a050 2024-04-20T11:21:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.124+0000 7f3360c14740 1 Processor -- start 2024-04-20T11:21:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.124+0000 7f3354c00640 1 -- 172.21.15.44:0/777329324 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3350046020 con 0x7f335810a050 2024-04-20T11:21:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.124+0000 7f3360c14740 1 -- start start 2024-04-20T11:21:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3360c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3360c14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af961f8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3360c14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3360c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55af961599d0 con 0x55af961f2b40 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3360c14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af96100270 con 0x55af961f86a0 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3360c14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af96155990 con 0x55af961f3530 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f334fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f334fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54766/0 (socket says 172.21.15.44:54766) 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f334fe00640 1 -- 172.21.15.44:0/780846965 learned_addr learned my addr 172.21.15.44:0/780846965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3357400640 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af961f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f334fe00640 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3357400640 1 -- 172.21.15.44:0/780846965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 msgr2=0x55af961fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3357400640 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3357400640 1 -- 172.21.15.44:0/780846965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 msgr2=0x55af961f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3357400640 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3354c00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f333c0a3bb0 0x7f333c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.125+0000 7f3357400640 1 -- 172.21.15.44:0/780846965 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af95e6c2d0 con 0x55af961f3530 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f334fe00640 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f3354c00640 1 -- 172.21.15.44:0/777329324 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f33500f2af0 con 0x7f335810a050 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f3357e00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f333c0a3bb0 0x7f333c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f3357400640 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af961f8030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3348005fd0 tx=0x7f3348011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a603d99ff57c7fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f3347e00640 1 -- 172.21.15.44:0/780846965 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334801e070 con 0x55af961f3530 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f3347e00640 1 -- 172.21.15.44:0/780846965 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3348009db0 con 0x55af961f3530 2024-04-20T11:21:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.126+0000 7f3347e00640 1 -- 172.21.15.44:0/780846965 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480196b0 con 0x55af961f3530 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 -- 172.21.15.44:0/780846965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 msgr2=0x55af961f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af961f8030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3348005fd0 tx=0x7f3348011810 comp rx=0 tx=0).stop 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3357e00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f333c0a3bb0 0x7f333c0a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f33500096f0 tx=0x7f3350033a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 -- 172.21.15.44:0/780846965 shutdown_connections 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af961f8030 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 --2- 172.21.15.44:0/780846965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 -- 172.21.15.44:0/780846965 >> 172.21.15.44:0/780846965 conn(0x55af961ee0b0 msgr2=0x55af961ee4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 -- 172.21.15.44:0/780846965 shutdown_connections 2024-04-20T11:21:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.127+0000 7f3360c14740 1 -- 172.21.15.44:0/780846965 wait complete. 2024-04-20T11:21:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 -- start start 2024-04-20T11:21:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 Processor -- start 2024-04-20T11:21:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f2b40 0x55af961fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 0x55af961fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96204600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55af961fb650 con 0x55af961f3530 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af961135e0 con 0x55af961f86a0 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.128+0000 7f3360c14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af961fb7d0 con 0x55af961f2b40 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3360c14740 10 client.0 _ll_register_callbacks cb 0x55af95e6bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 0x55af961fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 0x55af961fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54776/0 (socket says 172.21.15.44:54776) 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 -- 172.21.15.44:0/3856477393 learned_addr learned my addr 172.21.15.44:0/3856477393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f334fe00640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f2b40 0x55af961fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f334fe00640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96204600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 -- 172.21.15.44:0/3856477393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f2b40 msgr2=0x55af961fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f2b40 0x55af961fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 -- 172.21.15.44:0/3856477393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 msgr2=0x55af96204600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96204600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f3357400640 1 -- 172.21.15.44:0/3856477393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af95f366b0 con 0x55af961f3530 2024-04-20T11:21:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.129+0000 7f334fe00640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96204600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3357400640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 0x55af961fd5c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3348000c00 tx=0x7f3348005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8896ba709d24d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3344c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3344c00640 1 -- 172.21.15.44:0/3856477393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348027020 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3344c00640 1 -- 172.21.15.44:0/3856477393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f3348003e00 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3348009db0 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3344c00640 1 -- 172.21.15.44:0/3856477393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f334801e070 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3360c14740 10 client.14937 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af962079c0 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.130+0000 7f3344c00640 1 -- 172.21.15.44:0/3856477393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3348009db0 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3344c00640 1 -- 172.21.15.44:0/3856477393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f3348004bc0 con 0x55af961f3530 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3360c14740 10 client.14937 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3360c14740 10 client.14937 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3360c14740 20 client.14937 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3360c14740 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af96209d30 0x55af9620b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3360c14740 4 client.14937 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:21:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af961673f0 con 0x55af96209d30 2024-04-20T11:21:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.131+0000 7f334fe00640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af96209d30 0x55af9620b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.132+0000 7f334fe00640 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af96209d30 0x55af9620b1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d059847608c533e server_cookie=1286e0d4600a0f95 in_seq=0 out_seq=0 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.132+0000 7f3344c00640 10 client.14937 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3344c00640 1 -- 172.21.15.44:0/3856477393 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af961673f0 con 0x55af96209d30 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3344c00640 10 client.14937 handle_command_reply: tid=0 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 client.14937 shutdown 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 msgr2=0x55af961fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 0x55af961fd5c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3348000c00 tx=0x7f3348005270 comp rx=0 tx=0).stop 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 shutdown_connections 2024-04-20T11:21:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af96209d30 0x55af9620b1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96204600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f3530 0x55af961fd5c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 --2- 172.21.15.44:0/3856477393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f2b40 0x55af961fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3344c00640 0 client.14937 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 >> 172.21.15.44:0/3856477393 conn(0x55af961ee0b0 msgr2=0x55af961ed3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 shutdown_connections 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.133+0000 7f3360c14740 1 -- 172.21.15.44:0/3856477393 wait complete. 2024-04-20T11:21:51.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.134+0000 7f3360c14740 20 client.14937 trim_cache size 0 max 16384 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 Processor -- start 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 -- start start 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af962045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55af962094e0 con 0x55af961f2b40 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af96209bb0 con 0x55af961f86a0 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3360c14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af9620a280 con 0x55af961f3530 2024-04-20T11:21:51.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f334fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f334fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961fb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54782/0 (socket says 172.21.15.44:54782) 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f334fe00640 1 -- 172.21.15.44:0/17959228 learned_addr learned my addr 172.21.15.44:0/17959228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.154+0000 7f3357400640 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af962045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 -- 172.21.15.44:0/17959228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 msgr2=0x55af962045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af962045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 -- 172.21.15.44:0/17959228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 msgr2=0x55af961f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 -- 172.21.15.44:0/17959228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af961df950 con 0x55af961f2b40 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f3357400640 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af962045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f334fe00640 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961fb530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55af9620b8e0 tx=0x7f3340009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a90db5dce4dc09 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f3347400640 1 -- 172.21.15.44:0/17959228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334002c070 con 0x55af961f2b40 2024-04-20T11:21:51.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f3347400640 1 -- 172.21.15.44:0/17959228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3340004230 con 0x55af961f2b40 2024-04-20T11:21:51.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.155+0000 7f3347400640 1 -- 172.21.15.44:0/17959228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334001c040 con 0x55af961f2b40 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 -- 172.21.15.44:0/17959228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 msgr2=0x55af961fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961fb530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55af9620b8e0 tx=0x7f3340009780 comp rx=0 tx=0).stop 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 -- 172.21.15.44:0/17959228 shutdown_connections 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af961f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af962045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 --2- 172.21.15.44:0/17959228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961fb530 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 -- 172.21.15.44:0/17959228 >> 172.21.15.44:0/17959228 conn(0x55af961ee0b0 msgr2=0x55af961f1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 -- 172.21.15.44:0/17959228 shutdown_connections 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.156+0000 7f3360c14740 1 -- 172.21.15.44:0/17959228 wait complete. 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 -- start start 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 Processor -- start 2024-04-20T11:21:51.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af96200360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96203380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55af9620b020 con 0x55af961f2b40 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af9620a950 con 0x55af961f86a0 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af9620b6f0 con 0x55af961f3530 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.157+0000 7f3360c14740 10 client.0 _ll_register_callbacks cb 0x7f334801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f334fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af96200360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f334fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54790/0 (socket says 172.21.15.44:54790) 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f334fe00640 1 -- 172.21.15.44:0/2924102201 learned_addr learned my addr 172.21.15.44:0/2924102201 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 -- 172.21.15.44:0/2924102201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 msgr2=0x55af96203380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96203380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 -- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 msgr2=0x55af961ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 -- 172.21.15.44:0/2924102201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af96203970 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3357400640 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af96200360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33480059f0 tx=0x7f33480056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d9d142e70f42a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.158+0000 7f3344c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.159+0000 7f3344c00640 1 -- 172.21.15.44:0/2924102201 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348027070 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.159+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3340004230 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.159+0000 7f3344c00640 1 -- 172.21.15.44:0/2924102201 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f33480039d0 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.159+0000 7f3344c00640 1 -- 172.21.15.44:0/2924102201 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3348003bd0 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.159+0000 7f3360c14740 10 client.25051 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.159+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af9626fe60 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3344c00640 1 -- 172.21.15.44:0/2924102201 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3340004230 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3344c00640 1 -- 172.21.15.44:0/2924102201 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f334802eaf0 con 0x55af961f3530 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3360c14740 10 client.25051 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:51.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3360c14740 10 client.25051 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-20T11:21:51.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3360c14740 20 client.25051 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:51.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3360c14740 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af962749a0 0x55af962736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3360c14740 4 client.25051 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:21:51.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "status"}) -- 0x55af96161260 con 0x55af962749a0 2024-04-20T11:21:51.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.160+0000 7f334fe00640 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af962749a0 0x55af962736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f334fe00640 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af962749a0 0x55af962736b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b46da46642debfb server_cookie=39a66f0554eca686 in_seq=0 out_seq=0 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3344c00640 10 client.25051 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3344c00640 1 -- 172.21.15.44:0/2924102201 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55af96161260 con 0x55af962749a0 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3344c00640 10 client.25051 handle_command_reply: tid=0 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 client.25051 shutdown 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 msgr2=0x55af96200360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af96200360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33480059f0 tx=0x7f33480056e0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 shutdown_connections 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55af962749a0 0x55af962736b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af961f86a0 0x55af96203380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af961f3530 0x55af96200360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 --2- 172.21.15.44:0/2924102201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55af961f2b40 0x55af961ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3344c00640 0 client.25051 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.161+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 >> 172.21.15.44:0/2924102201 conn(0x55af961ee0b0 msgr2=0x55af961ee490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 shutdown_connections 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f3360c14740 1 -- 172.21.15.44:0/2924102201 wait complete. 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f3360c14740 20 client.25051 trim_cache size 0 max 16384 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-04-20T11:21:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24562, 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 105.10764315999999, 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 121.095397143 2024-04-20T11:21:51.166 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f333c0a3bb0 msgr2=0x7f333c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f333c0a3bb0 0x7f333c0a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f33500096f0 tx=0x7f3350033a10 comp rx=0 tx=0).stop 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 msgr2=0x7f33581a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 0x7f33581a5d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3350033550 tx=0x7f3350033820 comp rx=0 tx=0).stop 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 shutdown_connections 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f333c0a3bb0 0x7f333c0a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f335810a050 0x7f33581a5d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3358109690 0x7f335819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 --2- 172.21.15.44:0/777329324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33581084a0 0x7f335819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.162+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 >> 172.21.15.44:0/777329324 conn(0x7f3358077430 msgr2=0x7f335810c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.163+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 shutdown_connections 2024-04-20T11:21:51.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.163+0000 7f335ca00640 1 -- 172.21.15.44:0/777329324 wait complete. 2024-04-20T11:21:51.179 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 -- 172.21.15.44:0/1333666812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5698100e50 msgr2=0x7f56981012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 --2- 172.21.15.44:0/1333666812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5698100e50 0x7f56981012b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5680009ca0 tx=0x7f568002c100 comp rx=0 tx=0).stop 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 -- 172.21.15.44:0/1333666812 shutdown_connections 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 --2- 172.21.15.44:0/1333666812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56981017f0 0x7f5698102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 --2- 172.21.15.44:0/1333666812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5698100e50 0x7f56981012b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 --2- 172.21.15.44:0/1333666812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56980ffd20 0x7f5698100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 -- 172.21.15.44:0/1333666812 >> 172.21.15.44:0/1333666812 conn(0x7f56980fb620 msgr2=0x7f56980fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 -- 172.21.15.44:0/1333666812 shutdown_connections 2024-04-20T11:21:51.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.385+0000 7f5697e00640 1 -- 172.21.15.44:0/1333666812 wait complete. 2024-04-20T11:21:51.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.386+0000 7f5697e00640 1 Processor -- start 2024-04-20T11:21:51.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.386+0000 7f5697e00640 1 -- start start 2024-04-20T11:21:51.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56980ffd20 0x7f569819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5698100e50 0x7f569819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56981017f0 0x7f56981a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f569810fe20 con 0x7f5698100e50 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f569810fca0 con 0x7f56981017f0 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f569810ffa0 con 0x7f56980ffd20 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5696000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5698100e50 0x7f569819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5697400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56981017f0 0x7f56981a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5696000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5698100e50 0x7f569819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54796/0 (socket says 172.21.15.44:54796) 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5696000640 1 -- 172.21.15.44:0/1392537652 learned_addr learned my addr 172.21.15.44:0/1392537652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.387+0000 7f5696a00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56980ffd20 0x7f569819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697400640 1 -- 172.21.15.44:0/1392537652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56980ffd20 msgr2=0x7f569819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697400640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56980ffd20 0x7f569819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697400640 1 -- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5698100e50 msgr2=0x7f569819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697400640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5698100e50 0x7f569819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697400640 1 -- 172.21.15.44:0/1392537652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56981a20c0 con 0x7f56981017f0 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697400640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56981017f0 0x7f56981a1a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f569000e970 tx=0x7f569000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f568f400640 1 -- 172.21.15.44:0/1392537652 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569000ccc0 con 0x7f56981017f0 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f568f400640 1 -- 172.21.15.44:0/1392537652 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5690005020 con 0x7f56981017f0 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f568f400640 1 -- 172.21.15.44:0/1392537652 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690004a20 con 0x7f56981017f0 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56981a23b0 con 0x7f56981017f0 2024-04-20T11:21:51.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.388+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981a28b0 con 0x7f56981017f0 2024-04-20T11:21:51.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.393+0000 7f56a05ca740 1 Processor -- start 2024-04-20T11:21:51.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.393+0000 7f568f400640 1 -- 172.21.15.44:0/1392537652 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5690004c40 con 0x7f56981017f0 2024-04-20T11:21:51.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.393+0000 7f56a05ca740 1 -- start start 2024-04-20T11:21:51.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6215b40 0x5649e6215f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e621b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e621db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f56a05ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649e617c9d0 con 0x5649e621b6a0 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f56a05ca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649e6123270 con 0x5649e6215b40 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f56a05ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649e6178990 con 0x5649e6216530 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e621db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e621db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54812/0 (socket says 172.21.15.44:54812) 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 -- 172.21.15.44:0/1226296963 learned_addr learned my addr 172.21.15.44:0/1226296963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6215b40 0x5649e6215f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696a00640 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e621b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 -- 172.21.15.44:0/1226296963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 msgr2=0x5649e621b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e621b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 -- 172.21.15.44:0/1226296963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6215b40 msgr2=0x5649e6215f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6215b40 0x5649e6215f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.394+0000 7f5696000640 1 -- 172.21.15.44:0/1226296963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e5e8f2d0 con 0x5649e621b6a0 2024-04-20T11:21:51.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f5696000640 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e621db30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f568002c710 tx=0x7f568002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4383f5420e9b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f568f400640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f567c0a3bb0 0x7f567c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f5687e00640 1 -- 172.21.15.44:0/1226296963 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568004a070 con 0x5649e621b6a0 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f5687e00640 1 -- 172.21.15.44:0/1226296963 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5680032940 con 0x5649e621b6a0 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f5687e00640 1 -- 172.21.15.44:0/1226296963 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680039040 con 0x5649e621b6a0 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f568f400640 1 -- 172.21.15.44:0/1392537652 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f56900d0410 con 0x7f56981017f0 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.395+0000 7f5696a00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f567c0a3bb0 0x7f567c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 -- 172.21.15.44:0/1226296963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 msgr2=0x5649e621db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f5696a00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f567c0a3bb0 0x7f567c0a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5649e6216930 tx=0x7f5688005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e621db30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f568002c710 tx=0x7f568002e040 comp rx=0 tx=0).stop 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 -- 172.21.15.44:0/1226296963 shutdown_connections 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e621db30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e621b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 --2- 172.21.15.44:0/1226296963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6215b40 0x5649e6215f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 -- 172.21.15.44:0/1226296963 >> 172.21.15.44:0/1226296963 conn(0x5649e62110b0 msgr2=0x5649e62114b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 -- 172.21.15.44:0/1226296963 shutdown_connections 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.396+0000 7f56a05ca740 1 -- 172.21.15.44:0/1226296963 wait complete. 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.397+0000 7f56a05ca740 1 -- start start 2024-04-20T11:21:51.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.397+0000 7f56a05ca740 1 Processor -- start 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.397+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e6220080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 0x5649e62205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6227600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f56a05ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649e621e5f0 con 0x5649e6216530 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f56a05ca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649e61365e0 con 0x5649e621b6a0 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f56a05ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649e621e770 con 0x5649e6215b40 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f56a05ca740 10 client.0 _ll_register_callbacks cb 0x5649e5e8eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:51.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 0x5649e62205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 0x5649e62205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54824/0 (socket says 172.21.15.44:54824) 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 -- 172.21.15.44:0/3270074709 learned_addr learned my addr 172.21.15.44:0/3270074709 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696a00640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e6220080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5697400640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6227600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 -- 172.21.15.44:0/3270074709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 msgr2=0x5649e6220080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e6220080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 -- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 msgr2=0x5649e6227600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6227600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.398+0000 7f5696000640 1 -- 172.21.15.44:0/3270074709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e5f596b0 con 0x5649e6216530 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f5696000640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 0x5649e62205c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5680000f80 tx=0x7f5680003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e792f015216c270 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f5684c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f5684c00640 1 -- 172.21.15.44:0/3270074709 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680053060 con 0x5649e6216530 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f5684c00640 1 -- 172.21.15.44:0/3270074709 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f56800043a0 con 0x5649e6216530 2024-04-20T11:21:51.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5680032940 con 0x5649e6216530 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f5684c00640 1 -- 172.21.15.44:0/3270074709 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5680039070 con 0x5649e6216530 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f56a05ca740 10 client.14955 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649e622a970 con 0x5649e6216530 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.399+0000 7f5684c00640 1 -- 172.21.15.44:0/3270074709 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5680032940 con 0x5649e6216530 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f5684c00640 1 -- 172.21.15.44:0/3270074709 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f5680058070 con 0x5649e6216530 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f56a05ca740 10 client.14955 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f56a05ca740 10 client.14955 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f56a05ca740 20 client.14955 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:51.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f56a05ca740 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e622bcc0 0x5649e622de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f56a05ca740 4 client.14955 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649e618a3f0 con 0x5649e622bcc0 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.400+0000 7f5696a00640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e622bcc0 0x5649e622de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.401+0000 7f5696a00640 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e622bcc0 0x5649e622de60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e3819bff1a1973 server_cookie=9e48e1f48cb41dba in_seq=0 out_seq=0 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.401+0000 7f5684c00640 10 client.14955 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f5684c00640 1 -- 172.21.15.44:0/3270074709 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649e618a3f0 con 0x5649e622bcc0 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f5684c00640 10 client.14955 handle_command_reply: tid=0 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 client.14955 shutdown 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 msgr2=0x5649e62205c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 0x5649e62205c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5680000f80 tx=0x7f5680003d70 comp rx=0 tx=0).stop 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 shutdown_connections 2024-04-20T11:21:51.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e622bcc0 0x5649e622de60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6227600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6216530 0x5649e62205c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 --2- 172.21.15.44:0/3270074709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e6220080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 >> 172.21.15.44:0/3270074709 conn(0x5649e62110b0 msgr2=0x5649e62103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.402+0000 7f5684c00640 0 client.14955 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.403+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 shutdown_connections 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.403+0000 7f56a05ca740 1 -- 172.21.15.44:0/3270074709 wait complete. 2024-04-20T11:21:51.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.403+0000 7f56a05ca740 20 client.14955 trim_cache size 0 max 16384 2024-04-20T11:21:51.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 Processor -- start 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 -- start start 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e621eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 0x5649e621f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e620fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649e622c1a0 con 0x5649e621b6a0 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649e622c870 con 0x5649e6216530 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f56a05ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649e622cf40 con 0x5649e6215b40 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f5696000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 0x5649e621f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f5696000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 0x5649e621f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54134/0 (socket says 172.21.15.44:54134) 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f5696000640 1 -- 172.21.15.44:0/747947271 learned_addr learned my addr 172.21.15.44:0/747947271 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.423+0000 7f5697400640 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e620fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696a00640 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e621eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696000640 1 -- 172.21.15.44:0/747947271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 msgr2=0x5649e621eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696000640 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e621eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696000640 1 -- 172.21.15.44:0/747947271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 msgr2=0x5649e620fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696000640 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e620fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696000640 1 -- 172.21.15.44:0/747947271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e6202950 con 0x5649e6216530 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696a00640 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e621eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5696000640 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 0x5649e621f050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5680005e30 tx=0x7f5680034b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa27bcb62511c2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5687400640 1 -- 172.21.15.44:0/747947271 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568005e070 con 0x5649e6216530 2024-04-20T11:21:51.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5687400640 1 -- 172.21.15.44:0/747947271 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5680004d40 con 0x5649e6216530 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.424+0000 7f5687400640 1 -- 172.21.15.44:0/747947271 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568004a040 con 0x5649e6216530 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 -- 172.21.15.44:0/747947271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 msgr2=0x5649e621f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 0x5649e621f050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5680005e30 tx=0x7f5680034b80 comp rx=0 tx=0).stop 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 -- 172.21.15.44:0/747947271 shutdown_connections 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e621b6a0 0x5649e620fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e6216530 0x5649e621f050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 --2- 172.21.15.44:0/747947271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6215b40 0x5649e621eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 -- 172.21.15.44:0/747947271 >> 172.21.15.44:0/747947271 conn(0x5649e62110b0 msgr2=0x5649e6211490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 -- 172.21.15.44:0/747947271 shutdown_connections 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.425+0000 7f56a05ca740 1 -- 172.21.15.44:0/747947271 wait complete. 2024-04-20T11:21:51.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 -- start start 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 Processor -- start 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 0x5649e62229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e6222f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6225eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649e622dce0 con 0x5649e6215b40 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649e622d610 con 0x5649e621b6a0 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649e622e3b0 con 0x5649e6216530 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.426+0000 7f56a05ca740 10 client.0 _ll_register_callbacks cb 0x7f5680048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 0x5649e62229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 0x5649e62229f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54840/0 (socket says 172.21.15.44:54840) 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 -- 172.21.15.44:0/2480813696 learned_addr learned my addr 172.21.15.44:0/2480813696 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5697400640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6225eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696000640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e6222f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 -- 172.21.15.44:0/2480813696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 msgr2=0x5649e6222f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e6222f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 -- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 msgr2=0x5649e6225eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6225eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 -- 172.21.15.44:0/2480813696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e62264a0 con 0x5649e6215b40 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5697400640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6225eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5696a00640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 0x5649e62229f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5688013010 tx=0x7f56880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ada30dd4a1a2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.427+0000 7f5684c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f5684c00640 1 -- 172.21.15.44:0/2480813696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688004470 con 0x5649e6215b40 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f5684c00640 1 -- 172.21.15.44:0/2480813696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f568802b380 con 0x5649e6215b40 2024-04-20T11:21:51.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5680004d40 con 0x5649e6215b40 2024-04-20T11:21:51.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f5684c00640 1 -- 172.21.15.44:0/2480813696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f568802b580 con 0x5649e6215b40 2024-04-20T11:21:51.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f56a05ca740 10 client.14961 fetch_fsmap learned FSMap version 26 2024-04-20T11:21:51.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649e6293170 con 0x5649e6215b40 2024-04-20T11:21:51.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.428+0000 7f5684c00640 1 -- 172.21.15.44:0/2480813696 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5680004d40 con 0x5649e6215b40 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f5684c00640 1 -- 172.21.15.44:0/2480813696 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f5688004a00 con 0x5649e6215b40 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f56a05ca740 10 client.14961 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f56a05ca740 10 client.14961 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f56a05ca740 20 client.14961 populate_metadata read hostname 'smithi044' 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f56a05ca740 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e6297e00 0x5649e6294240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f56a05ca740 4 client.14961 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:21:51.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "status"}) -- 0x5649e6184260 con 0x5649e6297e00 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.429+0000 7f5696000640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e6297e00 0x5649e6294240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f5696000640 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e6297e00 0x5649e6294240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f1a1979b89fa00 server_cookie=13d7cd10aa510e46 in_seq=0 out_seq=0 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f5684c00640 10 client.14961 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f5684c00640 1 -- 172.21.15.44:0/2480813696 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5649e6184260 con 0x5649e6297e00 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f5684c00640 10 client.14961 handle_command_reply: tid=0 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 client.14961 shutdown 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 msgr2=0x5649e62229f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 0x5649e62229f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5688013010 tx=0x7f56880046c0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 shutdown_connections 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649e6297e00 0x5649e6294240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649e621b6a0 0x5649e6225eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649e6216530 0x5649e6222f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f5684c00640 0 client.14961 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:21:51.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 --2- 172.21.15.44:0/2480813696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649e6215b40 0x5649e62229f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 >> 172.21.15.44:0/2480813696 conn(0x5649e62110b0 msgr2=0x5649e6217a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 shutdown_connections 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 1 -- 172.21.15.44:0/2480813696 wait complete. 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.430+0000 7f56a05ca740 20 client.14961 trim_cache size 0 max 16384 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24596, 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:21:51.435 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:21:51.436 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 108.741131329, 2024-04-20T11:21:51.436 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:21:51.436 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:21:51.436 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:21:51.436 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 114.021589572 2024-04-20T11:21:51.436 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f567c0a3bb0 msgr2=0x7f567c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f567c0a3bb0 0x7f567c0a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5649e6216930 tx=0x7f5688005f90 comp rx=0 tx=0).stop 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56981017f0 msgr2=0x7f56981a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56981017f0 0x7f56981a1a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f569000e970 tx=0x7f569000ee30 comp rx=0 tx=0).stop 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 shutdown_connections 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f567c0a3bb0 0x7f567c0a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56981017f0 0x7f56981a1a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5698100e50 0x7f569819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 --2- 172.21.15.44:0/1392537652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56980ffd20 0x7f569819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.431+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 >> 172.21.15.44:0/1392537652 conn(0x7f56980fb620 msgr2=0x7f56980fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:51.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.432+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 shutdown_connections 2024-04-20T11:21:51.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:51.432+0000 7f5697e00640 1 -- 172.21.15.44:0/1392537652 wait complete. 2024-04-20T11:21:51.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:51 smithi044 ceph-mon[16855]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:51.448 INFO:tasks.cephadm:Setup complete, yielding 2024-04-20T11:21:51.449 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:21:51.459 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:21:51.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph orch status' 2024-04-20T11:21:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:51 smithi078 ceph-mon[21464]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:51 smithi165 ceph-mon[21975]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:51.905 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:53.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:53 smithi044 ceph-mon[16855]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:53 smithi078 ceph-mon[21464]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:53 smithi165 ceph-mon[21975]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:21:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 -- 172.21.15.44:0/1267783165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570101330 msgr2=0x7fc570101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 --2- 172.21.15.44:0/1267783165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570101330 0x7fc570101750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc564009f90 tx=0x7fc56400b330 comp rx=0 tx=0).stop 2024-04-20T11:21:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 -- 172.21.15.44:0/1267783165 shutdown_connections 2024-04-20T11:21:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 --2- 172.21.15.44:0/1267783165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc570102650 0x7fc570102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 --2- 172.21.15.44:0/1267783165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570101330 0x7fc570101750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 --2- 172.21.15.44:0/1267783165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101c90 0x7fc570102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 -- 172.21.15.44:0/1267783165 >> 172.21.15.44:0/1267783165 conn(0x7fc5700fcf50 msgr2=0x7fc5700ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 -- 172.21.15.44:0/1267783165 shutdown_connections 2024-04-20T11:21:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.278+0000 7fc578600700 1 -- 172.21.15.44:0/1267783165 wait complete. 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.279+0000 7fc578600700 1 Processor -- start 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.279+0000 7fc578600700 1 -- start start 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc578600700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 0x7fc57019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc578600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc570101c90 0x7fc57019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc578600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570102650 0x7fc5701a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc578600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701109e0 con 0x7fc570101c90 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc578600700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc570110860 con 0x7fc570101330 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc578600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc570110b60 con 0x7fc570102650 2024-04-20T11:21:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc575c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 0x7fc57019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc575c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 0x7fc57019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54168/0 (socket says 172.21.15.44:54168) 2024-04-20T11:21:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc575200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc570101c90 0x7fc57019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc575c00700 1 -- 172.21.15.44:0/1232712989 learned_addr learned my addr 172.21.15.44:0/1232712989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.280+0000 7fc576600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570102650 0x7fc5701a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc575c00700 1 -- 172.21.15.44:0/1232712989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570102650 msgr2=0x7fc5701a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc575c00700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570102650 0x7fc5701a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc575c00700 1 -- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc570101c90 msgr2=0x7fc57019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc575c00700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc570101c90 0x7fc57019b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc575c00700 1 -- 172.21.15.44:0/1232712989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5701a2c60 con 0x7fc570101330 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc575c00700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 0x7fc57019b000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc564009b60 tx=0x7fc5640092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564021070 con 0x7fc570101330 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5701a2ef0 con 0x7fc570101330 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc564007bc0 con 0x7fc570101330 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.281+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564005330 con 0x7fc570101330 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.282+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5701a32e0 con 0x7fc570101330 2024-04-20T11:21:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.282+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc570069740 con 0x7fc570101330 2024-04-20T11:21:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.285+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc564024070 con 0x7fc570101330 2024-04-20T11:21:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.286+0000 7fc56e000700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5580a3c40 0x7fc5580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.286+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fc56400f020 con 0x7fc570101330 2024-04-20T11:21:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.286+0000 7fc575200700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5580a3c40 0x7fc5580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.287+0000 7fc575200700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5580a3c40 0x7fc5580a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc55c005fd0 tx=0x7fc55c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.288+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc56402b020 con 0x7fc570101330 2024-04-20T11:21:54.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.540+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc570062e90 con 0x7fc5580a3c40 2024-04-20T11:21:54.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.542+0000 7fc56e000700 1 -- 172.21.15.44:0/1232712989 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc570062e90 con 0x7fc5580a3c40 2024-04-20T11:21:54.542 INFO:teuthology.orchestra.run.smithi044.stdout:Backend: cephadm 2024-04-20T11:21:54.542 INFO:teuthology.orchestra.run.smithi044.stdout:Available: Yes 2024-04-20T11:21:54.542 INFO:teuthology.orchestra.run.smithi044.stdout:Paused: No 2024-04-20T11:21:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5580a3c40 msgr2=0x7fc5580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5580a3c40 0x7fc5580a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc55c005fd0 tx=0x7fc55c009310 comp rx=0 tx=0).stop 2024-04-20T11:21:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 msgr2=0x7fc57019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 0x7fc57019b000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc564009b60 tx=0x7fc5640092e0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 shutdown_connections 2024-04-20T11:21:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc5580a3c40 0x7fc5580a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc570102650 0x7fc5701a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc570101330 0x7fc57019b000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 --2- 172.21.15.44:0/1232712989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc570101c90 0x7fc57019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.545+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 >> 172.21.15.44:0/1232712989 conn(0x7fc5700fcf50 msgr2=0x7fc5700fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.546+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 shutdown_connections 2024-04-20T11:21:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:54.546+0000 7fc578600700 1 -- 172.21.15.44:0/1232712989 wait complete. 2024-04-20T11:21:55.362 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-20T11:21:55.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:55 smithi078 ceph-mon[21464]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:55.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:55 smithi078 ceph-mon[21464]: from='client.25085 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:55.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:55 smithi044 ceph-mon[16855]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:55 smithi044 ceph-mon[16855]: from='client.25085 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:55 smithi165 ceph-mon[21975]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:55 smithi165 ceph-mon[21975]: from='client.25085 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:55.810 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:57.432 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:57 smithi044 ceph-mon[16855]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:57 smithi078 ceph-mon[21464]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:57 smithi165 ceph-mon[21975]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:21:57.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 -- 172.21.15.44:0/1644578115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 msgr2=0x7f5468101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:57.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 --2- 172.21.15.44:0/1644578115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f5468101100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5458009990 tx=0x7f5458009ca0 comp rx=0 tx=0).stop 2024-04-20T11:21:57.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 -- 172.21.15.44:0/1644578115 shutdown_connections 2024-04-20T11:21:57.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 --2- 172.21.15.44:0/1644578115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468102880 0x7f5468109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 --2- 172.21.15.44:0/1644578115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f5468101100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 --2- 172.21.15.44:0/1644578115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468101ee0 0x7f5468102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 -- 172.21.15.44:0/1644578115 >> 172.21.15.44:0/1644578115 conn(0x7f54680fc2a0 msgr2=0x7f54680fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 -- 172.21.15.44:0/1644578115 shutdown_connections 2024-04-20T11:21:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.948+0000 7f546ce00700 1 -- 172.21.15.44:0/1644578115 wait complete. 2024-04-20T11:21:57.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 Processor -- start 2024-04-20T11:21:57.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 -- start start 2024-04-20T11:21:57.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f546819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468101ee0 0x7f546819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468102880 0x7f54681a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f546810d760 con 0x7f5468100ce0 2024-04-20T11:21:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f546810d5e0 con 0x7f5468101ee0 2024-04-20T11:21:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.949+0000 7f546ce00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f546810d8e0 con 0x7f5468102880 2024-04-20T11:21:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f546819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:57.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f546819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54880/0 (socket says 172.21.15.44:54880) 2024-04-20T11:21:57.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465e00700 1 -- 172.21.15.44:0/1940084372 learned_addr learned my addr 172.21.15.44:0/1940084372 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:21:57.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465400700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468101ee0 0x7f546819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:57.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5466800700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468102880 0x7f54681a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465400700 1 -- 172.21.15.44:0/1940084372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468102880 msgr2=0x7f54681a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465400700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468102880 0x7f54681a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465400700 1 -- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 msgr2=0x7f546819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465400700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f546819afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465400700 1 -- 172.21.15.44:0/1940084372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54681a2c30 con 0x7f5468101ee0 2024-04-20T11:21:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5466800700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468102880 0x7f54681a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.950+0000 7f5465e00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f546819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:21:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.951+0000 7f5465400700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468101ee0 0x7f546819b510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f545400e9f0 tx=0x7f545400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.951+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545400c9c0 con 0x7f5468101ee0 2024-04-20T11:21:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.951+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f545400cb60 con 0x7f5468101ee0 2024-04-20T11:21:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.951+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54681a2f20 con 0x7f5468101ee0 2024-04-20T11:21:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.951+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454018710 con 0x7f5468101ee0 2024-04-20T11:21:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.951+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54681a33d0 con 0x7f5468101ee0 2024-04-20T11:21:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.952+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5468069770 con 0x7f5468101ee0 2024-04-20T11:21:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.955+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f54540188b0 con 0x7f5468101ee0 2024-04-20T11:21:57.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.956+0000 7f545ea00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f544c0a3c40 0x7f544c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:21:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.956+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f54540d0960 con 0x7f5468101ee0 2024-04-20T11:21:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.956+0000 7f5465e00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f544c0a3c40 0x7f544c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:21:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.957+0000 7f5465e00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f544c0a3c40 0x7f544c0a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f545800b2e0 tx=0x7f545801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:21:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:57.958+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5468069770 con 0x7f5468101ee0 2024-04-20T11:21:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.201+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f54680fe3d0 con 0x7f544c0a3c40 2024-04-20T11:21:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.241+0000 7f545ea00700 1 -- 172.21.15.44:0/1940084372 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f54680fe3d0 con 0x7f544c0a3c40 2024-04-20T11:21:58.242 INFO:teuthology.orchestra.run.smithi044.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T11:21:58.242 INFO:teuthology.orchestra.run.smithi044.stdout:mds.a smithi044 running (2m) 26s ago 2m 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 c83028cae8ad 2024-04-20T11:21:58.242 INFO:teuthology.orchestra.run.smithi044.stdout:mds.b smithi078 running (2m) 52s ago 2m 26.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 731415ecc45a 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.c smithi165 running (2m) 52s ago 2m 17.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 da122010b2c8 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.d smithi044 running (2m) 26s ago 2m 16.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 cdbcc3e69204 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.e smithi078 running (2m) 52s ago 2m 17.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 8c5dd656b8dd 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.f smithi165 running (2m) 52s ago 2m 17.1M - 19.0.0-3133-ga8c84f59 76bc6b91b726 76a599a47916 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.g smithi044 running (2m) 26s ago 2m 16.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 0c0b08834976 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.h smithi078 running (2m) 52s ago 2m 19.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 84fbf08d33c4 2024-04-20T11:21:58.243 INFO:teuthology.orchestra.run.smithi044.stdout:mds.i smithi165 running (2m) 52s ago 2m 19.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 1e7b0f13131a 2024-04-20T11:21:58.244 INFO:teuthology.orchestra.run.smithi044.stdout:mds.j smithi044 running (2m) 26s ago 2m 15.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 5d318443ab46 2024-04-20T11:21:58.244 INFO:teuthology.orchestra.run.smithi044.stdout:mds.k smithi078 running (2m) 52s ago 2m 18.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 af72995753b6 2024-04-20T11:21:58.244 INFO:teuthology.orchestra.run.smithi044.stdout:mds.l smithi165 running (2m) 52s ago 2m 16.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 c3f64b075a58 2024-04-20T11:21:58.244 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.x smithi044 *:9283,8765 running (11m) 26s ago 11m 514M - 19.0.0-3133-ga8c84f59 76bc6b91b726 afe2669f9fac 2024-04-20T11:21:58.244 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.y smithi078 *:8443,8765 running (8m) 52s ago 8m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 aebacfb96992 2024-04-20T11:21:58.244 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.z smithi165 *:8443,8765 running (8m) 52s ago 8m 454M - 19.0.0-3133-ga8c84f59 76bc6b91b726 caa509b16bb3 2024-04-20T11:21:58.245 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a smithi044 running (11m) 26s ago 11m 48.5M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 fc8cefb00f96 2024-04-20T11:21:58.245 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b smithi078 running (9m) 52s ago 9m 45.0M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 b8f80487171f 2024-04-20T11:21:58.245 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c smithi165 running (9m) 52s ago 9m 43.8M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 dad7540733eb 2024-04-20T11:21:58.245 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 smithi044 running (8m) 26s ago 8m 44.1M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 0672bb214e03 2024-04-20T11:21:58.245 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 smithi078 running (7m) 52s ago 7m 56.6M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 611ceb3023b3 2024-04-20T11:21:58.245 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 smithi165 running (7m) 52s ago 7m 57.4M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 99b5f688a932 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 smithi044 running (6m) 26s ago 6m 41.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 7e9087503ce6 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 smithi078 running (6m) 52s ago 6m 53.5M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 3acb431990db 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 smithi165 running (6m) 52s ago 6m 45.7M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 92faf21cb1d4 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 smithi044 running (5m) 26s ago 5m 51.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 21143043d532 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 smithi078 running (4m) 52s ago 4m 44.0M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 58b0a3f52806 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 smithi165 running (4m) 52s ago 4m 43.1M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 3f4b99bfbd26 2024-04-20T11:21:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 smithi044 running (3m) 26s ago 3m 51.4M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 f1a4ca5c8bb9 2024-04-20T11:21:58.247 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 smithi078 running (3m) 52s ago 3m 42.1M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 83e3c596f304 2024-04-20T11:21:58.247 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 smithi165 running (3m) 52s ago 3m 40.9M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 148ed31345fb 2024-04-20T11:21:58.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.244+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f544c0a3c40 msgr2=0x7f544c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:58.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.244+0000 7f546ce00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f544c0a3c40 0x7f544c0a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f545800b2e0 tx=0x7f545801a040 comp rx=0 tx=0).stop 2024-04-20T11:21:58.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.244+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468101ee0 msgr2=0x7f546819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:21:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.244+0000 7f546ce00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468101ee0 0x7f546819b510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f545400e9f0 tx=0x7f545400ed00 comp rx=0 tx=0).stop 2024-04-20T11:21:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 shutdown_connections 2024-04-20T11:21:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f544c0a3c40 0x7f544c0a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5468102880 0x7f54681a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5468100ce0 0x7f546819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 --2- 172.21.15.44:0/1940084372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5468101ee0 0x7f546819b510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:21:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 >> 172.21.15.44:0/1940084372 conn(0x7f54680fc2a0 msgr2=0x7f54680fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:21:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 shutdown_connections 2024-04-20T11:21:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:21:58.245+0000 7f546ce00700 1 -- 172.21.15.44:0/1940084372 wait complete. 2024-04-20T11:21:59.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-20T11:21:59.611 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:21:59.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:59 smithi078 ceph-mon[21464]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:59.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:21:59 smithi078 ceph-mon[21464]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:59.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:59 smithi044 ceph-mon[16855]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:59.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:21:59 smithi044 ceph-mon[16855]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:21:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:59 smithi165 ceph-mon[21975]: pgmap v381: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:21:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:21:59 smithi165 ceph-mon[21975]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:01.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:01 smithi044 ceph-mon[16855]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:01.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:01 smithi078 ceph-mon[21464]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:01 smithi165 ceph-mon[21975]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.894+0000 7f04b0600700 1 -- 172.21.15.44:0/3315223465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8106890 msgr2=0x7f04a810d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.894+0000 7f04b0600700 1 --2- 172.21.15.44:0/3315223465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8106890 0x7f04a810d140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009fc0 tx=0x7f04a000b330 comp rx=0 tx=0).stop 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.894+0000 7f04b0600700 1 -- 172.21.15.44:0/3315223465 shutdown_connections 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.894+0000 7f04b0600700 1 --2- 172.21.15.44:0/3315223465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8106890 0x7f04a810d140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.894+0000 7f04b0600700 1 --2- 172.21.15.44:0/3315223465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a8105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.894+0000 7f04b0600700 1 --2- 172.21.15.44:0/3315223465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8105ef0 0x7f04a8106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.895+0000 7f04b0600700 1 -- 172.21.15.44:0/3315223465 >> 172.21.15.44:0/3315223465 conn(0x7f04a81002b0 msgr2=0x7f04a81026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.895+0000 7f04b0600700 1 -- 172.21.15.44:0/3315223465 shutdown_connections 2024-04-20T11:22:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.895+0000 7f04b0600700 1 -- 172.21.15.44:0/3315223465 wait complete. 2024-04-20T11:22:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 Processor -- start 2024-04-20T11:22:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 -- start start 2024-04-20T11:22:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8105ef0 0x7f04a819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 0x7f04a81a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8110a70 con 0x7f04a8105ef0 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04a81108f0 con 0x7f04a8104cf0 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.896+0000 7f04b0600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8110bf0 con 0x7f04a8106890 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04ae600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 0x7f04a81a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04ad200700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8105ef0 0x7f04a819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04ae600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 0x7f04a81a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:43314/0 (socket says 172.21.15.44:43314) 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a819af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39214/0 (socket says 172.21.15.44:39214) 2024-04-20T11:22:01.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04ae600700 1 -- 172.21.15.44:0/1806860429 learned_addr learned my addr 172.21.15.44:0/1806860429 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 -- 172.21.15.44:0/1806860429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 msgr2=0x7f04a81a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 0x7f04a81a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 -- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8105ef0 msgr2=0x7f04a819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8105ef0 0x7f04a819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04adc00700 1 -- 172.21.15.44:0/1806860429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a81a2b90 con 0x7f04a8104cf0 2024-04-20T11:22:01.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.897+0000 7f04ae600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 0x7f04a81a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.898+0000 7f04adc00700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a819af30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e9c0 tx=0x7f049c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.898+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c00cb70 con 0x7f04a8104cf0 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.898+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f049c00ee50 con 0x7f04a8104cf0 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.898+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c018710 con 0x7f04a8104cf0 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.898+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a81a2e80 con 0x7f04a8104cf0 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.898+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a81a3280 con 0x7f04a8104cf0 2024-04-20T11:22:01.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.899+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a8069740 con 0x7f04a8104cf0 2024-04-20T11:22:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.902+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f049c01f020 con 0x7f04a8104cf0 2024-04-20T11:22:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.903+0000 7f04a6000700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04900a3cc0 0x7f04900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.903+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f049c0d06f0 con 0x7f04a8104cf0 2024-04-20T11:22:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.904+0000 7f04ad200700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04900a3cc0 0x7f04900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.904+0000 7f04ad200700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04900a3cc0 0x7f04900a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0494005fd0 tx=0x7f0494009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:01.905+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f049c09d0d0 con 0x7f04a8104cf0 2024-04-20T11:22:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.146+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f04a8062e90 con 0x7f04900a3cc0 2024-04-20T11:22:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.164+0000 7f04a6000700 1 -- 172.21.15.44:0/1806860429 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f04a8062e90 con 0x7f04900a3cc0 2024-04-20T11:22:02.164 INFO:teuthology.orchestra.run.smithi044.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-20T11:22:02.164 INFO:teuthology.orchestra.run.smithi044.stdout:mds.all 12/12 56s ago 2m smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-20T11:22:02.164 INFO:teuthology.orchestra.run.smithi044.stdout:mgr 3/3 56s ago 8m smithi044=x;smithi078=y;smithi165=z;count:3 2024-04-20T11:22:02.164 INFO:teuthology.orchestra.run.smithi044.stdout:mon 3/3 56s ago 9m smithi044:172.21.15.44=a;smithi078:172.21.15.78=b;smithi165:172.21.15.165=c;count:3 2024-04-20T11:22:02.164 INFO:teuthology.orchestra.run.smithi044.stdout:osd 12 56s ago - 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04900a3cc0 msgr2=0x7f04900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04900a3cc0 0x7f04900a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0494005fd0 tx=0x7f0494009310 comp rx=0 tx=0).stop 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 msgr2=0x7f04a819af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a819af30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e9c0 tx=0x7f049c00ecd0 comp rx=0 tx=0).stop 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 shutdown_connections 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f04900a3cc0 0x7f04900a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04a8106890 0x7f04a81a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04a8104cf0 0x7f04a819af30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 --2- 172.21.15.44:0/1806860429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a8105ef0 0x7f04a819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 >> 172.21.15.44:0/1806860429 conn(0x7f04a81002b0 msgr2=0x7f04a8109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 shutdown_connections 2024-04-20T11:22:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:02.167+0000 7f04b0600700 1 -- 172.21.15.44:0/1806860429 wait complete. 2024-04-20T11:22:03.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-20T11:22:03.530 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:22:03.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:03 smithi078 ceph-mon[21464]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:03 smithi078 ceph-mon[21464]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:03.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:03 smithi044 ceph-mon[16855]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:03.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:03 smithi044 ceph-mon[16855]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:03 smithi165 ceph-mon[21975]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:03 smithi165 ceph-mon[21975]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:05.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:05 smithi044 ceph-mon[16855]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:05.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:05 smithi078 ceph-mon[21464]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:05 smithi165 ceph-mon[21975]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:05.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 -- 172.21.15.44:0/2116396350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 msgr2=0x7fc81c106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:05.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 --2- 172.21.15.44:0/2116396350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c106250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc808009990 tx=0x7fc808009ca0 comp rx=0 tx=0).stop 2024-04-20T11:22:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 -- 172.21.15.44:0/2116396350 shutdown_connections 2024-04-20T11:22:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 --2- 172.21.15.44:0/2116396350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c106790 0x7fc81c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 --2- 172.21.15.44:0/2116396350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c104bf0 0x7fc81c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 --2- 172.21.15.44:0/2116396350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c106250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 -- 172.21.15.44:0/2116396350 >> 172.21.15.44:0/2116396350 conn(0x7fc81c100190 msgr2=0x7fc81c1025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 -- 172.21.15.44:0/2116396350 shutdown_connections 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.835+0000 7fc822000700 1 -- 172.21.15.44:0/2116396350 wait complete. 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.836+0000 7fc822000700 1 Processor -- start 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 -- start start 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 0x7fc81c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c106790 0x7fc81c1a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1108f0 con 0x7fc81c106790 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c110770 con 0x7fc81c104bf0 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc822000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c110a70 con 0x7fc81c105df0 2024-04-20T11:22:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.837+0000 7fc820c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 0x7fc81c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc81be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc821600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c106790 0x7fc81c1a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 0x7fc81c19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39236/0 (socket says 172.21.15.44:39236) 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 -- 172.21.15.44:0/3532406713 learned_addr learned my addr 172.21.15.44:0/3532406713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 -- 172.21.15.44:0/3532406713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 msgr2=0x7fc81c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 -- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c106790 msgr2=0x7fc81c1a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c106790 0x7fc81c1a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 -- 172.21.15.44:0/3532406713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c1a2ae0 con 0x7fc81c104bf0 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc81be00700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc821600700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c106790 0x7fc81c1a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.838+0000 7fc820c00700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 0x7fc81c19ae80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e970 tx=0x7fc81000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.839+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810004220 con 0x7fc81c104bf0 2024-04-20T11:22:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.839+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc81000cce0 con 0x7fc81c104bf0 2024-04-20T11:22:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.839+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c1a2dd0 con 0x7fc81c104bf0 2024-04-20T11:22:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.839+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810010430 con 0x7fc81c104bf0 2024-04-20T11:22:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.839+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1a3220 con 0x7fc81c104bf0 2024-04-20T11:22:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.840+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc81c069740 con 0x7fc81c104bf0 2024-04-20T11:22:05.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.842+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc81000c7e0 con 0x7fc81c104bf0 2024-04-20T11:22:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.843+0000 7fc819600700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc8040a3c40 0x7fc8040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.844+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fc8100d0590 con 0x7fc81c104bf0 2024-04-20T11:22:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.844+0000 7fc81be00700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc8040a3c40 0x7fc8040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:05.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.844+0000 7fc81be00700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc8040a3c40 0x7fc8040a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b2e0 tx=0x7fc80800b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:05.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:05.847+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc81009cf70 con 0x7fc81c104bf0 2024-04-20T11:22:06.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.084+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc81c062e90 con 0x7fc8040a3c40 2024-04-20T11:22:06.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.087+0000 7fc819600700 1 -- 172.21.15.44:0/3532406713 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc81c062e90 con 0x7fc8040a3c40 2024-04-20T11:22:06.087 INFO:teuthology.orchestra.run.smithi044.stdout:HOST ADDR LABELS STATUS 2024-04-20T11:22:06.087 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 172.21.15.44 2024-04-20T11:22:06.087 INFO:teuthology.orchestra.run.smithi044.stdout:smithi078 172.21.15.78 2024-04-20T11:22:06.087 INFO:teuthology.orchestra.run.smithi044.stdout:smithi165 172.21.15.165 2024-04-20T11:22:06.087 INFO:teuthology.orchestra.run.smithi044.stdout:3 hosts in cluster 2024-04-20T11:22:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.090+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc8040a3c40 msgr2=0x7fc8040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.090+0000 7fc822000700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc8040a3c40 0x7fc8040a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b2e0 tx=0x7fc80800b8f0 comp rx=0 tx=0).stop 2024-04-20T11:22:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.090+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 msgr2=0x7fc81c19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.090+0000 7fc822000700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 0x7fc81c19ae80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e970 tx=0x7fc81000ec80 comp rx=0 tx=0).stop 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 shutdown_connections 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fc8040a3c40 0x7fc8040a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc81c106790 0x7fc81c1a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c104bf0 0x7fc81c19ae80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 --2- 172.21.15.44:0/3532406713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc81c105df0 0x7fc81c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 >> 172.21.15.44:0/3532406713 conn(0x7fc81c100190 msgr2=0x7fc81c1090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 shutdown_connections 2024-04-20T11:22:06.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:06.091+0000 7fc822000700 1 -- 172.21.15.44:0/3532406713 wait complete. 2024-04-20T11:22:06.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-20T11:22:07.336 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:22:07.644 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:07 smithi044 ceph-mon[16855]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:07.645 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:07 smithi044 ceph-mon[16855]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:07.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:07 smithi078 ceph-mon[21464]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:07.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:07 smithi078 ceph-mon[21464]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:07 smithi165 ceph-mon[21975]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:07 smithi165 ceph-mon[21975]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 -- 172.21.15.44:0/2432933469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c104690 msgr2=0x7f587c10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 --2- 172.21.15.44:0/2432933469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c104690 0x7f587c10af40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f587400b0d0 tx=0x7f587400b3e0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.625 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:09 smithi044 ceph-mon[16855]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 -- 172.21.15.44:0/2432933469 shutdown_connections 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 --2- 172.21.15.44:0/2432933469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c104690 0x7f587c10af40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 --2- 172.21.15.44:0/2432933469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c102af0 0x7f587c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 --2- 172.21.15.44:0/2432933469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587c103cf0 0x7f587c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.624+0000 7f5884000700 1 -- 172.21.15.44:0/2432933469 >> 172.21.15.44:0/2432933469 conn(0x7f587c0fe090 msgr2=0x7f587c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.625+0000 7f5884000700 1 -- 172.21.15.44:0/2432933469 shutdown_connections 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.625+0000 7f5884000700 1 -- 172.21.15.44:0/2432933469 wait complete. 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 Processor -- start 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 -- start start 2024-04-20T11:22:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 0x7f587c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587c103cf0 0x7f587c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c104690 0x7f587c1a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10e870 con 0x7f587c102af0 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10e6f0 con 0x7f587c104690 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.626+0000 7f5884000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10e9f0 con 0x7f587c103cf0 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5881600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 0x7f587c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5882000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c104690 0x7f587c1a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5881600700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 0x7f587c19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50108/0 (socket says 172.21.15.44:50108) 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5881600700 1 -- 172.21.15.44:0/882972994 learned_addr learned my addr 172.21.15.44:0/882972994 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5880c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587c103cf0 0x7f587c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5882000700 1 -- 172.21.15.44:0/882972994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587c103cf0 msgr2=0x7f587c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5882000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587c103cf0 0x7f587c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5882000700 1 -- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 msgr2=0x7f587c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5882000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 0x7f587c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5882000700 1 -- 172.21.15.44:0/882972994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c1a2ba0 con 0x7f587c104690 2024-04-20T11:22:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.627+0000 7f5881600700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 0x7f587c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.628+0000 7f5882000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c104690 0x7f587c1a2500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f587400b770 tx=0x7f5874007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.628+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874003e10 con 0x7f587c104690 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.628+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c1a2e30 con 0x7f587c104690 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.628+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5874022070 con 0x7f587c104690 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.628+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c1a3220 con 0x7f587c104690 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.628+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587401b6c0 con 0x7f587c104690 2024-04-20T11:22:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.629+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c069740 con 0x7f587c104690 2024-04-20T11:22:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.631+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f587401d470 con 0x7f587c104690 2024-04-20T11:22:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.633+0000 7f587a000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f58640a3cc0 0x7f58640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.633+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f58740d1310 con 0x7f587c104690 2024-04-20T11:22:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.633+0000 7f5881600700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f58640a3cc0 0x7f58640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.634+0000 7f5881600700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f58640a3cc0 0x7f58640a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5870005950 tx=0x7f5870009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:09.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.637+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f587409dd80 con 0x7f587c104690 2024-04-20T11:22:09.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:09 smithi078 ceph-mon[21464]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:22:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:09 smithi165 ceph-mon[21975]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:22:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.884+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 --> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f587c0763a0 con 0x7f58640a3cc0 2024-04-20T11:22:09.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.887+0000 7f587a000700 1 -- 172.21.15.44:0/882972994 <== mgr.14152 v2:172.21.15.44:6800/2425002048 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f587c0763a0 con 0x7f58640a3cc0 2024-04-20T11:22:09.886 INFO:teuthology.orchestra.run.smithi044.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-20T11:22:09.887 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-20T11:22:09.887 INFO:teuthology.orchestra.run.smithi044.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-20T11:22:09.887 INFO:teuthology.orchestra.run.smithi044.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-20T11:22:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f58640a3cc0 msgr2=0x7f58640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f58640a3cc0 0x7f58640a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5870005950 tx=0x7f5870009260 comp rx=0 tx=0).stop 2024-04-20T11:22:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c104690 msgr2=0x7f587c1a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c104690 0x7f587c1a2500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f587400b770 tx=0x7f5874007d10 comp rx=0 tx=0).stop 2024-04-20T11:22:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 shutdown_connections 2024-04-20T11:22:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f58640a3cc0 0x7f58640a6180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f587c104690 0x7f587c1a2500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c102af0 0x7f587c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 --2- 172.21.15.44:0/882972994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587c103cf0 0x7f587c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:09.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 >> 172.21.15.44:0/882972994 conn(0x7f587c0fe090 msgr2=0x7f587c074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:09.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 shutdown_connections 2024-04-20T11:22:09.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:09.890+0000 7f5884000700 1 -- 172.21.15.44:0/882972994 wait complete. 2024-04-20T11:22:10.772 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T11:22:10.781 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:22:10.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T11:22:11.262 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:22:11.628 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:11 smithi044 ceph-mon[16855]: from='client.25109 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:11.629 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:11 smithi044 ceph-mon[16855]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:11.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:11 smithi078 ceph-mon[21464]: from='client.25109 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:11.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:11 smithi078 ceph-mon[21464]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:11 smithi165 ceph-mon[21975]: from='client.25109 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:11 smithi165 ceph-mon[21975]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:13.500 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:13 smithi044 ceph-mon[16855]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 -- 172.21.15.44:0/651795793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84105ef0 msgr2=0x7f0c84106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 --2- 172.21.15.44:0/651795793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84105ef0 0x7f0c84106350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009990 tx=0x7f0c70009ca0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 -- 172.21.15.44:0/651795793 shutdown_connections 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 --2- 172.21.15.44:0/651795793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84106890 0x7f0c8410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 --2- 172.21.15.44:0/651795793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84104cf0 0x7f0c84105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 --2- 172.21.15.44:0/651795793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84105ef0 0x7f0c84106350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 -- 172.21.15.44:0/651795793 >> 172.21.15.44:0/651795793 conn(0x7f0c841002b0 msgr2=0x7f0c841026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 -- 172.21.15.44:0/651795793 shutdown_connections 2024-04-20T11:22:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.541+0000 7f0c88e00700 1 -- 172.21.15.44:0/651795793 wait complete. 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.542+0000 7f0c88e00700 1 Processor -- start 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.542+0000 7f0c88e00700 1 -- start start 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.542+0000 7f0c88e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 0x7f0c8419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.542+0000 7f0c88e00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84105ef0 0x7f0c8419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c88e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84106890 0x7f0c841a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c88e00700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84110a40 con 0x7f0c84105ef0 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c88e00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841108c0 con 0x7f0c84104cf0 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c88e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84110bc0 con 0x7f0c84106890 2024-04-20T11:22:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84105ef0 0x7f0c8419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 0x7f0c8419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84105ef0 0x7f0c8419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50136/0 (socket says 172.21.15.44:50136) 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 0x7f0c8419afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59658/0 (socket says 172.21.15.44:59658) 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81400700 1 -- 172.21.15.44:0/2903962677 learned_addr learned my addr 172.21.15.44:0/2903962677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c82800700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84106890 0x7f0c841a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 -- 172.21.15.44:0/2903962677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84106890 msgr2=0x7f0c841a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84106890 0x7f0c841a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 -- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84105ef0 msgr2=0x7f0c8419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84105ef0 0x7f0c8419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c81e00700 1 -- 172.21.15.44:0/2903962677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c841a2c30 con 0x7f0c84104cf0 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.543+0000 7f0c82800700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84106890 0x7f0c841a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.544+0000 7f0c81e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 0x7f0c8419afd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9c0 tx=0x7f0c7400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.544+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400cb70 con 0x7f0c84104cf0 2024-04-20T11:22:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.544+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0c7400cd10 con 0x7f0c84104cf0 2024-04-20T11:22:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.544+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c740187a0 con 0x7f0c84104cf0 2024-04-20T11:22:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.544+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841a2f20 con 0x7f0c84104cf0 2024-04-20T11:22:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.544+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841a3320 con 0x7f0c84104cf0 2024-04-20T11:22:13.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.545+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8402df90 con 0x7f0c84104cf0 2024-04-20T11:22:13.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.547+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c7401f020 con 0x7f0c84104cf0 2024-04-20T11:22:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.549+0000 7f0c7aa00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c680a3cc0 0x7f0c680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.549+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f0c740d0210 con 0x7f0c84104cf0 2024-04-20T11:22:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.549+0000 7f0c81400700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c680a3cc0 0x7f0c680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.550+0000 7f0c81400700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c680a3cc0 0x7f0c680a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b2e0 tx=0x7f0c7000b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:13.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.553+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c7409cbf0 con 0x7f0c84104cf0 2024-04-20T11:22:13.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:13 smithi078 ceph-mon[21464]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:13 smithi165 ceph-mon[21975]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.830+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0c840637b0 con 0x7f0c84104cf0 2024-04-20T11:22:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.831+0000 7f0c7aa00700 1 -- 172.21.15.44:0/2903962677 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3097 (secure 0 0 0) 0x7f0c7409c4f0 con 0x7f0c84104cf0 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:e26 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:legacy client fscid: 1 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:Filesystem 'cephfs' (1) 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:fs_name cephfs 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 26 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-04-20T11:20:02.149413+0000 2024-04-20T11:22:13.832 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-04-20T11:20:09.840804+0000 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:tableserver 0 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:root 0 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:session_timeout 300 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:session_autoclose 300 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:max_file_size 1099511627776 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:max_xattr_size 65536 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:required_client_features {} 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure 0 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure_osd_epoch 0 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:snap_rstat 1 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:max_mds 5 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:in 0,1,2,3,4 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:up {0=24596,1=24562,2=24550,3=24506,4=24494} 2024-04-20T11:22:13.833 INFO:teuthology.orchestra.run.smithi044.stdout:failed 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:damaged 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:stopped 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:data_pools [3] 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:metadata_pool 2 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:inline_data disabled 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:balancer 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:bal_rank_mask -1 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:standby_count_wanted 0 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:qdb_cluster leader: 24596 members: 24550,24494,24596,24506,24562 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b{0:24596} state up:active seq 3 addr [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.h{0:24539} state up:standby-replay seq 1 addr [v2:172.21.15.78:6836/2547716992,v1:172.21.15.78:6837/2547716992] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.g{1:24562} state up:active seq 6 addr [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c{1:24523} state up:standby-replay seq 1 addr [v2:172.21.15.165:6836/2414137592,v1:172.21.15.165:6837/2414137592] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.834 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.i{2:24550} state up:active seq 7 addr [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f{2:24511} state up:standby-replay seq 1 addr [v2:172.21.15.165:6834/1721181859,v1:172.21.15.165:6835/1721181859] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e{3:24506} state up:active seq 10 addr [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.j{3:24476} state up:standby-replay seq 1 addr [v2:172.21.15.44:6834/858649103,v1:172.21.15.44:6835/858649103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d{4:24494} state up:active seq 11 addr [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.k{4:24458} state up:standby-replay seq 1 addr [v2:172.21.15.78:6832/715928335,v1:172.21.15.78:6833/715928335] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:Standby daemons: 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a{-1:14475} state up:standby seq 1 addr [v2:172.21.15.44:6840/503506102,v1:172.21.15.44:6841/503506102] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.835 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.165:6832/3129181151,v1:172.21.15.165:6833/3129181151] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T11:22:13.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.834+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c680a3cc0 msgr2=0x7f0c680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:13.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.834+0000 7f0c88e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c680a3cc0 0x7f0c680a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b2e0 tx=0x7f0c7000b8f0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.834+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 msgr2=0x7f0c8419afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.834+0000 7f0c88e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 0x7f0c8419afd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9c0 tx=0x7f0c7400ecd0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 shutdown_connections 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f0c680a3cc0 0x7f0c680a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c84106890 0x7f0c841a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c84104cf0 0x7f0c8419afd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 --2- 172.21.15.44:0/2903962677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c84105ef0 0x7f0c8419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 >> 172.21.15.44:0/2903962677 conn(0x7f0c841002b0 msgr2=0x7f0c84109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 shutdown_connections 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:13.835+0000 7f0c88e00700 1 -- 172.21.15.44:0/2903962677 wait complete. 2024-04-20T11:22:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:22:14.656 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:14 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2903962677' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:14.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:14 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2903962677' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:14.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:14 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2903962677' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T11:22:14.735 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-20T11:22:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:22:15.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:15 smithi078 ceph-mon[21464]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:15.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:15 smithi044 ceph-mon[16855]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:15.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:15 smithi165 ceph-mon[21975]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 -- 172.21.15.44:0/418602001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0105df0 msgr2=0x7fd2f0106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 --2- 172.21.15.44:0/418602001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0105df0 0x7fd2f0106250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009990 tx=0x7fd2dc009ca0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 -- 172.21.15.44:0/418602001 shutdown_connections 2024-04-20T11:22:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 --2- 172.21.15.44:0/418602001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0106790 0x7fd2f010d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 --2- 172.21.15.44:0/418602001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0104bf0 0x7fd2f0105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 --2- 172.21.15.44:0/418602001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0105df0 0x7fd2f0106250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 -- 172.21.15.44:0/418602001 >> 172.21.15.44:0/418602001 conn(0x7fd2f0100190 msgr2=0x7fd2f01025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:17.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 -- 172.21.15.44:0/418602001 shutdown_connections 2024-04-20T11:22:17.500 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:17 smithi044 ceph-mon[16855]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.498+0000 7fd2f5400700 1 -- 172.21.15.44:0/418602001 wait complete. 2024-04-20T11:22:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.499+0000 7fd2f5400700 1 Processor -- start 2024-04-20T11:22:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.499+0000 7fd2f5400700 1 -- start start 2024-04-20T11:22:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2f5400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0104bf0 0x7fd2f019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2f5400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 0x7fd2f019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2f5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0106790 0x7fd2f01a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2f5400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0110960 con 0x7fd2f0104bf0 2024-04-20T11:22:17.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2f5400700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01107e0 con 0x7fd2f0105df0 2024-04-20T11:22:17.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2f5400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0110ae0 con 0x7fd2f0106790 2024-04-20T11:22:17.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2ef400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 0x7fd2f019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:17.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2ef400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 0x7fd2f019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59668/0 (socket says 172.21.15.44:59668) 2024-04-20T11:22:17.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2ef400700 1 -- 172.21.15.44:0/2276889708 learned_addr learned my addr 172.21.15.44:0/2276889708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.500+0000 7fd2efe00700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0104bf0 0x7fd2f019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2f4a00700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0106790 0x7fd2f01a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2ef400700 1 -- 172.21.15.44:0/2276889708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0106790 msgr2=0x7fd2f01a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2ef400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0106790 0x7fd2f01a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2ef400700 1 -- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0104bf0 msgr2=0x7fd2f019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2ef400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0104bf0 0x7fd2f019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2ef400700 1 -- 172.21.15.44:0/2276889708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f01a2b70 con 0x7fd2f0105df0 2024-04-20T11:22:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2efe00700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0104bf0 0x7fd2f019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2f4a00700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0106790 0x7fd2f01a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.501+0000 7fd2ef400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 0x7fd2f019b450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005910 tx=0x7fd2dc00bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.502+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc026070 con 0x7fd2f0105df0 2024-04-20T11:22:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.502+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f01a2e00 con 0x7fd2f0105df0 2024-04-20T11:22:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.502+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2dc003d90 con 0x7fd2f0105df0 2024-04-20T11:22:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.502+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc021670 con 0x7fd2f0105df0 2024-04-20T11:22:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.502+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f01a3250 con 0x7fd2f0105df0 2024-04-20T11:22:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.503+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f0069740 con 0x7fd2f0105df0 2024-04-20T11:22:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.505+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd2dc029070 con 0x7fd2f0105df0 2024-04-20T11:22:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.506+0000 7fd2ecc00700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd2d40a3cc0 0x7fd2d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.507+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fd2dc0d1180 con 0x7fd2f0105df0 2024-04-20T11:22:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.507+0000 7fd2efe00700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd2d40a3cc0 0x7fd2d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.507+0000 7fd2efe00700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd2d40a3cc0 0x7fd2d40a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0105c50 tx=0x7fd2e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.511+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd2dc05dff0 con 0x7fd2f0105df0 2024-04-20T11:22:17.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:17 smithi078 ceph-mon[21464]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:17 smithi165 ceph-mon[21975]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:17.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.746+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd2f0063660 con 0x7fd2f0105df0 2024-04-20T11:22:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.748+0000 7fd2ecc00700 1 -- 172.21.15.44:0/2276889708 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5216 (secure 0 0 0) 0x7fd2dc013070 con 0x7fd2f0105df0 2024-04-20T11:22:17.749 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 87 2024-04-20T11:22:17.749 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:22:17.749 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-04-20T11:10:40.176355+0000 2024-04-20T11:22:17.749 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-04-20T11:20:51.201931+0000 2024-04-20T11:22:17.749 INFO:teuthology.orchestra.run.smithi044.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-20T11:22:17.749 INFO:teuthology.orchestra.run.smithi044.stdout:crush_version 26 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:full_ratio 0.95 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:backfillfull_ratio 0.9 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:nearfull_ratio 0.85 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:require_min_compat_client luminous 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:min_compat_client luminous 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:require_osd_release squid 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:stretch_mode_enabled false 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:max_osd 12 2024-04-20T11:22:17.750 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6802/694121710,v1:172.21.15.44:6803/694121710] [v2:172.21.15.44:6804/694121710,v1:172.21.15.44:6805/694121710] exists,up e2d4d49b-1218-474f-8e22-6d528d61f28d 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6800/2898064203,v1:172.21.15.78:6801/2898064203] [v2:172.21.15.78:6802/2898064203,v1:172.21.15.78:6803/2898064203] exists,up 7e99f802-a969-42c1-8c47-75ac8cc78349 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/1225247316,v1:172.21.15.165:6801/1225247316] [v2:172.21.15.165:6802/1225247316,v1:172.21.15.165:6803/1225247316] exists,up 25612bf3-e5b7-49e1-bf3c-929ddf669fc4 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6810/3780220117,v1:172.21.15.44:6811/3780220117] [v2:172.21.15.44:6812/3780220117,v1:172.21.15.44:6813/3780220117] exists,up e081b5a8-ec52-40c2-bebd-3dd47ac73986 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6808/856268583,v1:172.21.15.78:6809/856268583] [v2:172.21.15.78:6810/856268583,v1:172.21.15.78:6811/856268583] exists,up 97110c0d-1f21-4f21-be1f-9b026c6d37ba 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/2633955642,v1:172.21.15.165:6809/2633955642] [v2:172.21.15.165:6810/2633955642,v1:172.21.15.165:6811/2633955642] exists,up 1e45eea6-98f1-48a0-aac0-8836c0e46fd1 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6818/2930726508,v1:172.21.15.44:6819/2930726508] [v2:172.21.15.44:6820/2930726508,v1:172.21.15.44:6821/2930726508] exists,up e4fd9d25-8aa7-45e3-a5da-50ccddec8555 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6816/1412804643,v1:172.21.15.78:6817/1412804643] [v2:172.21.15.78:6818/1412804643,v1:172.21.15.78:6819/1412804643] exists,up 29b9c0d1-e314-4337-b6cd-0a7c7514f8dc 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/2878265181,v1:172.21.15.165:6817/2878265181] [v2:172.21.15.165:6818/2878265181,v1:172.21.15.165:6819/2878265181] exists,up 3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 up in weight 1 up_from 67 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6826/3770179466,v1:172.21.15.44:6827/3770179466] [v2:172.21.15.44:6828/3770179466,v1:172.21.15.44:6829/3770179466] exists,up 7e0db795-eae2-41b0-b0f6-38e6fc4b56b9 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6824/259959588,v1:172.21.15.78:6825/259959588] [v2:172.21.15.78:6826/259959588,v1:172.21.15.78:6827/259959588] exists,up 35e05c9e-6f8c-4dce-ad79-fa9597cb0743 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/1018684667,v1:172.21.15.165:6825/1018684667] [v2:172.21.15.165:6826/1018684667,v1:172.21.15.165:6827/1018684667] exists,up 97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.8 [2,11] 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.10 [8,11] 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.1 [4,9] 2024-04-20T11:22:17.751 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.10 [5,9] 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.1b [4,8] 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.36 [4,9] 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/1299451251 expires 2024-04-21T11:11:49.892352+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/1299451251 expires 2024-04-21T11:11:49.892352+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/3474255824 expires 2024-04-21T11:11:49.892352+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/3779581802 expires 2024-04-21T11:11:49.892352+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/1801045838 expires 2024-04-21T11:11:14.579278+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/1801045838 expires 2024-04-21T11:11:14.579278+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2620659555 expires 2024-04-21T11:11:14.579278+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/135755932 expires 2024-04-21T11:11:14.579278+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/393908752 expires 2024-04-21T11:11:49.892352+0000 2024-04-20T11:22:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/3844329137 expires 2024-04-21T11:11:14.579278+0000 2024-04-20T11:22:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd2d40a3cc0 msgr2=0x7fd2d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd2d40a3cc0 0x7fd2d40a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0105c50 tx=0x7fd2e0009260 comp rx=0 tx=0).stop 2024-04-20T11:22:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 msgr2=0x7fd2f019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 0x7fd2f019b450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc005910 tx=0x7fd2dc00bc00 comp rx=0 tx=0).stop 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 shutdown_connections 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd2d40a3cc0 0x7fd2d40a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2f0106790 0x7fd2f01a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2f0104bf0 0x7fd2f019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 --2- 172.21.15.44:0/2276889708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2f0105df0 0x7fd2f019b450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 >> 172.21.15.44:0/2276889708 conn(0x7fd2f0100190 msgr2=0x7fd2f01090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 shutdown_connections 2024-04-20T11:22:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:17.753+0000 7fd2f5400700 1 -- 172.21.15.44:0/2276889708 wait complete. 2024-04-20T11:22:18.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:18 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2276889708' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T11:22:18.623 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-20T11:22:18.636 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:18.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:18 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2276889708' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T11:22:18.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:18 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2276889708' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T11:22:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 -- 172.21.15.44:0/2489229414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c1051d0 msgr2=0x7f5c5c105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2489229414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c1051d0 0x7f5c5c105650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009900 tx=0x7f5c4802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 -- 172.21.15.44:0/2489229414 shutdown_connections 2024-04-20T11:22:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2489229414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c5c105b90 0x7f5c5c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2489229414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c1051d0 0x7f5c5c105650 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2489229414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.860+0000 7f5c62e00640 1 -- 172.21.15.44:0/2489229414 >> 172.21.15.44:0/2489229414 conn(0x7f5c5c0ff570 msgr2=0x7f5c5c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.861+0000 7f5c62e00640 1 -- 172.21.15.44:0/2489229414 shutdown_connections 2024-04-20T11:22:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.861+0000 7f5c62e00640 1 -- 172.21.15.44:0/2489229414 wait complete. 2024-04-20T11:22:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 Processor -- start 2024-04-20T11:22:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 -- start start 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c5c1051d0 0x7f5c5c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c105b90 0x7f5c5c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10fd40 con 0x7f5c5c103fe0 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10fbc0 con 0x7f5c5c105b90 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10fec0 con 0x7f5c5c1051d0 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c61a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c61a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45992/0 (socket says 172.21.15.44:45992) 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c61a00640 1 -- 172.21.15.44:0/2300309778 learned_addr learned my addr 172.21.15.44:0/2300309778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c61000640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c5c1051d0 0x7f5c5c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.862+0000 7f5c62400640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c105b90 0x7f5c5c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c61a00640 1 -- 172.21.15.44:0/2300309778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c5c1051d0 msgr2=0x7f5c5c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c61a00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c5c1051d0 0x7f5c5c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c61a00640 1 -- 172.21.15.44:0/2300309778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c105b90 msgr2=0x7f5c5c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c61a00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c105b90 0x7f5c5c1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c61a00640 1 -- 172.21.15.44:0/2300309778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c1a2010 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c61a00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c19a3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000e9c0 tx=0x7f5c5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000cde0 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c1a2300 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c50010930 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.863+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50018a60 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.864+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a2870 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.865+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c10b2a0 con 0x7f5c5c103fe0 2024-04-20T11:22:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.867+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5c5001f020 con 0x7f5c5c103fe0 2024-04-20T11:22:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.868+0000 7f5c5a000640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5c440a3bb0 0x7f5c440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.868+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f5c500d05b0 con 0x7f5c5c103fe0 2024-04-20T11:22:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.869+0000 7f5c61000640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5c440a3bb0 0x7f5c440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.869+0000 7f5c61000640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5c440a3bb0 0x7f5c440a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005ab0 tx=0x7f5c48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:18.873+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5c5009cf90 con 0x7f5c5c103fe0 2024-04-20T11:22:19.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.085+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c5c062b60 con 0x7f5c5c103fe0 2024-04-20T11:22:19.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.089+0000 7f5c5a000640 1 -- 172.21.15.44:0/2300309778 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22511 (secure 0 0 0) 0x7f5c50085e20 con 0x7f5c5c103fe0 2024-04-20T11:22:19.089 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:19.089 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:51.201931+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5c440a3bb0 msgr2=0x7f5c440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5c440a3bb0 0x7f5c440a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005ab0 tx=0x7f5c48005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 msgr2=0x7f5c5c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c19a3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000e9c0 tx=0x7f5c5000ee80 comp rx=0 tx=0).stop 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 shutdown_connections 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5c440a3bb0 0x7f5c440a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c105b90 0x7f5c5c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c5c1051d0 0x7f5c5c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 --2- 172.21.15.44:0/2300309778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c103fe0 0x7f5c5c19a3f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.092+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 >> 172.21.15.44:0/2300309778 conn(0x7f5c5c0ff570 msgr2=0x7f5c5c108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.093+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 shutdown_connections 2024-04-20T11:22:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.093+0000 7f5c62e00640 1 -- 172.21.15.44:0/2300309778 wait complete. 2024-04-20T11:22:19.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:22:19.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:22:19.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.329+0000 7f3e43e00640 1 -- 172.21.15.44:0/1250509041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e44105b70 msgr2=0x7f3e4410c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.329+0000 7f3e43e00640 1 --2- 172.21.15.44:0/1250509041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e44105b70 0x7f3e4410c400 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009be0 tx=0x7f3e3c02bef0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 -- 172.21.15.44:0/1250509041 shutdown_connections 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 --2- 172.21.15.44:0/1250509041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e44105b70 0x7f3e4410c400 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 --2- 172.21.15.44:0/1250509041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e441051b0 0x7f3e44105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 --2- 172.21.15.44:0/1250509041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e441043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 -- 172.21.15.44:0/1250509041 >> 172.21.15.44:0/1250509041 conn(0x7f3e440ff530 msgr2=0x7f3e44101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 -- 172.21.15.44:0/1250509041 shutdown_connections 2024-04-20T11:22:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.330+0000 7f3e43e00640 1 -- 172.21.15.44:0/1250509041 wait complete. 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 Processor -- start 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 -- start start 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e4419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e441051b0 0x7f3e4419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e44105b70 0x7f3e441a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410fdb0 con 0x7f3e441051b0 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410fc30 con 0x7f3e44103fc0 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.331+0000 7f3e43e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410ff30 con 0x7f3e44105b70 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e4419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e441051b0 0x7f3e4419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e4419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58224/0 (socket says 172.21.15.44:58224) 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 -- 172.21.15.44:0/3218062627 learned_addr learned my addr 172.21.15.44:0/3218062627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e43400640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e44105b70 0x7f3e441a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 -- 172.21.15.44:0/3218062627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e44105b70 msgr2=0x7f3e441a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e44105b70 0x7f3e441a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 -- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e441051b0 msgr2=0x7f3e4419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e441051b0 0x7f3e4419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 -- 172.21.15.44:0/3218062627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a1f00 con 0x7f3e44103fc0 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42000640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e441051b0 0x7f3e4419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.332+0000 7f3e42a00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e4419a2e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400e9c0 tx=0x7f3e3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.333+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400cd80 con 0x7f3e44103fc0 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.333+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e340035f0 con 0x7f3e44103fc0 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.333+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a21f0 con 0x7f3e44103fc0 2024-04-20T11:22:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.333+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34010690 con 0x7f3e44103fc0 2024-04-20T11:22:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.333+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a2790 con 0x7f3e44103fc0 2024-04-20T11:22:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.334+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4410b210 con 0x7f3e44103fc0 2024-04-20T11:22:19.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.337+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e340037a0 con 0x7f3e44103fc0 2024-04-20T11:22:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.338+0000 7f3e3b400640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e280a3c30 0x7f3e280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.338+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f3e340d08b0 con 0x7f3e44103fc0 2024-04-20T11:22:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.338+0000 7f3e42000640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e280a3c30 0x7f3e280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.339+0000 7f3e42000640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e280a3c30 0x7f3e280a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3e300096c0 tx=0x7f3e30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:19.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.340+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e3409d320 con 0x7f3e44103fc0 2024-04-20T11:22:19.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.548+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e4402d880 con 0x7f3e44103fc0 2024-04-20T11:22:19.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.549+0000 7f3e3b400640 1 -- 172.21.15.44:0/3218062627 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f3e3409cc20 con 0x7f3e44103fc0 2024-04-20T11:22:19.548 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e280a3c30 msgr2=0x7f3e280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e280a3c30 0x7f3e280a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3e300096c0 tx=0x7f3e30009260 comp rx=0 tx=0).stop 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 msgr2=0x7f3e4419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e4419a2e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400e9c0 tx=0x7f3e3400ee80 comp rx=0 tx=0).stop 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 shutdown_connections 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e280a3c30 0x7f3e280a60f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e44105b70 0x7f3e441a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e441051b0 0x7f3e4419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 --2- 172.21.15.44:0/3218062627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e44103fc0 0x7f3e4419a2e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 >> 172.21.15.44:0/3218062627 conn(0x7f3e440ff530 msgr2=0x7f3e441083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:19.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 shutdown_connections 2024-04-20T11:22:19.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.552+0000 7f3e43e00640 1 -- 172.21.15.44:0/3218062627 wait complete. 2024-04-20T11:22:19.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:22:19.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:19 smithi078 ceph-mon[21464]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:22:19.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:19 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2300309778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:19.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:19 smithi044 ceph-mon[16855]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:22:19.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:19 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2300309778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:19.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:19 smithi165 ceph-mon[21975]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-20T11:22:19.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:19 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2300309778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 -- 172.21.15.44:0/2077671976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c101fb0 msgr2=0x7f8d1c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2077671976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1023d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009860 tx=0x7f8d0c02bb70 comp rx=0 tx=0).stop 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 -- 172.21.15.44:0/2077671976 shutdown_connections 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2077671976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c103b60 0x7f8d1c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2077671976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 0x7f8d1c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2077671976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1023d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 -- 172.21.15.44:0/2077671976 >> 172.21.15.44:0/2077671976 conn(0x7f8d1c0fd540 msgr2=0x7f8d1c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 -- 172.21.15.44:0/2077671976 shutdown_connections 2024-04-20T11:22:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.794+0000 7f8d21e00640 1 -- 172.21.15.44:0/2077671976 wait complete. 2024-04-20T11:22:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.795+0000 7f8d21e00640 1 Processor -- start 2024-04-20T11:22:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 -- start start 2024-04-20T11:22:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 0x7f8d1c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c103b60 0x7f8d1c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10dd60 con 0x7f8d1c1031a0 2024-04-20T11:22:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10dbe0 con 0x7f8d1c103b60 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.796+0000 7f8d21e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10dee0 con 0x7f8d1c101fb0 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 0x7f8d1c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41172/0 (socket says 172.21.15.44:41172) 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 -- 172.21.15.44:0/2759555414 learned_addr learned my addr 172.21.15.44:0/2759555414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1b400640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c103b60 0x7f8d1c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 -- 172.21.15.44:0/2759555414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c103b60 msgr2=0x7f8d1c19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c103b60 0x7f8d1c19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 -- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 msgr2=0x7f8d1c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 0x7f8d1c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 -- 172.21.15.44:0/2759555414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c19ffc0 con 0x7f8d1c101fb0 2024-04-20T11:22:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d13e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 0x7f8d1c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.797+0000 7f8d1aa00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1983a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c034a80 tx=0x7f8d0c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.798+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c041020 con 0x7f8d1c101fb0 2024-04-20T11:22:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.798+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d0c02ca10 con 0x7f8d1c101fb0 2024-04-20T11:22:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.798+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c1a0250 con 0x7f8d1c101fb0 2024-04-20T11:22:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.798+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c039040 con 0x7f8d1c101fb0 2024-04-20T11:22:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.798+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c1a0790 con 0x7f8d1c101fb0 2024-04-20T11:22:19.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.799+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c109350 con 0x7f8d1c101fb0 2024-04-20T11:22:19.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.801+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8d0c044020 con 0x7f8d1c101fb0 2024-04-20T11:22:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.803+0000 7f8d20a00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f8d000a3c30 0x7f8d000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.803+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f8d0c0f1b70 con 0x7f8d1c101fb0 2024-04-20T11:22:19.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.803+0000 7f8d13e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f8d000a3c30 0x7f8d000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:19.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.803+0000 7f8d13e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f8d000a3c30 0x7f8d000a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d040096c0 tx=0x7f8d04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:19.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:19.807+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8d0c04b020 con 0x7f8d1c101fb0 2024-04-20T11:22:20.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.013+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d1c02d7f0 con 0x7f8d1c101fb0 2024-04-20T11:22:20.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.014+0000 7f8d20a00640 1 -- 172.21.15.44:0/2759555414 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f8d0c0be550 con 0x7f8d1c101fb0 2024-04-20T11:22:20.015 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:22:20.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.016+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f8d000a3c30 msgr2=0x7f8d000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.016+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f8d000a3c30 0x7f8d000a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d040096c0 tx=0x7f8d04009260 comp rx=0 tx=0).stop 2024-04-20T11:22:20.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 msgr2=0x7f8d1c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1983a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c034a80 tx=0x7f8d0c005dc0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 shutdown_connections 2024-04-20T11:22:20.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f8d000a3c30 0x7f8d000a60f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d1c103b60 0x7f8d1c19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d1c1031a0 0x7f8d1c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 --2- 172.21.15.44:0/2759555414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1c101fb0 0x7f8d1c1983a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 >> 172.21.15.44:0/2759555414 conn(0x7f8d1c0fd540 msgr2=0x7f8d1c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:20.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 shutdown_connections 2024-04-20T11:22:20.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.017+0000 7f8d21e00640 1 -- 172.21.15.44:0/2759555414 wait complete. 2024-04-20T11:22:20.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 -- 172.21.15.44:0/2735518524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c1051b0 msgr2=0x7f801c105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 --2- 172.21.15.44:0/2735518524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c1051b0 0x7f801c105630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8004009860 tx=0x7f800402bb70 comp rx=0 tx=0).stop 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 -- 172.21.15.44:0/2735518524 shutdown_connections 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 --2- 172.21.15.44:0/2735518524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c105b70 0x7f801c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 --2- 172.21.15.44:0/2735518524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c1051b0 0x7f801c105630 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 --2- 172.21.15.44:0/2735518524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c103fc0 0x7f801c1043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 -- 172.21.15.44:0/2735518524 >> 172.21.15.44:0/2735518524 conn(0x7f801c0ff530 msgr2=0x7f801c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 -- 172.21.15.44:0/2735518524 shutdown_connections 2024-04-20T11:22:20.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.258+0000 7f801be00640 1 -- 172.21.15.44:0/2735518524 wait complete. 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.259+0000 7f801be00640 1 Processor -- start 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.259+0000 7f801be00640 1 -- start start 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 0x7f801c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c1051b0 0x7f801c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c105b70 0x7f801c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f801c10fd70 con 0x7f801c103fc0 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f801c10fbf0 con 0x7f801c1051b0 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f801c10fef0 con 0x7f801c105b70 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 0x7f801c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 0x7f801c19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46062/0 (socket says 172.21.15.44:46062) 2024-04-20T11:22:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801aa00640 1 -- 172.21.15.44:0/2927926243 learned_addr learned my addr 172.21.15.44:0/2927926243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f8013e00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c1051b0 0x7f801c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.260+0000 7f801b400640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c105b70 0x7f801c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801aa00640 1 -- 172.21.15.44:0/2927926243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c105b70 msgr2=0x7f801c1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801aa00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c105b70 0x7f801c1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801aa00640 1 -- 172.21.15.44:0/2927926243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c1051b0 msgr2=0x7f801c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801aa00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c1051b0 0x7f801c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801aa00640 1 -- 172.21.15.44:0/2927926243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c1a1f80 con 0x7f801c103fc0 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f8013e00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c1051b0 0x7f801c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801b400640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c105b70 0x7f801c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:20.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801aa00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 0x7f801c19a360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f800c00e9a0 tx=0x7f800c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c00cd80 con 0x7f801c103fc0 2024-04-20T11:22:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1a2270 con 0x7f801c103fc0 2024-04-20T11:22:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f800c010930 con 0x7f801c103fc0 2024-04-20T11:22:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c018a70 con 0x7f801c103fc0 2024-04-20T11:22:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.261+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1a2790 con 0x7f801c103fc0 2024-04-20T11:22:20.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.263+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c10b2e0 con 0x7f801c103fc0 2024-04-20T11:22:20.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.265+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f800c018c10 con 0x7f801c103fc0 2024-04-20T11:22:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.267+0000 7f8013400640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f80000a3bb0 0x7f80000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.267+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f800c0d07c0 con 0x7f801c103fc0 2024-04-20T11:22:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.267+0000 7f8013e00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f80000a3bb0 0x7f80000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.268+0000 7f8013e00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f80000a3bb0 0x7f80000a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f800402c180 tx=0x7f8004034b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.272+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f800c053ff0 con 0x7f801c103fc0 2024-04-20T11:22:20.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.486+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f801c062b60 con 0x7f801c103fc0 2024-04-20T11:22:20.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.487+0000 7f8013400640 1 -- 172.21.15.44:0/2927926243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f800c09d1a0 con 0x7f801c103fc0 2024-04-20T11:22:20.487 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:22:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f80000a3bb0 msgr2=0x7f80000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f80000a3bb0 0x7f80000a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f800402c180 tx=0x7f8004034b70 comp rx=0 tx=0).stop 2024-04-20T11:22:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 msgr2=0x7f801c19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 0x7f801c19a360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f800c00e9a0 tx=0x7f800c00ee60 comp rx=0 tx=0).stop 2024-04-20T11:22:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 shutdown_connections 2024-04-20T11:22:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f80000a3bb0 0x7f80000a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f801c105b70 0x7f801c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f801c1051b0 0x7f801c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 --2- 172.21.15.44:0/2927926243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f801c103fc0 0x7f801c19a360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 >> 172.21.15.44:0/2927926243 conn(0x7f801c0ff530 msgr2=0x7f801c1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:20.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.490+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 shutdown_connections 2024-04-20T11:22:20.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.491+0000 7f801be00640 1 -- 172.21.15.44:0/2927926243 wait complete. 2024-04-20T11:22:20.511 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:22:20.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:20 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3218062627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:20.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:20 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2759555414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:20.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:20 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3218062627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:20.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:20 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2759555414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:20.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.721+0000 7fd3ece00640 1 -- 172.21.15.44:0/873795241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 msgr2=0x7fd3e8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.721+0000 7fd3ece00640 1 --2- 172.21.15.44:0/873795241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e8109c30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009900 tx=0x7fd3d002bc50 comp rx=0 tx=0).stop 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 -- 172.21.15.44:0/873795241 shutdown_connections 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 --2- 172.21.15.44:0/873795241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 0x7fd3e8110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 --2- 172.21.15.44:0/873795241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e8109c30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 --2- 172.21.15.44:0/873795241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3e81085c0 0x7fd3e81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 -- 172.21.15.44:0/873795241 >> 172.21.15.44:0/873795241 conn(0x7fd3e8077580 msgr2=0x7fd3e80799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 -- 172.21.15.44:0/873795241 shutdown_connections 2024-04-20T11:22:20.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.722+0000 7fd3ece00640 1 -- 172.21.15.44:0/873795241 wait complete. 2024-04-20T11:22:20.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 Processor -- start 2024-04-20T11:22:20.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 -- start start 2024-04-20T11:22:20.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3e81085c0 0x7fd3e819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 0x7fd3e81a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8114370 con 0x7fd3e81097b0 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81141f0 con 0x7fd3e810a170 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.723+0000 7fd3ece00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81144f0 con 0x7fd3e81085c0 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46076/0 (socket says 172.21.15.44:46076) 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 -- 172.21.15.44:0/1686682026 learned_addr learned my addr 172.21.15.44:0/1686682026 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3e5e00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3e81085c0 0x7fd3e819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3e6800640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 0x7fd3e81a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 -- 172.21.15.44:0/1686682026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3e81085c0 msgr2=0x7fd3e819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3e81085c0 0x7fd3e819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 -- 172.21.15.44:0/1686682026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 msgr2=0x7fd3e81a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 0x7fd3e81a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 -- 172.21.15.44:0/1686682026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e81a6600 con 0x7fd3e81097b0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3e6800640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 0x7fd3e81a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.724+0000 7fd3dd400640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e819ef20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005ab0 tx=0x7fd3d0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.725+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0043020 con 0x7fd3e81097b0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.725+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3d0004250 con 0x7fd3e81097b0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.725+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e81a6890 con 0x7fd3e81097b0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.725+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0003bc0 con 0x7fd3e81097b0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.725+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81a6de0 con 0x7fd3e81097b0 2024-04-20T11:22:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.726+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e810f8e0 con 0x7fd3e81097b0 2024-04-20T11:22:20.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.728+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd3d0046020 con 0x7fd3e81097b0 2024-04-20T11:22:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.730+0000 7fd3df400640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd3cc0a3bb0 0x7fd3cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.730+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fd3d00f2970 con 0x7fd3e81097b0 2024-04-20T11:22:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.730+0000 7fd3e5e00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd3cc0a3bb0 0x7fd3cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:20 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3218062627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:20.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:20 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2759555414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.731+0000 7fd3e5e00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd3cc0a3bb0 0x7fd3cc0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8109610 tx=0x7fd3d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.732+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd3d004d020 con 0x7fd3e81097b0 2024-04-20T11:22:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.982+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3e802d5d0 con 0x7fd3e81097b0 2024-04-20T11:22:20.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.984+0000 7fd3df400640 1 -- 172.21.15.44:0/1686682026 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7fd3d00bf350 con 0x7fd3e81097b0 2024-04-20T11:22:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:20.984 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:20:09.840804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:22:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.986+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd3cc0a3bb0 msgr2=0x7fd3cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.986+0000 7fd3ece00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd3cc0a3bb0 0x7fd3cc0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8109610 tx=0x7fd3d8005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.986+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 msgr2=0x7fd3e819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.986+0000 7fd3ece00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e819ef20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005ab0 tx=0x7fd3d0004a30 comp rx=0 tx=0).stop 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 shutdown_connections 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd3cc0a3bb0 0x7fd3cc0a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e810a170 0x7fd3e81a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e81097b0 0x7fd3e819ef20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 --2- 172.21.15.44:0/1686682026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3e81085c0 0x7fd3e819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 >> 172.21.15.44:0/1686682026 conn(0x7fd3e8077580 msgr2=0x7fd3e810cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 shutdown_connections 2024-04-20T11:22:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:20.987+0000 7fd3ece00640 1 -- 172.21.15.44:0/1686682026 wait complete. 2024-04-20T11:22:21.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 -- 172.21.15.44:0/2323626864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 msgr2=0x7eff6c074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 --2- 172.21.15.44:0/2323626864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c074e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff58009930 tx=0x7eff5802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 -- 172.21.15.44:0/2323626864 shutdown_connections 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 --2- 172.21.15.44:0/2323626864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c10e3d0 0x7eff6c1107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 --2- 172.21.15.44:0/2323626864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c074e40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 --2- 172.21.15.44:0/2323626864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff6c075fa0 0x7eff6c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 -- 172.21.15.44:0/2323626864 >> 172.21.15.44:0/2323626864 conn(0x7eff6c0fd370 msgr2=0x7eff6c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 -- 172.21.15.44:0/2323626864 shutdown_connections 2024-04-20T11:22:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.222+0000 7eff72800640 1 -- 172.21.15.44:0/2323626864 wait complete. 2024-04-20T11:22:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.223+0000 7eff72800640 1 Processor -- start 2024-04-20T11:22:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.223+0000 7eff72800640 1 -- start start 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.223+0000 7eff72800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff72800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff6c075fa0 0x7eff6c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff72800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c10e3d0 0x7eff6c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff72800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c114160 con 0x7eff6c0749c0 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff72800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c113fe0 con 0x7eff6c10e3d0 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff72800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1142e0 con 0x7eff6c075fa0 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46094/0 (socket says 172.21.15.44:46094) 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 -- 172.21.15.44:0/2076485030 learned_addr learned my addr 172.21.15.44:0/2076485030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff70a00640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff6c075fa0 0x7eff6c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71e00640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c10e3d0 0x7eff6c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 -- 172.21.15.44:0/2076485030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff6c075fa0 msgr2=0x7eff6c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff6c075fa0 0x7eff6c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 -- 172.21.15.44:0/2076485030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c10e3d0 msgr2=0x7eff6c1a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c10e3d0 0x7eff6c1a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.224+0000 7eff71400640 1 -- 172.21.15.44:0/2076485030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1a6450 con 0x7eff6c0749c0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.225+0000 7eff71400640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c19e830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9f0 tx=0x7eff6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.225+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000cd80 con 0x7eff6c0749c0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.225+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff60010930 con 0x7eff6c0749c0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.225+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1a6740 con 0x7eff6c0749c0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.225+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60018b00 con 0x7eff6c0749c0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.225+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a6cb0 con 0x7eff6c0749c0 2024-04-20T11:22:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.226+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c104490 con 0x7eff6c0749c0 2024-04-20T11:22:21.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.228+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7eff60018ca0 con 0x7eff6c0749c0 2024-04-20T11:22:21.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.230+0000 7eff6a000640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7eff540a3c30 0x7eff540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.230+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7eff600d0720 con 0x7eff6c0749c0 2024-04-20T11:22:21.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.230+0000 7eff70a00640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7eff540a3c30 0x7eff540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.231+0000 7eff70a00640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7eff540a3c30 0x7eff540a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff58000c00 tx=0x7eff58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:21.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.234+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7eff6009cab0 con 0x7eff6c0749c0 2024-04-20T11:22:21.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.447+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff6c062b60 con 0x7eff6c0749c0 2024-04-20T11:22:21.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.450+0000 7eff6a000640 1 -- 172.21.15.44:0/2076485030 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22511 (secure 0 0 0) 0x7eff600a0db0 con 0x7eff6c0749c0 2024-04-20T11:22:21.450 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:21.450 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:51.201931+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.452+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7eff540a3c30 msgr2=0x7eff540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.452+0000 7eff72800640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7eff540a3c30 0x7eff540a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff58000c00 tx=0x7eff58005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.452+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 msgr2=0x7eff6c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.452+0000 7eff72800640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c19e830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9f0 tx=0x7eff6000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 shutdown_connections 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7eff540a3c30 0x7eff540a60f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c10e3d0 0x7eff6c1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff6c075fa0 0x7eff6c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 --2- 172.21.15.44:0/2076485030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6c0749c0 0x7eff6c19e830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 >> 172.21.15.44:0/2076485030 conn(0x7eff6c0fd370 msgr2=0x7eff6c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 shutdown_connections 2024-04-20T11:22:21.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.453+0000 7eff72800640 1 -- 172.21.15.44:0/2076485030 wait complete. 2024-04-20T11:22:21.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:22:21.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:22:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 -- 172.21.15.44:0/1530374958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b410a170 msgr2=0x7f81b4110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 --2- 172.21.15.44:0/1530374958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b410a170 0x7f81b4110a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009be0 tx=0x7f81b002bef0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 -- 172.21.15.44:0/1530374958 shutdown_connections 2024-04-20T11:22:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 --2- 172.21.15.44:0/1530374958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b410a170 0x7f81b4110a00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 --2- 172.21.15.44:0/1530374958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b41097b0 0x7f81b4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 --2- 172.21.15.44:0/1530374958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b41089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 -- 172.21.15.44:0/1530374958 >> 172.21.15.44:0/1530374958 conn(0x7f81b4077560 msgr2=0x7f81b40799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:21.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.685+0000 7f81bc800640 1 -- 172.21.15.44:0/1530374958 shutdown_connections 2024-04-20T11:22:21.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.686+0000 7f81bc800640 1 -- 172.21.15.44:0/1530374958 wait complete. 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 Processor -- start 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 -- start start 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b41097b0 0x7f81b419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b410a170 0x7f81b41a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4114360 con 0x7f81b41085c0 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41141e0 con 0x7f81b41097b0 2024-04-20T11:22:21.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.687+0000 7f81bc800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41144e0 con 0x7f81b410a170 2024-04-20T11:22:21.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:21 smithi078 ceph-mon[21464]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:21.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:21 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2927926243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:21.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:21 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1686682026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46116/0 (socket says 172.21.15.44:46116) 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81b9e00640 1 -- 172.21.15.44:0/1423238832 learned_addr learned my addr 172.21.15.44:0/1423238832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81ba800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b410a170 0x7f81b41a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81b9400640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b41097b0 0x7f81b419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81ba800640 1 -- 172.21.15.44:0/1423238832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b41097b0 msgr2=0x7f81b419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81ba800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b41097b0 0x7f81b419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81ba800640 1 -- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 msgr2=0x7f81b419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81ba800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81ba800640 1 -- 172.21.15.44:0/1423238832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b41a6530 con 0x7f81b410a170 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.688+0000 7f81b9e00640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.689+0000 7f81ba800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b410a170 0x7f81b41a5ec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81b002c430 tx=0x7f81b0033820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.689+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0043020 con 0x7f81b410a170 2024-04-20T11:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.689+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81b002ca80 con 0x7f81b410a170 2024-04-20T11:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.689+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b41a67c0 con 0x7f81b410a170 2024-04-20T11:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.689+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b003e740 con 0x7f81b410a170 2024-04-20T11:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.689+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41a6d10 con 0x7f81b410a170 2024-04-20T11:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.690+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b410f880 con 0x7f81b410a170 2024-04-20T11:22:21.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.692+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81b0046020 con 0x7f81b410a170 2024-04-20T11:22:21.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.694+0000 7f81aea00640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f819c0a3bb0 0x7f819c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:21.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.694+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f81b00f2950 con 0x7f81b410a170 2024-04-20T11:22:21.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.694+0000 7f81b9e00640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f819c0a3bb0 0x7f819c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:21.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:21 smithi044 ceph-mon[16855]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:21.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:21 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2927926243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:21.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:21 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1686682026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:21.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.695+0000 7f81b9e00640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f819c0a3bb0 0x7f819c0a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f81a80096c0 tx=0x7f81a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.698+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f81b004d020 con 0x7f81b410a170 2024-04-20T11:22:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:21 smithi165 ceph-mon[21975]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:21 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2927926243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:21 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1686682026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:21.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.897+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81b4062b60 con 0x7f81b410a170 2024-04-20T11:22:21.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.898+0000 7f81aea00640 1 -- 172.21.15.44:0/1423238832 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f81b00bf330 con 0x7f81b410a170 2024-04-20T11:22:21.897 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:22:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f819c0a3bb0 msgr2=0x7f819c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f819c0a3bb0 0x7f819c0a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f81a80096c0 tx=0x7f81a8009260 comp rx=0 tx=0).stop 2024-04-20T11:22:21.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b410a170 msgr2=0x7f81b41a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b410a170 0x7f81b41a5ec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81b002c430 tx=0x7f81b0033820 comp rx=0 tx=0).stop 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 shutdown_connections 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f819c0a3bb0 0x7f819c0a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81b410a170 0x7f81b41a5ec0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b41097b0 0x7f81b419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 --2- 172.21.15.44:0/1423238832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81b41085c0 0x7f81b419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 >> 172.21.15.44:0/1423238832 conn(0x7f81b4077560 msgr2=0x7f81b410ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 shutdown_connections 2024-04-20T11:22:21.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:21.900+0000 7f81bc800640 1 -- 172.21.15.44:0/1423238832 wait complete. 2024-04-20T11:22:21.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:22:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 -- 172.21.15.44:0/891329049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581053d0 msgr2=0x7f9858105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 --2- 172.21.15.44:0/891329049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581053d0 0x7f9858105850 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9840009900 tx=0x7f984002bc40 comp rx=0 tx=0).stop 2024-04-20T11:22:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 -- 172.21.15.44:0/891329049 shutdown_connections 2024-04-20T11:22:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 --2- 172.21.15.44:0/891329049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9858105d90 0x7f985810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 --2- 172.21.15.44:0/891329049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581053d0 0x7f9858105850 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 --2- 172.21.15.44:0/891329049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581041e0 0x7f9858104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 -- 172.21.15.44:0/891329049 >> 172.21.15.44:0/891329049 conn(0x7f98580ff750 msgr2=0x7f9858101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.129+0000 7f985ca00640 1 -- 172.21.15.44:0/891329049 shutdown_connections 2024-04-20T11:22:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.130+0000 7f985ca00640 1 -- 172.21.15.44:0/891329049 wait complete. 2024-04-20T11:22:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.130+0000 7f985ca00640 1 Processor -- start 2024-04-20T11:22:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 -- start start 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581041e0 0x7f985819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 0x7f985819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9858105d90 0x7f98581a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f985810ff90 con 0x7f98581053d0 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f985810fe10 con 0x7f9858105d90 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f985ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110110 con 0x7f98581041e0 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f984d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 0x7f985819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f9856800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9858105d90 0x7f98581a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f984d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 0x7f985819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46126/0 (socket says 172.21.15.44:46126) 2024-04-20T11:22:22.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f984d400640 1 -- 172.21.15.44:0/2114458775 learned_addr learned my addr 172.21.15.44:0/2114458775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.131+0000 7f9855e00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581041e0 0x7f985819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984d400640 1 -- 172.21.15.44:0/2114458775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581041e0 msgr2=0x7f985819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984d400640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581041e0 0x7f985819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984d400640 1 -- 172.21.15.44:0/2114458775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9858105d90 msgr2=0x7f98581a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984d400640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9858105d90 0x7f98581a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984d400640 1 -- 172.21.15.44:0/2114458775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98581a21c0 con 0x7f98581053d0 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984d400640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 0x7f985819aae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9840000c00 tx=0x7f9840004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:22.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840043020 con 0x7f98581053d0 2024-04-20T11:22:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98581a2450 con 0x7f98581053d0 2024-04-20T11:22:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9840004250 con 0x7f98581053d0 2024-04-20T11:22:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.132+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840003bc0 con 0x7f98581053d0 2024-04-20T11:22:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.133+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98581a29f0 con 0x7f98581053d0 2024-04-20T11:22:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.134+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985810b500 con 0x7f98581053d0 2024-04-20T11:22:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.136+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9840046020 con 0x7f98581053d0 2024-04-20T11:22:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.138+0000 7f984f400640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f983c0a3bb0 0x7f983c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.138+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f98400f2470 con 0x7f98581053d0 2024-04-20T11:22:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.138+0000 7f9855e00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f983c0a3bb0 0x7f983c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.139+0000 7f9855e00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f983c0a3bb0 0x7f983c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f98480095f0 tx=0x7f9848005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:22.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.142+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f984004d020 con 0x7f98581053d0 2024-04-20T11:22:22.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.349+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9858062b60 con 0x7f98581053d0 2024-04-20T11:22:22.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.350+0000 7f984f400640 1 -- 172.21.15.44:0/2114458775 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f98400bee50 con 0x7f98581053d0 2024-04-20T11:22:22.350 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:22:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f983c0a3bb0 msgr2=0x7f983c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f983c0a3bb0 0x7f983c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f98480095f0 tx=0x7f9848005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 msgr2=0x7f985819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 0x7f985819aae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9840000c00 tx=0x7f9840004a90 comp rx=0 tx=0).stop 2024-04-20T11:22:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 shutdown_connections 2024-04-20T11:22:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f983c0a3bb0 0x7f983c0a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9858105d90 0x7f98581a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98581053d0 0x7f985819aae0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 --2- 172.21.15.44:0/2114458775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98581041e0 0x7f985819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 >> 172.21.15.44:0/2114458775 conn(0x7f98580ff750 msgr2=0x7f98581086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 shutdown_connections 2024-04-20T11:22:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.353+0000 7f985ca00640 1 -- 172.21.15.44:0/2114458775 wait complete. 2024-04-20T11:22:22.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 -- 172.21.15.44:0/2902169956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c102800 msgr2=0x7f2d7c104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 --2- 172.21.15.44:0/2902169956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c102800 0x7f2d7c104bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009900 tx=0x7f2d7002bc50 comp rx=0 tx=0).stop 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 -- 172.21.15.44:0/2902169956 shutdown_connections 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 --2- 172.21.15.44:0/2902169956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c107a60 0x7f2d7c109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 --2- 172.21.15.44:0/2902169956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c105130 0x7f2d7c107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 --2- 172.21.15.44:0/2902169956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c102800 0x7f2d7c104bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 -- 172.21.15.44:0/2902169956 >> 172.21.15.44:0/2902169956 conn(0x7f2d7c0fc210 msgr2=0x7f2d7c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 -- 172.21.15.44:0/2902169956 shutdown_connections 2024-04-20T11:22:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.575+0000 7f2d82400640 1 -- 172.21.15.44:0/2902169956 wait complete. 2024-04-20T11:22:22.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.576+0000 7f2d82400640 1 Processor -- start 2024-04-20T11:22:22.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.576+0000 7f2d82400640 1 -- start start 2024-04-20T11:22:22.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.576+0000 7f2d82400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c102800 0x7f2d7c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.576+0000 7f2d82400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c105130 0x7f2d7c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d82400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c107a60 0x7f2d7c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d82400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c10db50 con 0x7f2d7c107a60 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d82400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c10d9d0 con 0x7f2d7c105130 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d82400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c10dcd0 con 0x7f2d7c102800 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c102800 0x7f2d7c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c102800 0x7f2d7c19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41284/0 (socket says 172.21.15.44:41284) 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81000640 1 -- 172.21.15.44:0/717357412 learned_addr learned my addr 172.21.15.44:0/717357412 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c107a60 0x7f2d7c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d7be00640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c105130 0x7f2d7c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 -- 172.21.15.44:0/717357412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c102800 msgr2=0x7f2d7c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c102800 0x7f2d7c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 -- 172.21.15.44:0/717357412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c105130 msgr2=0x7f2d7c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c105130 0x7f2d7c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 -- 172.21.15.44:0/717357412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c1a1fe0 con 0x7f2d7c107a60 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.577+0000 7f2d81a00640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c107a60 0x7f2d7c1a1970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ec50 tx=0x7f2d7400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.578+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74003b70 con 0x7f2d7c107a60 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.578+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2d74003d10 con 0x7f2d7c107a60 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.578+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c1a22d0 con 0x7f2d7c107a60 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.578+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74010920 con 0x7f2d7c107a60 2024-04-20T11:22:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.578+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1a2820 con 0x7f2d7c107a60 2024-04-20T11:22:22.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.579+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c1095b0 con 0x7f2d7c107a60 2024-04-20T11:22:22.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.581+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2d74004020 con 0x7f2d7c107a60 2024-04-20T11:22:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.582+0000 7f2d79600640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2d640a3c30 0x7f2d640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.583+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f2d740d0890 con 0x7f2d7c107a60 2024-04-20T11:22:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.583+0000 7f2d81000640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2d640a3c30 0x7f2d640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.584+0000 7f2d81000640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2d640a3c30 0x7f2d640a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005dc0 tx=0x7f2d70005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:22.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.586+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2d7409d270 con 0x7f2d7c107a60 2024-04-20T11:22:22.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:22 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2076485030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:22.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:22 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1423238832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:22.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:22 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2114458775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:22.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:22 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2076485030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:22.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:22 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1423238832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:22.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:22 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2114458775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:22.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:22 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2076485030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:22.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:22 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1423238832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:22.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:22 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2114458775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.781+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d7c062b60 con 0x7f2d7c107a60 2024-04-20T11:22:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.782+0000 7f2d79600640 1 -- 172.21.15.44:0/717357412 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f2d7409cb70 con 0x7f2d7c107a60 2024-04-20T11:22:22.782 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.784+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2d640a3c30 msgr2=0x7f2d640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.784+0000 7f2d82400640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2d640a3c30 0x7f2d640a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005dc0 tx=0x7f2d70005d50 comp rx=0 tx=0).stop 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.784+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c107a60 msgr2=0x7f2d7c1a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.784+0000 7f2d82400640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c107a60 0x7f2d7c1a1970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ec50 tx=0x7f2d7400c6a0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 shutdown_connections 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f2d640a3c30 0x7f2d640a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c107a60 0x7f2d7c1a1970 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d7c105130 0x7f2d7c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 --2- 172.21.15.44:0/717357412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d7c102800 0x7f2d7c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 >> 172.21.15.44:0/717357412 conn(0x7f2d7c0fc210 msgr2=0x7f2d7c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 shutdown_connections 2024-04-20T11:22:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:22.785+0000 7f2d82400640 1 -- 172.21.15.44:0/717357412 wait complete. 2024-04-20T11:22:22.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:22:23.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.012+0000 7f1391000640 1 -- 172.21.15.44:0/591761512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1097b0 msgr2=0x7f138c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.012+0000 7f1391000640 1 --2- 172.21.15.44:0/591761512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1097b0 0x7f138c109c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1378009900 tx=0x7f137802bc50 comp rx=0 tx=0).stop 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 -- 172.21.15.44:0/591761512 shutdown_connections 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 --2- 172.21.15.44:0/591761512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f138c10a170 0x7f138c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 --2- 172.21.15.44:0/591761512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1097b0 0x7f138c109c30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 --2- 172.21.15.44:0/591761512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 -- 172.21.15.44:0/591761512 >> 172.21.15.44:0/591761512 conn(0x7f138c077580 msgr2=0x7f138c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 -- 172.21.15.44:0/591761512 shutdown_connections 2024-04-20T11:22:23.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.013+0000 7f1391000640 1 -- 172.21.15.44:0/591761512 wait complete. 2024-04-20T11:22:23.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 Processor -- start 2024-04-20T11:22:23.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 -- start start 2024-04-20T11:22:23.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f138c1097b0 0x7f138c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c10a170 0x7f138c1a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f138c114370 con 0x7f138c10a170 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1141f0 con 0x7f138c1085c0 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.014+0000 7f1391000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1144f0 con 0x7f138c1097b0 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58380/0 (socket says 172.21.15.44:58380) 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 -- 172.21.15.44:0/1628833707 learned_addr learned my addr 172.21.15.44:0/1628833707 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f138a800640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c10a170 0x7f138c1a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389400640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f138c1097b0 0x7f138c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 -- 172.21.15.44:0/1628833707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f138c1097b0 msgr2=0x7f138c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f138c1097b0 0x7f138c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 -- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c10a170 msgr2=0x7f138c1a5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c10a170 0x7f138c1a5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 -- 172.21.15.44:0/1628833707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c1a65c0 con 0x7f138c1085c0 2024-04-20T11:22:23.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f138a800640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c10a170 0x7f138c1a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.015+0000 7f1389e00640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c19e9a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e9c0 tx=0x7f137c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.016+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c00cd80 con 0x7f138c1085c0 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.016+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f137c0035f0 con 0x7f138c1085c0 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.016+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1a68b0 con 0x7f138c1085c0 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.016+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c010690 con 0x7f138c1085c0 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.016+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c1a6dd0 con 0x7f138c1085c0 2024-04-20T11:22:23.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.017+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c10f8e0 con 0x7f138c1085c0 2024-04-20T11:22:23.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.019+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f137c0037a0 con 0x7f138c1085c0 2024-04-20T11:22:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.020+0000 7f1382a00640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f13700a3bb0 0x7f13700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.020+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f137c0d0a50 con 0x7f138c1085c0 2024-04-20T11:22:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.021+0000 7f1389400640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f13700a3bb0 0x7f13700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.021+0000 7f1389400640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f13700a3bb0 0x7f13700a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1378006010 tx=0x7f1378000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.023+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f137c09d4c0 con 0x7f138c1085c0 2024-04-20T11:22:23.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.261+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f138c02d840 con 0x7f138c1085c0 2024-04-20T11:22:23.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.262+0000 7f1382a00640 1 -- 172.21.15.44:0/1628833707 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f137c09cdc0 con 0x7f138c1085c0 2024-04-20T11:22:23.263 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:23.263 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:20:09.840804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:22:23.263 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-04-20T11:22:23.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f13700a3bb0 msgr2=0x7f13700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f13700a3bb0 0x7f13700a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1378006010 tx=0x7f1378000bc0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 msgr2=0x7f138c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c19e9a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e9c0 tx=0x7f137c00ee80 comp rx=0 tx=0).stop 2024-04-20T11:22:23.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 shutdown_connections 2024-04-20T11:22:23.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f13700a3bb0 0x7f13700a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c10a170 0x7f138c1a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f138c1097b0 0x7f138c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 --2- 172.21.15.44:0/1628833707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f138c1085c0 0x7f138c19e9a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.265+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 >> 172.21.15.44:0/1628833707 conn(0x7f138c077580 msgr2=0x7f138c10cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.266+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 shutdown_connections 2024-04-20T11:22:23.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.266+0000 7f1391000640 1 -- 172.21.15.44:0/1628833707 wait complete. 2024-04-20T11:22:23.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 -- 172.21.15.44:0/4032841116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28241085c0 msgr2=0x7f28241089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 --2- 172.21.15.44:0/4032841116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28241085c0 0x7f28241089e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2814009900 tx=0x7f281402bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 -- 172.21.15.44:0/4032841116 shutdown_connections 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 --2- 172.21.15.44:0/4032841116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f282410a170 0x7f2824110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 --2- 172.21.15.44:0/4032841116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 0x7f2824109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 --2- 172.21.15.44:0/4032841116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28241085c0 0x7f28241089e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 -- 172.21.15.44:0/4032841116 >> 172.21.15.44:0/4032841116 conn(0x7f2824077580 msgr2=0x7f28240799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 -- 172.21.15.44:0/4032841116 shutdown_connections 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.496+0000 7f282ae00640 1 -- 172.21.15.44:0/4032841116 wait complete. 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 Processor -- start 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 -- start start 2024-04-20T11:22:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 0x7f282419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 0x7f282419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282410a170 0x7f28241a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2824114310 con 0x7f282410a170 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2824114190 con 0x7f28241085c0 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.497+0000 7f282ae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2824114490 con 0x7f28241097b0 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 0x7f282419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 0x7f282419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 0x7f282419e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58396/0 (socket says 172.21.15.44:58396) 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 -- 172.21.15.44:0/3279894980 learned_addr learned my addr 172.21.15.44:0/3279894980 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2828e00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282410a170 0x7f28241a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 -- 172.21.15.44:0/3279894980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 msgr2=0x7f282419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 0x7f282419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 -- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282410a170 msgr2=0x7f28241a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282410a170 0x7f28241a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 -- 172.21.15.44:0/3279894980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28241a65e0 con 0x7f28241085c0 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823400640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 0x7f282419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2828e00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282410a170 0x7f28241a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.498+0000 7f2823e00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 0x7f282419e9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2814000c00 tx=0x7f2814004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.499+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814043020 con 0x7f28241085c0 2024-04-20T11:22:23.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.499+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2814004250 con 0x7f28241085c0 2024-04-20T11:22:23.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.499+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28241a6870 con 0x7f28241085c0 2024-04-20T11:22:23.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.499+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814003b90 con 0x7f28241085c0 2024-04-20T11:22:23.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.499+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28241a6e10 con 0x7f28241085c0 2024-04-20T11:22:23.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.500+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282410f8e0 con 0x7f28241085c0 2024-04-20T11:22:23.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.502+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2814046020 con 0x7f28241085c0 2024-04-20T11:22:23.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.503+0000 7f2820c00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f28080a3c30 0x7f28080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.503+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f28140f2400 con 0x7f28241085c0 2024-04-20T11:22:23.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.503+0000 7f2823400640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f28080a3c30 0x7f28080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.504+0000 7f2823400640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f28080a3c30 0x7f28080a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f282419fed0 tx=0x7f2810005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.507+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f281404d020 con 0x7f28241085c0 2024-04-20T11:22:23.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:23 smithi078 ceph-mon[21464]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:23.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:23 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/717357412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:23.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:23 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1628833707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:23 smithi044 ceph-mon[16855]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:23 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/717357412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:23.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:23 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1628833707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.712+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2824062b60 con 0x7f28241085c0 2024-04-20T11:22:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.714+0000 7f2820c00640 1 -- 172.21.15.44:0/3279894980 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22511 (secure 0 0 0) 0x7f28140bef00 con 0x7f28241085c0 2024-04-20T11:22:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:23.714 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":87,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:20:51.201931+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.716+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f28080a3c30 msgr2=0x7f28080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.716+0000 7f282ae00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f28080a3c30 0x7f28080a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f282419fed0 tx=0x7f2810005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.716+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 msgr2=0x7f282419e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.716+0000 7f282ae00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 0x7f282419e9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2814000c00 tx=0x7f2814004a30 comp rx=0 tx=0).stop 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 shutdown_connections 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f28080a3c30 0x7f28080a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282410a170 0x7f28241a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28241097b0 0x7f282419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 --2- 172.21.15.44:0/3279894980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28241085c0 0x7f282419e9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 >> 172.21.15.44:0/3279894980 conn(0x7f2824077580 msgr2=0x7f282410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 shutdown_connections 2024-04-20T11:22:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.717+0000 7f282ae00640 1 -- 172.21.15.44:0/3279894980 wait complete. 2024-04-20T11:22:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:23 smithi165 ceph-mon[21975]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-20T11:22:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:23 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/717357412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:23 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1628833707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:23.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T11:20:02.149413+0000', 'modified': '2024-04-20T11:20:09.840804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24596, 'mds_1': 24562, 'mds_2': 24550, 'mds_3': 24506, 'mds_4': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6833/715928335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 715928335}, {'type': 'v1', 'addr': '172.21.15.78:6833', 'nonce': 715928335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/858649103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 858649103}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 858649103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'd', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.44:6837/2623237731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 2623237731}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 2623237731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.78:6835/3770087097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 3770087097}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 3770087097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/1721181859', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 1721181859}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 1721181859}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2414137592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2414137592}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2414137592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/2547716992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2547716992}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2547716992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/4179547084', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 4179547084}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 4179547084}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'g', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.44:6839/2240418551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2240418551}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2240418551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6839/315239720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 315239720}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 315239720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24550, 24494, 24596, 24506, 24562]} 2024-04-20T11:22:23.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-20T11:22:23.739 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T11:22:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.949+0000 7efd52e00640 1 -- 172.21.15.44:0/47063420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 msgr2=0x7efd4c1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.949+0000 7efd52e00640 1 --2- 172.21.15.44:0/47063420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c1039d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd40009900 tx=0x7efd4002bc40 comp rx=0 tx=0).stop 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 -- 172.21.15.44:0/47063420 shutdown_connections 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 --2- 172.21.15.44:0/47063420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c106950 0x7efd4c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 --2- 172.21.15.44:0/47063420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c103f10 0x7efd4c106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 --2- 172.21.15.44:0/47063420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c1039d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 -- 172.21.15.44:0/47063420 >> 172.21.15.44:0/47063420 conn(0x7efd4c0fb620 msgr2=0x7efd4c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 -- 172.21.15.44:0/47063420 shutdown_connections 2024-04-20T11:22:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.950+0000 7efd52e00640 1 -- 172.21.15.44:0/47063420 wait complete. 2024-04-20T11:22:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 Processor -- start 2024-04-20T11:22:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 -- start start 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c103f10 0x7efd4c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c106950 0x7efd4c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10cb30 con 0x7efd4c106950 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10c9b0 con 0x7efd4c1015e0 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.951+0000 7efd52e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10ccb0 con 0x7efd4c103f10 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c106950 0x7efd4c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58416/0 (socket says 172.21.15.44:58416) 2024-04-20T11:22:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c103f10 0x7efd4c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 -- 172.21.15.44:0/3982881147 learned_addr learned my addr 172.21.15.44:0/3982881147 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 -- 172.21.15.44:0/3982881147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c103f10 msgr2=0x7efd4c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c103f10 0x7efd4c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 -- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c106950 msgr2=0x7efd4c1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c106950 0x7efd4c1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 -- 172.21.15.44:0/3982881147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c1a20e0 con 0x7efd4c1015e0 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd52400640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c106950 0x7efd4c1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51000640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c103f10 0x7efd4c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.952+0000 7efd51a00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c19a4c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd40000c00 tx=0x7efd400049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.953+0000 7efd4a000640 1 -- 172.21.15.44:0/3982881147 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40043020 con 0x7efd4c1015e0 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.953+0000 7efd4a000640 1 -- 172.21.15.44:0/3982881147 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd40004250 con 0x7efd4c1015e0 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.953+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c1a2370 con 0x7efd4c1015e0 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.953+0000 7efd4a000640 1 -- 172.21.15.44:0/3982881147 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40003a90 con 0x7efd4c1015e0 2024-04-20T11:22:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.953+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c1a28c0 con 0x7efd4c1015e0 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.956+0000 7efd5719d740 1 Processor -- start 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.956+0000 7efd5719d740 1 -- start start 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.956+0000 7efd4a000640 1 -- 172.21.15.44:0/3982881147 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efd40046020 con 0x7efd4c1015e0 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.956+0000 7efd5719d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afbcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.956+0000 7efd5719d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd5719d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afc4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd5719d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba4af239d0 con 0x55ba4afbcb40 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd5719d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ba4aeca270 con 0x55ba4afc26a0 2024-04-20T11:22:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd5719d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba4af1f990 con 0x55ba4afbd530 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afbcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afbcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46194/0 (socket says 172.21.15.44:46194) 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 -- 172.21.15.44:0/2642057748 learned_addr learned my addr 172.21.15.44:0/2642057748 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd51a00640 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd51000640 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afc4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 -- 172.21.15.44:0/2642057748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 msgr2=0x55ba4afc2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 -- 172.21.15.44:0/2642057748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 msgr2=0x55ba4afc4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afc4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.957+0000 7efd52400640 1 -- 172.21.15.44:0/2642057748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4ac362d0 con 0x55ba4afbcb40 2024-04-20T11:22:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.958+0000 7efd52400640 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afbcf60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd440117e0 tx=0x7efd44011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf701abbb418bfe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.958+0000 7efd3ea00640 1 -- 172.21.15.44:0/2642057748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44011310 con 0x55ba4afbcb40 2024-04-20T11:22:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.958+0000 7efd3ea00640 1 -- 172.21.15.44:0/2642057748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd4400f9a0 con 0x55ba4afbcb40 2024-04-20T11:22:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.958+0000 7efd3ea00640 1 -- 172.21.15.44:0/2642057748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44004630 con 0x55ba4afbcb40 2024-04-20T11:22:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.958+0000 7efd4a000640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efd340a3bb0 0x7efd340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd4a000640 1 -- 172.21.15.44:0/3982881147 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7efd400f24a0 con 0x7efd4c1015e0 2024-04-20T11:22:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 -- 172.21.15.44:0/2642057748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 msgr2=0x55ba4afbcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afbcf60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd440117e0 tx=0x7efd44011ca0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd51a00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efd340a3bb0 0x7efd340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 -- 172.21.15.44:0/2642057748 shutdown_connections 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afc4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 --2- 172.21.15.44:0/2642057748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afbcf60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 -- 172.21.15.44:0/2642057748 >> 172.21.15.44:0/2642057748 conn(0x55ba4afb80b0 msgr2=0x55ba4afb84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 -- 172.21.15.44:0/2642057748 shutdown_connections 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 -- 172.21.15.44:0/2642057748 wait complete. 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd51a00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efd340a3bb0 0x7efd340a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55ba4afbd930 tx=0x7efd400f5ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.959+0000 7efd5719d740 1 -- start start 2024-04-20T11:22:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 Processor -- start 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afc7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afc5650 con 0x55ba4afbcb40 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ba4aedd5e0 con 0x55ba4afc26a0 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afc57d0 con 0x55ba4afbd530 2024-04-20T11:22:23.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd5719d740 10 client.0 _ll_register_callbacks cb 0x55ba4ac35ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afc7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afc7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46206/0 (socket says 172.21.15.44:46206) 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd52400640 1 -- 172.21.15.44:0/847853547 learned_addr learned my addr 172.21.15.44:0/847853547 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.960+0000 7efd51000640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 -- 172.21.15.44:0/847853547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 msgr2=0x55ba4afc7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 -- 172.21.15.44:0/847853547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 msgr2=0x55ba4afce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 -- 172.21.15.44:0/847853547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4ad006b0 con 0x55ba4afbcb40 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd52400640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afc7110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd44011610 tx=0x7efd44004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c72c4cc540954f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd33400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd33400640 1 -- 172.21.15.44:0/847853547 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44020070 con 0x55ba4afbcb40 2024-04-20T11:22:23.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd33400640 1 -- 172.21.15.44:0/847853547 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7efd44005250 con 0x55ba4afbcb40 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.961+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4400f9a0 con 0x55ba4afbcb40 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd33400640 1 -- 172.21.15.44:0/847853547 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd44023040 con 0x55ba4afbcb40 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba4afd1a00 con 0x55ba4afbcb40 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd33400640 1 -- 172.21.15.44:0/847853547 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd4400f9a0 con 0x55ba4afbcb40 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd33400640 1 -- 172.21.15.44:0/847853547 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7efd44018020 con 0x55ba4afbcb40 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 10 client.15081 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 20 client.15081 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4afd3e10 0x55ba4afd5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 4 client.15081 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.962+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba4af313f0 con 0x55ba4afd3e10 2024-04-20T11:22:23.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.963+0000 7efd51000640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4afd3e10 0x55ba4afd5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.963+0000 7efd51000640 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4afd3e10 0x55ba4afd5280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9df4df43770a8194 server_cookie=a9eaa76f3184ee91 in_seq=0 out_seq=0 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.963+0000 7efd33400640 10 client.15081 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd33400640 1 -- 172.21.15.44:0/847853547 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba4af313f0 con 0x55ba4afd3e10 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd33400640 10 client.15081 handle_command_reply: tid=0 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 client.15081 shutdown 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 msgr2=0x55ba4afc7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afc7110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd44011610 tx=0x7efd44004060 comp rx=0 tx=0).stop 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 shutdown_connections 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4afd3e10 0x55ba4afd5280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afc26a0 0x55ba4afce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 --2- 172.21.15.44:0/847853547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbcb40 0x55ba4afc7110 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd33400640 0 client.15081 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:22:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 >> 172.21.15.44:0/847853547 conn(0x55ba4afb80b0 msgr2=0x55ba4afb73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 shutdown_connections 2024-04-20T11:22:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.964+0000 7efd5719d740 1 -- 172.21.15.44:0/847853547 wait complete. 2024-04-20T11:22:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.965+0000 7efd5719d740 20 client.15081 trim_cache size 0 max 16384 2024-04-20T11:22:23.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.984+0000 7efd5719d740 1 Processor -- start 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 -- start start 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afb3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 0x55ba4afb8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afc26a0 0x55ba4afc5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afd35c0 con 0x55ba4afbd530 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afd3c90 con 0x55ba4afbcb40 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd5719d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afd4360 con 0x55ba4afc26a0 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd51000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 0x55ba4afb8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd51000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 0x55ba4afb8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46210/0 (socket says 172.21.15.44:46210) 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd51000640 1 -- 172.21.15.44:0/522865465 learned_addr learned my addr 172.21.15.44:0/522865465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd52400640 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afb3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.985+0000 7efd52400640 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afc26a0 0x55ba4afc5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd51000640 1 -- 172.21.15.44:0/522865465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afc26a0 msgr2=0x55ba4afc5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd51000640 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afc26a0 0x55ba4afc5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd51000640 1 -- 172.21.15.44:0/522865465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 msgr2=0x55ba4afb3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd51000640 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afb3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd51000640 1 -- 172.21.15.44:0/522865465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4afc1db0 con 0x55ba4afbd530 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd52400640 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afb3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd51000640 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 0x55ba4afb8490 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55ba4afc8580 tx=0x7efd38005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb0ebbf91ca8962 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd3e000640 1 -- 172.21.15.44:0/522865465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38024030 con 0x55ba4afbd530 2024-04-20T11:22:23.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd3e000640 1 -- 172.21.15.44:0/522865465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd38005520 con 0x55ba4afbd530 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.986+0000 7efd3e000640 1 -- 172.21.15.44:0/522865465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38020410 con 0x55ba4afbd530 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 -- 172.21.15.44:0/522865465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 msgr2=0x55ba4afb8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 0x55ba4afb8490 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55ba4afc8580 tx=0x7efd38005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 -- 172.21.15.44:0/522865465 shutdown_connections 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afc26a0 0x55ba4afc5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afbd530 0x55ba4afb8490 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 --2- 172.21.15.44:0/522865465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afb3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 -- 172.21.15.44:0/522865465 >> 172.21.15.44:0/522865465 conn(0x55ba4afb80b0 msgr2=0x55ba4afd2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 -- 172.21.15.44:0/522865465 shutdown_connections 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 -- 172.21.15.44:0/522865465 wait complete. 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.987+0000 7efd5719d740 1 -- start start 2024-04-20T11:22:23.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 Processor -- start 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afc92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afc26a0 0x55ba4afcc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afd5100 con 0x55ba4afc26a0 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afd4a30 con 0x55ba4afbcb40 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba4afd57d0 con 0x55ba4afbd530 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd5719d740 10 client.0 _ll_register_callbacks cb 0x7efd38009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afc26a0 0x55ba4afcc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd52400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afc26a0 0x55ba4afcc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46212/0 (socket says 172.21.15.44:46212) 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd52400640 1 -- 172.21.15.44:0/2937659414 learned_addr learned my addr 172.21.15.44:0/2937659414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.988+0000 7efd52400640 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afc92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd51000640 1 -- 172.21.15.44:0/2937659414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 msgr2=0x55ba4afc92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd51000640 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afc92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd51000640 1 -- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afc26a0 msgr2=0x55ba4afcc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd51000640 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afc26a0 0x55ba4afcc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd51000640 1 -- 172.21.15.44:0/2937659414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4afccd80 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd51000640 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc97e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55ba4afbd470 tx=0x7efd380120f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd1259594b0a13a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd33400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.989+0000 7efd33400640 1 -- 172.21.15.44:0/2937659414 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38024030 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd38005520 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd33400640 1 -- 172.21.15.44:0/2937659414 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7efd38004880 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd33400640 1 -- 172.21.15.44:0/2937659414 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd38005370 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 10 client.25195 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba4b039dc0 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd33400640 1 -- 172.21.15.44:0/2937659414 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd38005520 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd33400640 1 -- 172.21.15.44:0/2937659414 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7efd380107f0 con 0x55ba4afbd530 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 10 client.25195 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 10 client.25195 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 20 client.25195 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4b03e620 0x55ba4b03d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 4 client.25195 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.990+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "status"}) -- 0x55ba4af2b260 con 0x55ba4b03e620 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.991+0000 7efd52400640 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4b03e620 0x55ba4b03d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.991+0000 7efd52400640 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4b03e620 0x55ba4b03d330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=14969b6985103fc4 server_cookie=840204fcb124f970 in_seq=0 out_seq=0 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.991+0000 7efd33400640 10 client.25195 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd33400640 1 -- 172.21.15.44:0/2937659414 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba4af2b260 con 0x55ba4b03e620 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd33400640 10 client.25195 handle_command_reply: tid=0 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 client.25195 shutdown 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 msgr2=0x55ba4afc97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc97e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55ba4afbd470 tx=0x7efd380120f0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 shutdown_connections 2024-04-20T11:22:23.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ba4afc26a0 0x55ba4afcc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba4afbd530 0x55ba4afc97e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ba4b03e620 0x55ba4b03d330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 --2- 172.21.15.44:0/2937659414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba4afbcb40 0x55ba4afc92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 >> 172.21.15.44:0/2937659414 conn(0x55ba4afb80b0 msgr2=0x55ba4afb3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd33400640 0 client.25195 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 shutdown_connections 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 1 -- 172.21.15.44:0/2937659414 wait complete. 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.992+0000 7efd5719d740 20 client.25195 trim_cache size 0 max 16384 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24494, 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 135.15910263200001, 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 173.22870344699999 2024-04-20T11:22:23.997 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efd340a3bb0 msgr2=0x7efd340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efd340a3bb0 0x7efd340a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55ba4afbd930 tx=0x7efd400f5ee0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 msgr2=0x7efd4c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c19a4c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd40000c00 tx=0x7efd400049e0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 shutdown_connections 2024-04-20T11:22:23.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7efd340a3bb0 0x7efd340a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd4c106950 0x7efd4c1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd4c103f10 0x7efd4c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 --2- 172.21.15.44:0/3982881147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd4c1015e0 0x7efd4c19a4c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 >> 172.21.15.44:0/3982881147 conn(0x7efd4c0fb620 msgr2=0x7efd4c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 shutdown_connections 2024-04-20T11:22:23.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:23.993+0000 7efd52e00640 1 -- 172.21.15.44:0/3982881147 wait complete. 2024-04-20T11:22:24.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 -- 172.21.15.44:0/1904052771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 msgr2=0x7f3c40107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 --2- 172.21.15.44:0/1904052771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c40107620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009900 tx=0x7f3c2c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 -- 172.21.15.44:0/1904052771 shutdown_connections 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 --2- 172.21.15.44:0/1904052771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c40107b60 0x7f3c40109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 --2- 172.21.15.44:0/1904052771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c40107620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 --2- 172.21.15.44:0/1904052771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40102900 0x7f3c40104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 -- 172.21.15.44:0/1904052771 >> 172.21.15.44:0/1904052771 conn(0x7f3c400fc310 msgr2=0x7f3c400fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 -- 172.21.15.44:0/1904052771 shutdown_connections 2024-04-20T11:22:24.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.207+0000 7f3c45800640 1 -- 172.21.15.44:0/1904052771 wait complete. 2024-04-20T11:22:24.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.208+0000 7f3c45800640 1 Processor -- start 2024-04-20T11:22:24.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.208+0000 7f3c45800640 1 -- start start 2024-04-20T11:22:24.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.208+0000 7f3c45800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c40102900 0x7f3c4019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.208+0000 7f3c45800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c4019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c45800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40107b60 0x7f3c401a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c45800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010dbf0 con 0x7f3c40105230 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c45800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010da70 con 0x7f3c40102900 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c45800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010dd70 con 0x7f3c40107b60 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c4019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c4019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46224/0 (socket says 172.21.15.44:46224) 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c3f400640 1 -- 172.21.15.44:0/1995713063 learned_addr learned my addr 172.21.15.44:0/1995713063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c44e00640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40107b60 0x7f3c401a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c40102900 0x7f3c4019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c44e00640 1 -- 172.21.15.44:0/1995713063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c40102900 msgr2=0x7f3c4019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c44e00640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c40102900 0x7f3c4019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c44e00640 1 -- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 msgr2=0x7f3c4019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c44e00640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c4019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c44e00640 1 -- 172.21.15.44:0/1995713063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401a20a0 con 0x7f3c40107b60 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.209+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c4019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.210+0000 7f3c44e00640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40107b60 0x7f3c401a1a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800eb20 tx=0x7f3c3800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.210+0000 7f3c3cc00640 1 -- 172.21.15.44:0/1995713063 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380039c0 con 0x7f3c40107b60 2024-04-20T11:22:24.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.210+0000 7f3c3cc00640 1 -- 172.21.15.44:0/1995713063 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3c38003b60 con 0x7f3c40107b60 2024-04-20T11:22:24.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.210+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401a2390 con 0x7f3c40107b60 2024-04-20T11:22:24.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.210+0000 7f3c3cc00640 1 -- 172.21.15.44:0/1995713063 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380108f0 con 0x7f3c40107b60 2024-04-20T11:22:24.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.210+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401a28b0 con 0x7f3c40107b60 2024-04-20T11:22:24.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.213+0000 7f3c49a39740 1 Processor -- start 2024-04-20T11:22:24.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c3cc00640 1 -- 172.21.15.44:0/1995713063 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3c38003d10 con 0x7f3c40107b60 2024-04-20T11:22:24.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 -- start start 2024-04-20T11:22:24.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d7ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f387ce69d0 con 0x55f387d7fb40 2024-04-20T11:22:24.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f387c8d270 con 0x55f387d80530 2024-04-20T11:22:24.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.214+0000 7f3c49a39740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f387ce2990 con 0x55f387d856a0 2024-04-20T11:22:24.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d7ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d7ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46236/0 (socket says 172.21.15.44:46236) 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3f400640 1 -- 172.21.15.44:0/1407669514 learned_addr learned my addr 172.21.15.44:0/1407669514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 -- 172.21.15.44:0/1407669514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 msgr2=0x55f387d87b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d87b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 -- 172.21.15.44:0/1407669514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 msgr2=0x55f387d7ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d7ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 -- 172.21.15.44:0/1407669514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3879f92d0 con 0x55f387d80530 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d7ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.215+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d85030 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009c00 tx=0x7f3c30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=920b194565263ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c3cc00640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3c240a3bb0 0x7f3c240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c35600640 1 -- 172.21.15.44:0/1407669514 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30004230 con 0x55f387d80530 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c35600640 1 -- 172.21.15.44:0/1407669514 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3c300043d0 con 0x55f387d80530 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c3cc00640 1 -- 172.21.15.44:0/1995713063 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f3c380d0ba0 con 0x7f3c40107b60 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c35600640 1 -- 172.21.15.44:0/1407669514 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30004e40 con 0x55f387d80530 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3c240a3bb0 0x7f3c240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c49a39740 1 -- 172.21.15.44:0/1407669514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 msgr2=0x55f387d85030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d85030 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009c00 tx=0x7f3c30011810 comp rx=0 tx=0).stop 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.216+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3c240a3bb0 0x7f3c240a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c02efd0 tx=0x7f3c2c005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 -- 172.21.15.44:0/1407669514 shutdown_connections 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d87b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d85030 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1407669514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d7ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 -- 172.21.15.44:0/1407669514 >> 172.21.15.44:0/1407669514 conn(0x55f387d7b0b0 msgr2=0x55f387d7b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 -- 172.21.15.44:0/1407669514 shutdown_connections 2024-04-20T11:22:24.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 -- 172.21.15.44:0/1407669514 wait complete. 2024-04-20T11:22:24.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 -- start start 2024-04-20T11:22:24.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.217+0000 7f3c49a39740 1 Processor -- start 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f387d88650 con 0x55f387d7fb40 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f387ca05e0 con 0x55f387d80530 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f387d887d0 con 0x55f387d856a0 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c49a39740 10 client.0 _ll_register_callbacks cb 0x55f3879f8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58482/0 (socket says 172.21.15.44:58482) 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 -- 172.21.15.44:0/1963853431 learned_addr learned my addr 172.21.15.44:0/1963853431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c44e00640 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 -- 172.21.15.44:0/1963853431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 msgr2=0x55f387d91600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d91600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 -- 172.21.15.44:0/1963853431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 msgr2=0x55f387d8a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.218+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c3f400640 1 -- 172.21.15.44:0/1963853431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f387ac36b0 con 0x55f387d80530 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8a5c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005ab0 tx=0x7f3c2c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc60e29c080a8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c2a000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c2a000640 1 -- 172.21.15.44:0/1963853431 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c051020 con 0x55f387d80530 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c300043d0 con 0x55f387d80530 2024-04-20T11:22:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c2a000640 1 -- 172.21.15.44:0/1963853431 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f3c2c0048d0 con 0x55f387d80530 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c2a000640 1 -- 172.21.15.44:0/1963853431 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c004ad0 con 0x55f387d80530 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c49a39740 10 client.25214 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.219+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f387d94a30 con 0x55f387d80530 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c2a000640 1 -- 172.21.15.44:0/1963853431 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3c300043d0 con 0x55f387d80530 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c2a000640 1 -- 172.21.15.44:0/1963853431 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f3c2c040040 con 0x55f387d80530 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c49a39740 10 client.25214 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c49a39740 10 client.25214 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c49a39740 20 client.25214 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387d97030 0x55f387d984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c49a39740 4 client.25214 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.220+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f387cf43f0 con 0x55f387d97030 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.221+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387d97030 0x55f387d984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.221+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387d97030 0x55f387d984a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfe42aaeb9ceb59c server_cookie=19d86f49b983658e in_seq=0 out_seq=0 2024-04-20T11:22:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.221+0000 7f3c2a000640 10 client.25214 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.222+0000 7f3c2a000640 1 -- 172.21.15.44:0/1963853431 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f387cf43f0 con 0x55f387d97030 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.222+0000 7f3c2a000640 10 client.25214 handle_command_reply: tid=0 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 client.25214 shutdown 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 msgr2=0x55f387d8a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8a5c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005ab0 tx=0x7f3c2c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 shutdown_connections 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d91600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8a5c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387d97030 0x55f387d984a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1963853431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 >> 172.21.15.44:0/1963853431 conn(0x55f387d7b0b0 msgr2=0x55f387d7a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c2a000640 0 client.25214 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:22:24.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 shutdown_connections 2024-04-20T11:22:24.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 1 -- 172.21.15.44:0/1963853431 wait complete. 2024-04-20T11:22:24.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.223+0000 7f3c49a39740 20 client.25214 trim_cache size 0 max 16384 2024-04-20T11:22:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.242+0000 7f3c49a39740 1 Processor -- start 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.242+0000 7f3c49a39740 1 -- start start 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d7fb40 0x55f387d8a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d856a0 0x55f387d88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c49a39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f387d967e0 con 0x55f387d856a0 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c49a39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f387d96eb0 con 0x55f387d80530 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c49a39740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f387d97580 con 0x55f387d7fb40 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c44e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d856a0 0x55f387d88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d915c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58490/0 (socket says 172.21.15.44:58490) 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c3f400640 1 -- 172.21.15.44:0/1360612352 learned_addr learned my addr 172.21.15.44:0/1360612352 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.243+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d7fb40 0x55f387d8a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c3f400640 1 -- 172.21.15.44:0/1360612352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d7fb40 msgr2=0x55f387d8a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d7fb40 0x55f387d8a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c3f400640 1 -- 172.21.15.44:0/1360612352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d856a0 msgr2=0x55f387d88b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d856a0 0x55f387d88b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c3f400640 1 -- 172.21.15.44:0/1360612352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f387d6c950 con 0x55f387d80530 2024-04-20T11:22:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c3f400640 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d915c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c02ca30 tx=0x7f3c2c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25bbe6960f77914 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c34c00640 1 -- 172.21.15.44:0/1360612352 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c051070 con 0x55f387d80530 2024-04-20T11:22:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c34c00640 1 -- 172.21.15.44:0/1360612352 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3c2c048050 con 0x55f387d80530 2024-04-20T11:22:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.244+0000 7f3c34c00640 1 -- 172.21.15.44:0/1360612352 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c003a60 con 0x55f387d80530 2024-04-20T11:22:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 -- 172.21.15.44:0/1360612352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 msgr2=0x55f387d915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d915c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c02ca30 tx=0x7f3c2c005df0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 -- 172.21.15.44:0/1360612352 shutdown_connections 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d856a0 0x55f387d88b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d915c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 --2- 172.21.15.44:0/1360612352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d7fb40 0x55f387d8a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 -- 172.21.15.44:0/1360612352 >> 172.21.15.44:0/1360612352 conn(0x55f387d7b0b0 msgr2=0x55f387d89c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 -- 172.21.15.44:0/1360612352 shutdown_connections 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 -- 172.21.15.44:0/1360612352 wait complete. 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.245+0000 7f3c49a39740 1 -- start start 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 Processor -- start 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d90720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f387d98320 con 0x55f387d7fb40 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f387d97c50 con 0x55f387d80530 2024-04-20T11:22:24.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f387d989f0 con 0x55f387d856a0 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c49a39740 10 client.0 _ll_register_callbacks cb 0x7f3c2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46248/0 (socket says 172.21.15.44:46248) 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c3fe00640 1 -- 172.21.15.44:0/2323298210 learned_addr learned my addr 172.21.15.44:0/2323298210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c3f400640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.246+0000 7f3c44e00640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d90720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3fe00640 1 -- 172.21.15.44:0/2323298210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 msgr2=0x55f387d90720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d90720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3fe00640 1 -- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 msgr2=0x55f387d8d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3fe00640 1 -- 172.21.15.44:0/2323298210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f387d90d10 con 0x55f387d7fb40 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3f400640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8d1c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009bd0 tx=0x7f3c30004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0e88cf363aa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c2a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c2a000640 1 -- 172.21.15.44:0/2323298210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30032060 con 0x55f387d7fb40 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c2a000640 1 -- 172.21.15.44:0/2323298210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f3c30026850 con 0x55f387d7fb40 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.247+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c048050 con 0x55f387d7fb40 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c2a000640 1 -- 172.21.15.44:0/2323298210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c3001e040 con 0x55f387d7fb40 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f387e05d00 con 0x55f387d7fb40 2024-04-20T11:22:24.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c2a000640 1 -- 172.21.15.44:0/2323298210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3c2c048050 con 0x55f387d7fb40 2024-04-20T11:22:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c2a000640 1 -- 172.21.15.44:0/2323298210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f3c30037070 con 0x55f387d7fb40 2024-04-20T11:22:24.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 10 client.15105 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 20 client.15105 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387e0a7d0 0x55f387e06c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 4 client.15105 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.248+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "status"}) -- 0x55f387cee260 con 0x55f387e0a7d0 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.249+0000 7f3c3f400640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387e0a7d0 0x55f387e06c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.249+0000 7f3c3f400640 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387e0a7d0 0x55f387e06c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f471eddff5ac2fa0 server_cookie=65054df16a6c8a87 in_seq=0 out_seq=0 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.249+0000 7f3c2a000640 10 client.15105 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.249+0000 7f3c2a000640 1 -- 172.21.15.44:0/2323298210 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f387cee260 con 0x55f387e0a7d0 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.249+0000 7f3c2a000640 10 client.15105 handle_command_reply: tid=0 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.249+0000 7f3c49a39740 1 client.15105 shutdown 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 msgr2=0x55f387d8d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8d1c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009bd0 tx=0x7f3c30004680 comp rx=0 tx=0).stop 2024-04-20T11:22:24.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 shutdown_connections 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f387d856a0 0x55f387d90720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f387d80530 0x55f387d8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55f387e0a7d0 0x55f387e06c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 --2- 172.21.15.44:0/2323298210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f387d7fb40 0x55f387d8d1c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 >> 172.21.15.44:0/2323298210 conn(0x55f387d7b0b0 msgr2=0x55f387d7e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c2a000640 0 client.15105 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 shutdown_connections 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 1 -- 172.21.15.44:0/2323298210 wait complete. 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.250+0000 7f3c49a39740 20 client.15105 trim_cache size 0 max 16384 2024-04-20T11:22:24.254 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24506, 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 136.43996940100001, 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 170.53763396900001 2024-04-20T11:22:24.255 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3c240a3bb0 msgr2=0x7f3c240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3c240a3bb0 0x7f3c240a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c02efd0 tx=0x7f3c2c005e60 comp rx=0 tx=0).stop 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40107b60 msgr2=0x7f3c401a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40107b60 0x7f3c401a1a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800eb20 tx=0x7f3c3800c2d0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 shutdown_connections 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3c240a3bb0 0x7f3c240a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c40107b60 0x7f3c401a1a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c40105230 0x7f3c4019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 --2- 172.21.15.44:0/1995713063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c40102900 0x7f3c4019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 >> 172.21.15.44:0/1995713063 conn(0x7f3c400fc310 msgr2=0x7f3c401020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 shutdown_connections 2024-04-20T11:22:24.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.251+0000 7f3c45800640 1 -- 172.21.15.44:0/1995713063 wait complete. 2024-04-20T11:22:24.267 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-20T11:22:24.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 -- 172.21.15.44:0/949164757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce541006f0 msgr2=0x7fce54102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 --2- 172.21.15.44:0/949164757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce541006f0 0x7fce54102b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce48009900 tx=0x7fce4802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 -- 172.21.15.44:0/949164757 shutdown_connections 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 --2- 172.21.15.44:0/949164757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce54105a10 0x7fce54105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 --2- 172.21.15.44:0/949164757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce54103060 0x7fce541054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 --2- 172.21.15.44:0/949164757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce541006f0 0x7fce54102b20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 -- 172.21.15.44:0/949164757 >> 172.21.15.44:0/949164757 conn(0x7fce540fc310 msgr2=0x7fce540fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 -- 172.21.15.44:0/949164757 shutdown_connections 2024-04-20T11:22:24.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.465+0000 7fce5a200640 1 -- 172.21.15.44:0/949164757 wait complete. 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.466+0000 7fce5a200640 1 Processor -- start 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.466+0000 7fce5a200640 1 -- start start 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce5a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 0x7fce5419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce5a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce54103060 0x7fce5419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce5a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce54105a10 0x7fce541a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce5a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce541141f0 con 0x7fce541006f0 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce5a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce54114070 con 0x7fce54105a10 2024-04-20T11:22:24.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce5a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fce54114370 con 0x7fce54103060 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 0x7fce5419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 0x7fce5419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46270/0 (socket says 172.21.15.44:46270) 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 -- 172.21.15.44:0/1637111511 learned_addr learned my addr 172.21.15.44:0/1637111511 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce59800640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce54105a10 0x7fce541a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce53e00640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce54103060 0x7fce5419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 -- 172.21.15.44:0/1637111511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce54103060 msgr2=0x7fce5419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce54103060 0x7fce5419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 -- 172.21.15.44:0/1637111511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce54105a10 msgr2=0x7fce541a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce54105a10 0x7fce541a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce58e00640 1 -- 172.21.15.44:0/1637111511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce541a6450 con 0x7fce541006f0 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.467+0000 7fce59800640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce54105a10 0x7fce541a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.468+0000 7fce58e00640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 0x7fce5419e880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fce48009230 tx=0x7fce48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.468+0000 7fce51600640 1 -- 172.21.15.44:0/1637111511 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48043020 con 0x7fce541006f0 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.468+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce541a66e0 con 0x7fce541006f0 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.468+0000 7fce51600640 1 -- 172.21.15.44:0/1637111511 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce48004250 con 0x7fce541006f0 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.468+0000 7fce51600640 1 -- 172.21.15.44:0/1637111511 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48003b50 con 0x7fce541006f0 2024-04-20T11:22:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.468+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce541a6c80 con 0x7fce541006f0 2024-04-20T11:22:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.471+0000 7fce5e5b8740 1 Processor -- start 2024-04-20T11:22:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce51600640 1 -- 172.21.15.44:0/1637111511 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fce48046020 con 0x7fce541006f0 2024-04-20T11:22:24.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 -- start start 2024-04-20T11:22:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 0x558e07fb2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fb8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb86a0 0x558e07fbab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558e07f199d0 con 0x558e07fb3530 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558e07ec0270 con 0x558e07fb2b40 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558e07f15990 con 0x558e07fb86a0 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce59800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 0x558e07fb2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce59800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 0x558e07fb2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58546/0 (socket says 172.21.15.44:58546) 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce59800640 1 -- 172.21.15.44:0/2636937470 learned_addr learned my addr 172.21.15.44:0/2636937470 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.472+0000 7fce58e00640 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fb8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce53e00640 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb86a0 0x558e07fbab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce59800640 1 -- 172.21.15.44:0/2636937470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb86a0 msgr2=0x558e07fbab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce59800640 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb86a0 0x558e07fbab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce59800640 1 -- 172.21.15.44:0/2636937470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 msgr2=0x558e07fb8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce59800640 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fb8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce59800640 1 -- 172.21.15.44:0/2636937470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e07c2c2d0 con 0x558e07fb2b40 2024-04-20T11:22:24.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce59800640 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 0x558e07fb2f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0117e0 tx=0x7fce4c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a02a04ddcd1062 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce42000640 1 -- 172.21.15.44:0/2636937470 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c011310 con 0x558e07fb2b40 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.473+0000 7fce51600640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fce3c0a3c30 0x7fce3c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce42000640 1 -- 172.21.15.44:0/2636937470 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce4c00f9a0 con 0x558e07fb2b40 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce42000640 1 -- 172.21.15.44:0/2636937470 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c004690 con 0x558e07fb2b40 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce58e00640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fce3c0a3c30 0x7fce3c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce51600640 1 -- 172.21.15.44:0/1637111511 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fce480f2470 con 0x7fce541006f0 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2636937470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 msgr2=0x558e07fb2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 0x558e07fb2f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0117e0 tx=0x7fce4c011ca0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce58e00640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fce3c0a3c30 0x7fce3c0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fce480c6e90 tx=0x7fce480f5eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.474+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2636937470 shutdown_connections 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb86a0 0x558e07fbab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fb8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2636937470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb2b40 0x558e07fb2f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2636937470 >> 172.21.15.44:0/2636937470 conn(0x558e07fae0b0 msgr2=0x558e07fae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2636937470 shutdown_connections 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2636937470 wait complete. 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 -- start start 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.475+0000 7fce5e5b8740 1 Processor -- start 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 0x558e07fbd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb3530 0x558e07fbd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558e07fbb650 con 0x558e07fb2b40 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558e07ed35e0 con 0x558e07fb86a0 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558e07fbb7d0 con 0x558e07fb3530 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce5e5b8740 10 client.0 _ll_register_callbacks cb 0x558e07c2bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce59800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 0x558e07fbd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce59800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 0x558e07fbd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46288/0 (socket says 172.21.15.44:46288) 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce59800640 1 -- 172.21.15.44:0/2825760400 learned_addr learned my addr 172.21.15.44:0/2825760400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce59800640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.476+0000 7fce53e00640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb3530 0x558e07fbd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce59800640 1 -- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb3530 msgr2=0x558e07fbd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce59800640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb3530 0x558e07fbd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce59800640 1 -- 172.21.15.44:0/2825760400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 msgr2=0x558e07fc4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce59800640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fc4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce59800640 1 -- 172.21.15.44:0/2825760400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e07cf66b0 con 0x558e07fb2b40 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce53e00640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb3530 0x558e07fbd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce59800640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 0x558e07fbd0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0099e0 tx=0x7fce4c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a42cdaee691b6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce36a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce36a00640 1 -- 172.21.15.44:0/2825760400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c022040 con 0x558e07fb2b40 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce36a00640 1 -- 172.21.15.44:0/2825760400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fce4c005250 con 0x558e07fb2b40 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce4c00f9a0 con 0x558e07fb2b40 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce36a00640 1 -- 172.21.15.44:0/2825760400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce4c003de0 con 0x558e07fb2b40 2024-04-20T11:22:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce5e5b8740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.477+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e07fc79b0 con 0x558e07fb2b40 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce36a00640 1 -- 172.21.15.44:0/2825760400 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce4c00f9a0 con 0x558e07fb2b40 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce36a00640 1 -- 172.21.15.44:0/2825760400 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fce4c028040 con 0x558e07fb2b40 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce5e5b8740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce5e5b8740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce5e5b8740 20 client.15117 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e07fc9d90 0x558e07fcb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce5e5b8740 4 client.15117 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.478+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e07f273f0 con 0x558e07fc9d90 2024-04-20T11:22:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.479+0000 7fce53e00640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e07fc9d90 0x558e07fcb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.479+0000 7fce53e00640 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e07fc9d90 0x558e07fcb200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8feeb7ecc321d0b1 server_cookie=a4d65a7256413139 in_seq=0 out_seq=0 2024-04-20T11:22:24.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.479+0000 7fce36a00640 10 client.15117 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce36a00640 1 -- 172.21.15.44:0/2825760400 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e07f273f0 con 0x558e07fc9d90 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce36a00640 10 client.15117 handle_command_reply: tid=0 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 client.15117 shutdown 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 msgr2=0x558e07fbd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 0x558e07fbd0c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0099e0 tx=0x7fce4c005bd0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 shutdown_connections 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e07fc9d90 0x558e07fcb200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb3530 0x558e07fbd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/2825760400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb2b40 0x558e07fbd0c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce36a00640 0 client.15117 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.480+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 >> 172.21.15.44:0/2825760400 conn(0x558e07fae0b0 msgr2=0x558e07fad3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.481+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 shutdown_connections 2024-04-20T11:22:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.481+0000 7fce5e5b8740 1 -- 172.21.15.44:0/2825760400 wait complete. 2024-04-20T11:22:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.481+0000 7fce5e5b8740 20 client.15117 trim_cache size 0 max 16384 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.501+0000 7fce5e5b8740 1 Processor -- start 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.501+0000 7fce5e5b8740 1 -- start start 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fa9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fc4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558e07fc9540 con 0x558e07fb3530 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558e07fc9c10 con 0x558e07fb86a0 2024-04-20T11:22:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558e07fca2e0 con 0x558e07fb2b40 2024-04-20T11:22:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fc4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fc4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46302/0 (socket says 172.21.15.44:46302) 2024-04-20T11:22:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 -- 172.21.15.44:0/1081256756 learned_addr learned my addr 172.21.15.44:0/1081256756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce59800640 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fa9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce59800640 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 -- 172.21.15.44:0/1081256756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 msgr2=0x558e07fa9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fa9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 -- 172.21.15.44:0/1081256756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 msgr2=0x558e07fae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce53e00640 1 -- 172.21.15.44:0/1081256756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e07fa9b20 con 0x558e07fb3530 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.502+0000 7fce59800640 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.503+0000 7fce53e00640 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fc4600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x558e07faf020 tx=0x7fce44005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2a650161f3b2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.503+0000 7fce41600640 1 -- 172.21.15.44:0/1081256756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44004020 con 0x558e07fb3530 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.503+0000 7fce41600640 1 -- 172.21.15.44:0/1081256756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce4400fcf0 con 0x558e07fb3530 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.503+0000 7fce41600640 1 -- 172.21.15.44:0/1081256756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440227d0 con 0x558e07fb3530 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1081256756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 msgr2=0x558e07fc4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fc4600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x558e07faf020 tx=0x7fce44005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1081256756 shutdown_connections 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e07fae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e07fc4600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1081256756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fa9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1081256756 >> 172.21.15.44:0/1081256756 conn(0x558e07fae0b0 msgr2=0x558e07fbbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1081256756 shutdown_connections 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1081256756 wait complete. 2024-04-20T11:22:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 -- start start 2024-04-20T11:22:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.504+0000 7fce5e5b8740 1 Processor -- start 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e0802bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e0802e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558e07fcb080 con 0x558e07fb3530 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558e07fca9b0 con 0x558e07fb86a0 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558e07fcb750 con 0x558e07fb2b40 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce5e5b8740 10 client.0 _ll_register_callbacks cb 0x7fce480f5d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e0802bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce59800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e0802e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e0802bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46314/0 (socket says 172.21.15.44:46314) 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce59800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e0802e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58574/0 (socket says 172.21.15.44:58574) 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce53e00640 1 -- 172.21.15.44:0/1583406500 learned_addr learned my addr 172.21.15.44:0/1583406500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce59800640 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce53e00640 1 -- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 msgr2=0x558e07fc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.505+0000 7fce53e00640 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce53e00640 1 -- 172.21.15.44:0/1583406500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 msgr2=0x558e0802e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce53e00640 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e0802e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce53e00640 1 -- 172.21.15.44:0/1583406500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e0802eb90 con 0x558e07fb3530 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce53e00640 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e0802bd20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x558e07fa9ec0 tx=0x7fce44005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a888fe3751b25af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce36a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce36a00640 1 -- 172.21.15.44:0/1583406500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4402e070 con 0x558e07fb3530 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce36a00640 1 -- 172.21.15.44:0/1583406500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fce4400f7f0 con 0x558e07fb3530 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.506+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce4400fcf0 con 0x558e07fb3530 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce36a00640 1 -- 172.21.15.44:0/1583406500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce4403a050 con 0x558e07fb3530 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e0802fea0 con 0x558e07fb3530 2024-04-20T11:22:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce36a00640 1 -- 172.21.15.44:0/1583406500 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce4400fcf0 con 0x558e07fb3530 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce36a00640 1 -- 172.21.15.44:0/1583406500 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fce44029040 con 0x558e07fb3530 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 20 client.15123 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e08034750 0x558e08033460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 4 client.15123 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "status"}) -- 0x558e07f21260 con 0x558e08034750 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.507+0000 7fce59800640 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e08034750 0x558e08033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.508+0000 7fce59800640 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e08034750 0x558e08033460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=857ee483859facfd server_cookie=656142e9135c4296 in_seq=0 out_seq=0 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.508+0000 7fce36a00640 10 client.15123 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.508+0000 7fce36a00640 1 -- 172.21.15.44:0/1583406500 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558e07f21260 con 0x558e08034750 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.508+0000 7fce36a00640 10 client.15123 handle_command_reply: tid=0 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.508+0000 7fce5e5b8740 1 client.15123 shutdown 2024-04-20T11:22:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 msgr2=0x558e0802bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e0802bd20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x558e07fa9ec0 tx=0x7fce44005760 comp rx=0 tx=0).stop 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 shutdown_connections 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558e08034750 0x558e08033460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e07fb86a0 0x558e0802e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558e07fb3530 0x558e0802bd20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce36a00640 0 client.15123 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 --2- 172.21.15.44:0/1583406500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e07fb2b40 0x558e07fc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 >> 172.21.15.44:0/1583406500 conn(0x558e07fae0b0 msgr2=0x558e07fa9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 shutdown_connections 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 1 -- 172.21.15.44:0/1583406500 wait complete. 2024-04-20T11:22:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.509+0000 7fce5e5b8740 20 client.15123 trim_cache size 0 max 16384 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24550, 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 137.441811769, 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 158.88770245200001 2024-04-20T11:22:24.513 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fce3c0a3c30 msgr2=0x7fce3c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fce3c0a3c30 0x7fce3c0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fce480c6e90 tx=0x7fce480f5eb0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 msgr2=0x7fce5419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 0x7fce5419e880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fce48009230 tx=0x7fce48004ab0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 shutdown_connections 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fce3c0a3c30 0x7fce3c0a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce54105a10 0x7fce541a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce54103060 0x7fce5419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 --2- 172.21.15.44:0/1637111511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce541006f0 0x7fce5419e880 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 >> 172.21.15.44:0/1637111511 conn(0x7fce540fc310 msgr2=0x7fce54103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 shutdown_connections 2024-04-20T11:22:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.510+0000 7fce5a200640 1 -- 172.21.15.44:0/1637111511 wait complete. 2024-04-20T11:22:24.524 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:24 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3279894980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:24.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-20T11:22:24.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:24 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3279894980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:24.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1863172283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc10a050 msgr2=0x7fa5fc1108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1863172283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1108e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009900 tx=0x7fa5f402bc40 comp rx=0 tx=0).stop 2024-04-20T11:22:24.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1863172283 shutdown_connections 2024-04-20T11:22:24.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1863172283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1108e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1863172283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc109690 0x7fa5fc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1863172283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1863172283 >> 172.21.15.44:0/1863172283 conn(0x7fa5fc077430 msgr2=0x7fa5fc079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1863172283 shutdown_connections 2024-04-20T11:22:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.725+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1863172283 wait complete. 2024-04-20T11:22:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.726+0000 7fa5fbe00640 1 Processor -- start 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 -- start start 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc109690 0x7fa5fc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc114250 con 0x7fa5fc1084a0 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1140d0 con 0x7fa5fc109690 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1143d0 con 0x7fa5fc10a050 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5faa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5fb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.727+0000 7fa5f2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc109690 0x7fa5fc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5faa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46332/0 (socket says 172.21.15.44:46332) 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5faa00640 1 -- 172.21.15.44:0/1861848564 learned_addr learned my addr 172.21.15.44:0/1861848564 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41490/0 (socket says 172.21.15.44:41490) 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 -- 172.21.15.44:0/1861848564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc109690 msgr2=0x7fa5fc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc109690 0x7fa5fc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 -- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 msgr2=0x7fa5fc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 -- 172.21.15.44:0/1861848564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc1a6490 con 0x7fa5fc10a050 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5faa00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5f2000640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc109690 0x7fa5fc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.728+0000 7fa5fb400640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1a5e20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5f402c180 tx=0x7fa5f402c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.729+0000 7fa5f3e00640 1 -- 172.21.15.44:0/1861848564 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4043020 con 0x7fa5fc10a050 2024-04-20T11:22:24.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.729+0000 7fa5f3e00640 1 -- 172.21.15.44:0/1861848564 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5f4004250 con 0x7fa5fc10a050 2024-04-20T11:22:24.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.729+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc1a6720 con 0x7fa5fc10a050 2024-04-20T11:22:24.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.729+0000 7fa5f3e00640 1 -- 172.21.15.44:0/1861848564 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4003ac0 con 0x7fa5fc10a050 2024-04-20T11:22:24.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.729+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc1a6c40 con 0x7fa5fc10a050 2024-04-20T11:22:24.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:24 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3279894980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:24.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.732+0000 7fa6047b3740 1 Processor -- start 2024-04-20T11:22:24.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.732+0000 7fa5f3e00640 1 -- 172.21.15.44:0/1861848564 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa5f4046020 con 0x7fa5fc10a050 2024-04-20T11:22:24.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.732+0000 7fa6047b3740 1 -- start start 2024-04-20T11:22:24.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.732+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0287f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c028d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c028fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa6047b3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c01ee9d0 con 0x55f2c0287b40 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa6047b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c0195270 con 0x55f2c0288530 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa6047b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c01ea990 con 0x55f2c028d6a0 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5f2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0287f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5f2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0287f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46346/0 (socket says 172.21.15.44:46346) 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5f2000640 1 -- 172.21.15.44:0/3938657065 learned_addr learned my addr 172.21.15.44:0/3938657065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c028d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5f2000640 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c028fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5faa00640 1 -- 172.21.15.44:0/3938657065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 msgr2=0x55f2c028fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c028fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5faa00640 1 -- 172.21.15.44:0/3938657065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 msgr2=0x55f2c0287f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0287f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.733+0000 7fa5faa00640 1 -- 172.21.15.44:0/3938657065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2bff012d0 con 0x55f2c0288530 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c028d030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec010550 tx=0x7fa5ec010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa9a26435253057 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5f3e00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa5e00a3bb0 0x7fa5e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5ebe00640 1 -- 172.21.15.44:0/3938657065 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec011760 con 0x55f2c0288530 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5ebe00640 1 -- 172.21.15.44:0/3938657065 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5ec011e00 con 0x55f2c0288530 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5ebe00640 1 -- 172.21.15.44:0/3938657065 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0193d0 con 0x55f2c0288530 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5f3e00640 1 -- 172.21.15.44:0/1861848564 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fa5f40f2470 con 0x7fa5fc10a050 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.734+0000 7fa5f2000640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa5e00a3bb0 0x7fa5e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 -- 172.21.15.44:0/3938657065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 msgr2=0x55f2c028d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c028d030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec010550 tx=0x7fa5ec010a10 comp rx=0 tx=0).stop 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa5f2000640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa5e00a3bb0 0x7fa5e00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f2c0288d90 tx=0x7fa5e4015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 -- 172.21.15.44:0/3938657065 shutdown_connections 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c028fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c028d030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3938657065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0287f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 -- 172.21.15.44:0/3938657065 >> 172.21.15.44:0/3938657065 conn(0x55f2c02830b0 msgr2=0x55f2c02834b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 -- 172.21.15.44:0/3938657065 shutdown_connections 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 -- 172.21.15.44:0/3938657065 wait complete. 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.735+0000 7fa6047b3740 1 -- start start 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 Processor -- start 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0291fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0292510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c02994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c0290650 con 0x55f2c0287b40 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c01a85e0 con 0x55f2c0288530 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c02907d0 con 0x55f2c028d6a0 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa6047b3740 10 client.0 _ll_register_callbacks cb 0x55f2bff00ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa5faa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0291fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa5faa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0291fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46348/0 (socket says 172.21.15.44:46348) 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa5faa00640 1 -- 172.21.15.44:0/223033967 learned_addr learned my addr 172.21.15.44:0/223033967 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.736+0000 7fa5f2000640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0292510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5fb400640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c02994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5faa00640 1 -- 172.21.15.44:0/223033967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 msgr2=0x55f2c02994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5faa00640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c02994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5faa00640 1 -- 172.21.15.44:0/223033967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 msgr2=0x55f2c0292510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5faa00640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0292510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5faa00640 1 -- 172.21.15.44:0/223033967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2bffcb6b0 con 0x55f2c0287b40 2024-04-20T11:22:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5faa00640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0291fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec011fd0 tx=0x7fa5ec00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573e3db4919293c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.737+0000 7fa5e8c00640 1 -- 172.21.15.44:0/223033967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec027030 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa5e8c00640 1 -- 172.21.15.44:0/223033967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fa5ec01f070 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5ec011e00 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa5e8c00640 1 -- 172.21.15.44:0/223033967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5ec00e430 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa6047b3740 10 client.15135 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2c029c8c0 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa5e8c00640 1 -- 172.21.15.44:0/223033967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5ec011e00 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.738+0000 7fa5e8c00640 1 -- 172.21.15.44:0/223033967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fa5ec011740 con 0x55f2c0287b40 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa6047b3740 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa6047b3740 10 client.15135 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa6047b3740 20 client.15135 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa6047b3740 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c029ec00 0x55f2c02a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa6047b3740 4 client.15135 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:24.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2c01fc3f0 con 0x55f2c029ec00 2024-04-20T11:22:24.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.739+0000 7fa5f2000640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c029ec00 0x55f2c02a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.740+0000 7fa5f2000640 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c029ec00 0x55f2c02a0070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f719d843a72b7b5 server_cookie=58d75622df5430d8 in_seq=0 out_seq=0 2024-04-20T11:22:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.740+0000 7fa5e8c00640 10 client.15135 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:22:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.740+0000 7fa5e8c00640 1 -- 172.21.15.44:0/223033967 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2c01fc3f0 con 0x55f2c029ec00 2024-04-20T11:22:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.740+0000 7fa5e8c00640 10 client.15135 handle_command_reply: tid=0 2024-04-20T11:22:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 client.15135 shutdown 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 msgr2=0x55f2c0291fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0291fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec011fd0 tx=0x7fa5ec00eed0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 shutdown_connections 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c029ec00 0x55f2c02a0070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c028d6a0 0x55f2c02994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0292510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 --2- 172.21.15.44:0/223033967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c0287b40 0x55f2c0291fd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa5e8c00640 0 client.15135 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 >> 172.21.15.44:0/223033967 conn(0x55f2c02830b0 msgr2=0x55f2c02823f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 shutdown_connections 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 1 -- 172.21.15.44:0/223033967 wait complete. 2024-04-20T11:22:24.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.741+0000 7fa6047b3740 20 client.15135 trim_cache size 0 max 16384 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.761+0000 7fa6047b3740 1 Processor -- start 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 -- start start 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0287b40 0x55f2c0286ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0283490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c02912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c029e3b0 con 0x55f2c028d6a0 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c029ea80 con 0x55f2c0288530 2024-04-20T11:22:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa6047b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c029f150 con 0x55f2c0287b40 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa5f2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0283490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa5f2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0283490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58602/0 (socket says 172.21.15.44:58602) 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa5f2000640 1 -- 172.21.15.44:0/3736333979 learned_addr learned my addr 172.21.15.44:0/3736333979 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0287b40 0x55f2c0286ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.762+0000 7fa5fb400640 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c02912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5f2000640 1 -- 172.21.15.44:0/3736333979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0287b40 msgr2=0x55f2c0286ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5f2000640 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0287b40 0x55f2c0286ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5f2000640 1 -- 172.21.15.44:0/3736333979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 msgr2=0x55f2c02912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5f2000640 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c02912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5f2000640 1 -- 172.21.15.44:0/3736333979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c0288910 con 0x55f2c0288530 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5f2000640 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0283490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4013bb0 tx=0x7fa5e4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c959082cc2f2dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5eb400640 1 -- 172.21.15.44:0/3736333979 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4004b20 con 0x55f2c0288530 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5eb400640 1 -- 172.21.15.44:0/3736333979 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5e4004cc0 con 0x55f2c0288530 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.763+0000 7fa5eb400640 1 -- 172.21.15.44:0/3736333979 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005440 con 0x55f2c0288530 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 -- 172.21.15.44:0/3736333979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 msgr2=0x55f2c0283490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0283490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4013bb0 tx=0x7fa5e4004ab0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 -- 172.21.15.44:0/3736333979 shutdown_connections 2024-04-20T11:22:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c02912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0288530 0x55f2c0283490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3736333979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0287b40 0x55f2c0286ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 -- 172.21.15.44:0/3736333979 >> 172.21.15.44:0/3736333979 conn(0x55f2c02830b0 msgr2=0x55f2c0288fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 -- 172.21.15.44:0/3736333979 shutdown_connections 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 -- 172.21.15.44:0/3736333979 wait complete. 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.764+0000 7fa6047b3740 1 -- start start 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 Processor -- start 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0287b40 0x55f2c0295630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0288530 0x55f2c0295b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c0298730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f2c029fef0 con 0x55f2c028d6a0 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2c029f820 con 0x55f2c0287b40 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c02a05c0 con 0x55f2c0288530 2024-04-20T11:22:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa6047b3740 10 client.0 _ll_register_callbacks cb 0x7fa5ec00ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa5fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c0298730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa5fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c0298730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46362/0 (socket says 172.21.15.44:46362) 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa5faa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0287b40 0x55f2c0295630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.765+0000 7fa5fb400640 1 -- 172.21.15.44:0/3691281669 learned_addr learned my addr 172.21.15.44:0/3691281669 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5f2000640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0288530 0x55f2c0295b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5fb400640 1 -- 172.21.15.44:0/3691281669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0288530 msgr2=0x55f2c0295b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5fb400640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0288530 0x55f2c0295b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5fb400640 1 -- 172.21.15.44:0/3691281669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0287b40 msgr2=0x55f2c0295630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5fb400640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0287b40 0x55f2c0295630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5fb400640 1 -- 172.21.15.44:0/3691281669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c0298d20 con 0x55f2c028d6a0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0287b40 0x55f2c0295630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5fb400640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c0298730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40be6e0 tx=0x7fa5f40be350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6a788a087832e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5e8c00640 1 -- 172.21.15.44:0/3691281669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40c2b00 con 0x55f2c028d6a0 2024-04-20T11:22:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa5e8c00640 1 -- 172.21.15.44:0/3691281669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fa5f40a90b0 con 0x55f2c028d6a0 2024-04-20T11:22:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.766+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e4004cc0 con 0x55f2c028d6a0 2024-04-20T11:22:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa5e8c00640 1 -- 172.21.15.44:0/3691281669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5f40c73c0 con 0x55f2c028d6a0 2024-04-20T11:22:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2c0305450 con 0x55f2c028d6a0 2024-04-20T11:22:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa5e8c00640 1 -- 172.21.15.44:0/3691281669 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5e4004cc0 con 0x55f2c028d6a0 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa5e8c00640 1 -- 172.21.15.44:0/3691281669 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fa5f40a7ce0 con 0x55f2c028d6a0 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 10 client.15141 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 20 client.15141 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c030a0f0 0x55f2c0308e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 4 client.15141 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:22:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.767+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "status"}) -- 0x55f2c01f6260 con 0x55f2c030a0f0 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.768+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c030a0f0 0x55f2c0308e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.768+0000 7fa5faa00640 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c030a0f0 0x55f2c0308e00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df3d3aa95cbb26f9 server_cookie=33fcddccec98bce2 in_seq=0 out_seq=0 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.768+0000 7fa5e8c00640 10 client.15141 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa5e8c00640 1 -- 172.21.15.44:0/3691281669 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f2c01f6260 con 0x55f2c030a0f0 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa5e8c00640 10 client.15141 handle_command_reply: tid=0 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 client.15141 shutdown 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 msgr2=0x55f2c0298730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c0298730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40be6e0 tx=0x7fa5f40be350 comp rx=0 tx=0).stop 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 shutdown_connections 2024-04-20T11:22:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f2c030a0f0 0x55f2c0308e00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c028d6a0 0x55f2c0298730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c0288530 0x55f2c0295b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 --2- 172.21.15.44:0/3691281669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2c0287b40 0x55f2c0295630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa5e8c00640 0 client.15141 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 >> 172.21.15.44:0/3691281669 conn(0x55f2c02830b0 msgr2=0x55f2c0286ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 shutdown_connections 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 1 -- 172.21.15.44:0/3691281669 wait complete. 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.769+0000 7fa6047b3740 20 client.15141 trim_cache size 0 max 16384 2024-04-20T11:22:24.773 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24562, 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 138.71506830199999, 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 154.70282152799999 2024-04-20T11:22:24.774 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa5e00a3bb0 msgr2=0x7fa5e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa5e00a3bb0 0x7fa5e00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f2c0288d90 tx=0x7fa5e4015040 comp rx=0 tx=0).stop 2024-04-20T11:22:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 msgr2=0x7fa5fc1a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1a5e20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5f402c180 tx=0x7fa5f402c6b0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 shutdown_connections 2024-04-20T11:22:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fa5e00a3bb0 0x7fa5e00a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5fc10a050 0x7fa5fc1a5e20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc109690 0x7fa5fc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 --2- 172.21.15.44:0/1861848564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5fc1084a0 0x7fa5fc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 >> 172.21.15.44:0/1861848564 conn(0x7fa5fc077430 msgr2=0x7fa5fc10c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 shutdown_connections 2024-04-20T11:22:24.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:24.770+0000 7fa5fbe00640 1 -- 172.21.15.44:0/1861848564 wait complete. 2024-04-20T11:22:24.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T11:22:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 -- 172.21.15.44:0/249380161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 msgr2=0x7fec44109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 --2- 172.21.15.44:0/249380161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec44109b10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009900 tx=0x7fec2c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 -- 172.21.15.44:0/249380161 shutdown_connections 2024-04-20T11:22:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 --2- 172.21.15.44:0/249380161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec4410a050 0x7fec441108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 --2- 172.21.15.44:0/249380161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec44109b10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 --2- 172.21.15.44:0/249380161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec441084a0 0x7fec441088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.001+0000 7fec48a00640 1 -- 172.21.15.44:0/249380161 >> 172.21.15.44:0/249380161 conn(0x7fec44077430 msgr2=0x7fec44079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.002+0000 7fec48a00640 1 -- 172.21.15.44:0/249380161 shutdown_connections 2024-04-20T11:22:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.002+0000 7fec48a00640 1 -- 172.21.15.44:0/249380161 wait complete. 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 Processor -- start 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 -- start start 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 0x7fec4419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec4419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec4410a050 0x7fec441a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fec44114200 con 0x7fec441084a0 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec44114080 con 0x7fec44109690 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.003+0000 7fec48a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fec44114380 con 0x7fec4410a050 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 0x7fec4419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 0x7fec4419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46370/0 (socket says 172.21.15.44:46370) 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 -- 172.21.15.44:0/168760156 learned_addr learned my addr 172.21.15.44:0/168760156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec39e00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec4419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec43200640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec4410a050 0x7fec441a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 -- 172.21.15.44:0/168760156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec4410a050 msgr2=0x7fec441a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec4410a050 0x7fec441a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 -- 172.21.15.44:0/168760156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 msgr2=0x7fec4419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec4419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 -- 172.21.15.44:0/168760156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec441a6460 con 0x7fec441084a0 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec39e00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec4419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.004+0000 7fec42800640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 0x7fec4419e840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fec3400e9c0 tx=0x7fec3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.005+0000 7fec3be00640 1 -- 172.21.15.44:0/168760156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3400cd80 con 0x7fec441084a0 2024-04-20T11:22:25.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.005+0000 7fec3be00640 1 -- 172.21.15.44:0/168760156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec340035f0 con 0x7fec441084a0 2024-04-20T11:22:25.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.005+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec441a6750 con 0x7fec441084a0 2024-04-20T11:22:25.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.005+0000 7fec3be00640 1 -- 172.21.15.44:0/168760156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34010670 con 0x7fec441084a0 2024-04-20T11:22:25.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.005+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441a6c20 con 0x7fec441084a0 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.008+0000 7fec3be00640 1 -- 172.21.15.44:0/168760156 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fec340037a0 con 0x7fec441084a0 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.008+0000 7fec4c03c740 1 Processor -- start 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 -- start start 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 0x5563707d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707df030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707df6a0 0x5563707e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5563707409d0 con 0x5563707da530 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563706e7270 con 0x5563707d9b40 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec4c03c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55637073c990 con 0x5563707df6a0 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec42800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec43200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 0x5563707d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec43200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 0x5563707d9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58638/0 (socket says 172.21.15.44:58638) 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec43200640 1 -- 172.21.15.44:0/502506386 learned_addr learned my addr 172.21.15.44:0/502506386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.009+0000 7fec39e00640 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707df6a0 0x5563707e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec43200640 1 -- 172.21.15.44:0/502506386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707df6a0 msgr2=0x5563707e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec43200640 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707df6a0 0x5563707e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec43200640 1 -- 172.21.15.44:0/502506386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 msgr2=0x5563707df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec43200640 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707df030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec43200640 1 -- 172.21.15.44:0/502506386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563704532d0 con 0x5563707d9b40 2024-04-20T11:22:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec43200640 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 0x5563707d9f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009fd0 tx=0x7fec3c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d8bd4a4586ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec3be00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fec280a3bb0 0x7fec280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec33400640 1 -- 172.21.15.44:0/502506386 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c004d20 con 0x5563707d9b40 2024-04-20T11:22:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec33400640 1 -- 172.21.15.44:0/502506386 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec3c004050 con 0x5563707d9b40 2024-04-20T11:22:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.010+0000 7fec33400640 1 -- 172.21.15.44:0/502506386 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c005280 con 0x5563707d9b40 2024-04-20T11:22:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec42800640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fec280a3bb0 0x7fec280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec3be00640 1 -- 172.21.15.44:0/168760156 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fec340d0760 con 0x7fec441084a0 2024-04-20T11:22:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec42800640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fec280a3bb0 0x7fec280a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec340a5180 tx=0x7fec340d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 -- 172.21.15.44:0/502506386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 msgr2=0x5563707d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 0x5563707d9f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009fd0 tx=0x7fec3c0118d0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 -- 172.21.15.44:0/502506386 shutdown_connections 2024-04-20T11:22:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707df6a0 0x5563707e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 --2- 172.21.15.44:0/502506386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707d9b40 0x5563707d9f60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 -- 172.21.15.44:0/502506386 >> 172.21.15.44:0/502506386 conn(0x5563707d50b0 msgr2=0x5563707d54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 -- 172.21.15.44:0/502506386 shutdown_connections 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.011+0000 7fec4c03c740 1 -- 172.21.15.44:0/502506386 wait complete. 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.012+0000 7fec4c03c740 1 -- start start 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.012+0000 7fec4c03c740 1 Processor -- start 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5563707e2650 con 0x5563707df6a0 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563706fa5e0 con 0x5563707da530 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5563707e27d0 con 0x5563707d9b40 2024-04-20T11:22:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec4c03c740 10 client.0 _ll_register_callbacks cb 0x556370452ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58644/0 (socket says 172.21.15.44:58644) 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 -- 172.21.15.44:0/2549881488 learned_addr learned my addr 172.21.15.44:0/2549881488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec43200640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec43200640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 -- 172.21.15.44:0/2549881488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 msgr2=0x5563707e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 -- 172.21.15.44:0/2549881488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 msgr2=0x5563707eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec39e00640 1 -- 172.21.15.44:0/2549881488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637051d6b0 con 0x5563707da530 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.013+0000 7fec43200640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec39e00640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e4550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02ca30 tx=0x7fec2c02c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc0fece082ed8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec27e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:22:25.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec27e00640 1 -- 172.21.15.44:0/2549881488 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c045070 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c004050 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec27e00640 1 -- 172.21.15.44:0/2549881488 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fec2c005250 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec27e00640 1 -- 172.21.15.44:0/2549881488 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec2c051050 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec4c03c740 10 client.25268 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.014+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5563707ee9c0 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec27e00640 1 -- 172.21.15.44:0/2549881488 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec3c004050 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec27e00640 1 -- 172.21.15.44:0/2549881488 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fec2c0037f0 con 0x5563707da530 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec4c03c740 10 client.25268 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec4c03c740 10 client.25268 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec4c03c740 20 client.25268 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5563707f0f60 0x5563707f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec4c03c740 4 client.25268 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.015+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637074e3f0 con 0x5563707f0f60 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.016+0000 7fec43200640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5563707f0f60 0x5563707f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.016+0000 7fec43200640 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5563707f0f60 0x5563707f23d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8099691bad24474 server_cookie=55d9928626e04b64 in_seq=0 out_seq=0 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.016+0000 7fec27e00640 10 client.25268 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec27e00640 1 -- 172.21.15.44:0/2549881488 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55637074e3f0 con 0x5563707f0f60 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec27e00640 10 client.25268 handle_command_reply: tid=0 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 client.25268 shutdown 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 msgr2=0x5563707e4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e4550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02ca30 tx=0x7fec2c02c800 comp rx=0 tx=0).stop 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 shutdown_connections 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5563707f0f60 0x5563707f23d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e4550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2549881488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec27e00640 0 client.25268 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 >> 172.21.15.44:0/2549881488 conn(0x5563707d50b0 msgr2=0x5563707d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.017+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 shutdown_connections 2024-04-20T11:22:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.018+0000 7fec4c03c740 1 -- 172.21.15.44:0/2549881488 wait complete. 2024-04-20T11:22:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.018+0000 7fec4c03c740 20 client.25268 trim_cache size 0 max 16384 2024-04-20T11:22:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 Processor -- start 2024-04-20T11:22:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 -- start start 2024-04-20T11:22:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707df6a0 0x5563707d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5563707f0710 con 0x5563707da530 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563707f0de0 con 0x5563707df6a0 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec4c03c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5563707f14b0 con 0x5563707d9b40 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec39e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec39e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707eb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46394/0 (socket says 172.21.15.44:46394) 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.038+0000 7fec39e00640 1 -- 172.21.15.44:0/2072989414 learned_addr learned my addr 172.21.15.44:0/2072989414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec43200640 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec43200640 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707df6a0 0x5563707d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec39e00640 1 -- 172.21.15.44:0/2072989414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 msgr2=0x5563707e2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec39e00640 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec39e00640 1 -- 172.21.15.44:0/2072989414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707df6a0 msgr2=0x5563707d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec39e00640 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707df6a0 0x5563707d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec39e00640 1 -- 172.21.15.44:0/2072989414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563707decb0 con 0x5563707da530 2024-04-20T11:22:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec43200640 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707df6a0 0x5563707d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:22:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec39e00640 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707eb550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02c170 tx=0x7fec2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f7c37f9aab74af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.039+0000 7fec32a00640 1 -- 172.21.15.44:0/2072989414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c045070 con 0x5563707da530 2024-04-20T11:22:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec32a00640 1 -- 172.21.15.44:0/2072989414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec2c009780 con 0x5563707da530 2024-04-20T11:22:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec32a00640 1 -- 172.21.15.44:0/2072989414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c004a60 con 0x5563707da530 2024-04-20T11:22:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 -- 172.21.15.44:0/2072989414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 msgr2=0x5563707eb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707eb550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02c170 tx=0x7fec2c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 -- 172.21.15.44:0/2072989414 shutdown_connections 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707df6a0 0x5563707d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707da530 0x5563707eb550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 --2- 172.21.15.44:0/2072989414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 -- 172.21.15.44:0/2072989414 >> 172.21.15.44:0/2072989414 conn(0x5563707d50b0 msgr2=0x5563707d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 -- 172.21.15.44:0/2072989414 shutdown_connections 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.040+0000 7fec4c03c740 1 -- 172.21.15.44:0/2072989414 wait complete. 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 -- start start 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 Processor -- start 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5563707f2250 con 0x5563707df6a0 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563707f1b80 con 0x5563707da530 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.041+0000 7fec4c03c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5563707f2920 con 0x5563707d9b40 2024-04-20T11:22:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec4c03c740 10 client.0 _ll_register_callbacks cb 0x7fec34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41598/0 (socket says 172.21.15.44:41598) 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec39e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 -- 172.21.15.44:0/1205065774 learned_addr learned my addr 172.21.15.44:0/1205065774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 -- 172.21.15.44:0/1205065774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 msgr2=0x5563707e76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 -- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 msgr2=0x5563707e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec43200640 1 -- 172.21.15.44:0/1205065774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563707eaed0 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.042+0000 7fec39e00640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec43200640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707ea8e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec3c003a50 tx=0x7fec3c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8c64a8b6185856 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec27e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec27e00640 1 -- 172.21.15.44:0/1205065774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003d80 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec27e00640 1 -- 172.21.15.44:0/1205065774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fec3c02fb50 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec2c009780 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec27e00640 1 -- 172.21.15.44:0/1205065774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec3c02fd50 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec4c03c740 10 client.15159 fetch_fsmap learned FSMap version 26 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556370857630 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.043+0000 7fec27e00640 1 -- 172.21.15.44:0/1205065774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec2c009780 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec27e00640 1 -- 172.21.15.44:0/1205065774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fec3c025040 con 0x5563707df6a0 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec4c03c740 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec4c03c740 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T11:22:25.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec4c03c740 20 client.15159 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:25.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec4c03c740 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55637085c0b0 0x55637085adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec4c03c740 4 client.15159 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T11:22:25.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "status"}) -- 0x556370748260 con 0x55637085c0b0 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.044+0000 7fec39e00640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55637085c0b0 0x55637085adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec39e00640 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55637085c0b0 0x55637085adc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609a98f208f5b74a server_cookie=b3f6f42664550c6b in_seq=0 out_seq=0 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec27e00640 10 client.15159 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec27e00640 1 -- 172.21.15.44:0/1205065774 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556370748260 con 0x55637085c0b0 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec27e00640 10 client.15159 handle_command_reply: tid=0 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 client.15159 shutdown 2024-04-20T11:22:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 msgr2=0x5563707ea8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707ea8e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec3c003a50 tx=0x7fec3c0118d0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 shutdown_connections 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55637085c0b0 0x55637085adc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5563707df6a0 0x5563707ea8e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5563707da530 0x5563707e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 --2- 172.21.15.44:0/1205065774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563707d9b40 0x5563707e76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec27e00640 0 client.15159 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.045+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 >> 172.21.15.44:0/1205065774 conn(0x5563707d50b0 msgr2=0x5563707d8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 shutdown_connections 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec4c03c740 1 -- 172.21.15.44:0/1205065774 wait complete. 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec4c03c740 20 client.15159 trim_cache size 0 max 16384 2024-04-20T11:22:25.049 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "759aac0c-ff06-11ee-bc92-c7b262605968", 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24596, 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 142.35633091400001, 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 87, 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 147.636807315 2024-04-20T11:22:25.050 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fec280a3bb0 msgr2=0x7fec280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fec280a3bb0 0x7fec280a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec340a5180 tx=0x7fec340d6530 comp rx=0 tx=0).stop 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 msgr2=0x7fec4419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 0x7fec4419e840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fec3400e9c0 tx=0x7fec3400ee80 comp rx=0 tx=0).stop 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 shutdown_connections 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fec280a3bb0 0x7fec280a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec4410a050 0x7fec441a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec44109690 0x7fec4419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 --2- 172.21.15.44:0/168760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec441084a0 0x7fec4419e840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.046+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 >> 172.21.15.44:0/168760156 conn(0x7fec44077430 msgr2=0x7fec4410c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.047+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 shutdown_connections 2024-04-20T11:22:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.047+0000 7fec48a00640 1 -- 172.21.15.44:0/168760156 wait complete. 2024-04-20T11:22:25.065 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T11:22:25.077 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T11:22:25.077 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T11:22:25.078 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T11:22:25.078 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-20T11:22:25.079 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T11:22:25.079 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:22:25.079 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 -- 172.21.15.44:0/2017410184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105210 msgr2=0x7f5790105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 --2- 172.21.15.44:0/2017410184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105210 0x7f5790105690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5780009860 tx=0x7f578002bb70 comp rx=0 tx=0).stop 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 -- 172.21.15.44:0/2017410184 shutdown_connections 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 --2- 172.21.15.44:0/2017410184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105bd0 0x7f579010c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 --2- 172.21.15.44:0/2017410184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105210 0x7f5790105690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 --2- 172.21.15.44:0/2017410184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f5790104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 -- 172.21.15.44:0/2017410184 >> 172.21.15.44:0/2017410184 conn(0x7f57900ff5d0 msgr2=0x7f57901019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 -- 172.21.15.44:0/2017410184 shutdown_connections 2024-04-20T11:22:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.285+0000 7f5797200640 1 -- 172.21.15.44:0/2017410184 wait complete. 2024-04-20T11:22:25.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.286+0000 7f5797200640 1 Processor -- start 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.286+0000 7f5797200640 1 -- start start 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.286+0000 7f5797200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f579019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.286+0000 7f5797200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105210 0x7f579019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5797200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105bd0 0x7f57901a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5797200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f579010fdd0 con 0x7f5790105210 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5797200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f579010fc50 con 0x7f5790105bd0 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5797200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f579010ff50 con 0x7f5790104020 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f579019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105210 0x7f579019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f579019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41616/0 (socket says 172.21.15.44:41616) 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795e00640 1 -- 172.21.15.44:0/3569012289 learned_addr learned my addr 172.21.15.44:0/3569012289 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5796800640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105bd0 0x7f57901a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795400640 1 -- 172.21.15.44:0/3569012289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 msgr2=0x7f579019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795400640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f579019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795400640 1 -- 172.21.15.44:0/3569012289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105bd0 msgr2=0x7f57901a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795400640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105bd0 0x7f57901a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795400640 1 -- 172.21.15.44:0/3569012289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57901a1fc0 con 0x7f5790105210 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5795e00640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f579019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.287+0000 7f5796800640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105bd0 0x7f57901a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.288+0000 7f5795400640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105210 0x7f579019a8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f578002c0b0 tx=0x7f5780035f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.288+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780041020 con 0x7f5790105210 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.288+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f578002ca10 con 0x7f5790105210 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.288+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57901a2250 con 0x7f5790105210 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.288+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780039040 con 0x7f5790105210 2024-04-20T11:22:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.288+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57901a2760 con 0x7f5790105210 2024-04-20T11:22:25.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.289+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579010b2e0 con 0x7f5790105210 2024-04-20T11:22:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.291+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5780044020 con 0x7f5790105210 2024-04-20T11:22:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.293+0000 7f578aa00640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f57780a3c20 0x7f57780a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.293+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f57800f20b0 con 0x7f5790105210 2024-04-20T11:22:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.293+0000 7f5795e00640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f57780a3c20 0x7f57780a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.294+0000 7f5795e00640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f57780a3c20 0x7f57780a60e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f57840104e0 tx=0x7f578400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.296+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f578004b020 con 0x7f5790105210 2024-04-20T11:22:25.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:25.557+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f579002d760 con 0x7f5790105210 2024-04-20T11:22:25.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:25 smithi078 ceph-mon[21464]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:25.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:25 smithi044 ceph-mon[16855]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:25.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:25 smithi165 ceph-mon[21975]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-20T11:22:26.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.454+0000 7f578aa00640 1 -- 172.21.15.44:0/3569012289 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f57800beb10 con 0x7f5790105210 2024-04-20T11:22:26.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f57780a3c20 msgr2=0x7f57780a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:26.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f57780a3c20 0x7f57780a60e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f57840104e0 tx=0x7f578400b410 comp rx=0 tx=0).stop 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105210 msgr2=0x7f579019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105210 0x7f579019a8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f578002c0b0 tx=0x7f5780035f80 comp rx=0 tx=0).stop 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 shutdown_connections 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f57780a3c20 0x7f57780a60e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790105bd0 0x7f57901a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5790105210 0x7f579019a8e0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f578002c0b0 tx=0x7f5780035f80 comp rx=0 tx=0).stop 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 --2- 172.21.15.44:0/3569012289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5790104020 0x7f579019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.463+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 >> 172.21.15.44:0/3569012289 conn(0x7f57900ff5d0 msgr2=0x7f57901084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:26.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.468+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 shutdown_connections 2024-04-20T11:22:26.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.468+0000 7f5797200640 1 -- 172.21.15.44:0/3569012289 wait complete. 2024-04-20T11:22:26.497 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 -- 172.21.15.44:0/3475551472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4441041f0 msgr2=0x7fd444104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 --2- 172.21.15.44:0/3475551472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4441041f0 0x7fd444104610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd43402bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 -- 172.21.15.44:0/3475551472 shutdown_connections 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 --2- 172.21.15.44:0/3475551472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd444105da0 0x7fd44410c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 --2- 172.21.15.44:0/3475551472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441053e0 0x7fd444105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 --2- 172.21.15.44:0/3475551472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4441041f0 0x7fd444104610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 -- 172.21.15.44:0/3475551472 >> 172.21.15.44:0/3475551472 conn(0x7fd4440ff780 msgr2=0x7fd444101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 -- 172.21.15.44:0/3475551472 shutdown_connections 2024-04-20T11:22:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.686+0000 7fd44b200640 1 -- 172.21.15.44:0/3475551472 wait complete. 2024-04-20T11:22:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.687+0000 7fd44b200640 1 Processor -- start 2024-04-20T11:22:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 -- start start 2024-04-20T11:22:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 0x7fd44419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4441053e0 0x7fd44419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd444105da0 0x7fd4441a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd44410ff50 con 0x7fd444105da0 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd44410fdd0 con 0x7fd4441053e0 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd44b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441100d0 con 0x7fd4441041f0 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.688+0000 7fd443e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 0x7fd44419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd449200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd444105da0 0x7fd4441a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 0x7fd44419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41646/0 (socket says 172.21.15.44:41646) 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 -- 172.21.15.44:0/3209035207 learned_addr learned my addr 172.21.15.44:0/3209035207 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4441053e0 0x7fd44419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 -- 172.21.15.44:0/3209035207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4441053e0 msgr2=0x7fd44419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4441053e0 0x7fd44419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 -- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd444105da0 msgr2=0x7fd4441a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd444105da0 0x7fd4441a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 -- 172.21.15.44:0/3209035207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4441a2100 con 0x7fd4441041f0 2024-04-20T11:22:26.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.689+0000 7fd443e00640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 0x7fd44419a4e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:26.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.690+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434043020 con 0x7fd4441041f0 2024-04-20T11:22:26.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.690+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd434004250 con 0x7fd4441041f0 2024-04-20T11:22:26.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.690+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434003b90 con 0x7fd4441041f0 2024-04-20T11:22:26.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.690+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4441a2390 con 0x7fd4441041f0 2024-04-20T11:22:26.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.690+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4441a2930 con 0x7fd4441041f0 2024-04-20T11:22:26.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.691+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44410b4b0 con 0x7fd4441041f0 2024-04-20T11:22:26.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.693+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fd434046020 con 0x7fd4441041f0 2024-04-20T11:22:26.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.694+0000 7fd440c00640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:26.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.694+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7fd4340f2440 con 0x7fd4441041f0 2024-04-20T11:22:26.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.694+0000 7fd443400640 1 -- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 msgr2=0x7fd4280a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:26.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.694+0000 7fd443400640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:22:26.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:26 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3569012289' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T11:22:26.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: ignoring --setuser ceph since I am not root 2024-04-20T11:22:26.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: ignoring --setgroup ceph since I am not root 2024-04-20T11:22:26.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.699+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd43404d020 con 0x7fd4441041f0 2024-04-20T11:22:26.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3569012289' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T11:22:26.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: ignoring --setuser ceph since I am not root 2024-04-20T11:22:26.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: ignoring --setgroup ceph since I am not root 2024-04-20T11:22:26.730 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-mgr[23838]: -- 172.21.15.165:0/2448633553 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6bc2ce5a0 con 0x55d6bc2a2c00 2024-04-20T11:22:26.730 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:26.636+0000 7faa73fcf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:22:26.820 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3569012289' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T11:22:26.820 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: ignoring --setuser ceph since I am not root 2024-04-20T11:22:26.821 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: ignoring --setgroup ceph since I am not root 2024-04-20T11:22:26.821 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-mgr[22860]: -- 172.21.15.78:0/2284465307 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556bd4d805a0 con 0x556bd4d54800 2024-04-20T11:22:26.821 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:26.634+0000 7f3b642a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:22:26.821 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:26.820+0000 7f3b642a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:22:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.895+0000 7fd443400640 1 -- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 msgr2=0x7fd4280a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.895+0000 7fd443400640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:22:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.915+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd44402d5d0 con 0x7fd4441041f0 2024-04-20T11:22:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.922+0000 7fd440c00640 1 -- 172.21.15.44:0/3209035207 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fd4340bee20 con 0x7fd4441041f0 2024-04-20T11:22:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 msgr2=0x7fd4280a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 msgr2=0x7fd44419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 0x7fd44419a4e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004a30 comp rx=0 tx=0).stop 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 shutdown_connections 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd444105da0 0x7fd4441a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4441053e0 0x7fd44419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 --2- 172.21.15.44:0/3209035207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4441041f0 0x7fd44419a4e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 >> 172.21.15.44:0/3209035207 conn(0x7fd4440ff780 msgr2=0x7fd444108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 shutdown_connections 2024-04-20T11:22:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:26.925+0000 7fd44b200640 1 -- 172.21.15.44:0/3209035207 wait complete. 2024-04-20T11:22:26.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-20T11:22:27.059 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:26.800+0000 7f6256869200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T11:22:27.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 -- 172.21.15.44:0/3205932879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0105190 msgr2=0x7f24e0107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:27.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3205932879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0105190 0x7f24e0107580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009900 tx=0x7f24cc02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:27.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 -- 172.21.15.44:0/3205932879 shutdown_connections 2024-04-20T11:22:27.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3205932879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0107ac0 0x7f24e0109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3205932879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0105190 0x7f24e0107580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3205932879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e0102860 0x7f24e0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.134+0000 7f24e6c00640 1 -- 172.21.15.44:0/3205932879 >> 172.21.15.44:0/3205932879 conn(0x7f24e00fc250 msgr2=0x7f24e00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.135+0000 7f24e6c00640 1 -- 172.21.15.44:0/3205932879 shutdown_connections 2024-04-20T11:22:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.135+0000 7f24e6c00640 1 -- 172.21.15.44:0/3205932879 wait complete. 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 Processor -- start 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 -- start start 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 0x7f24e019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e0105190 0x7f24e019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0107ac0 0x7f24e01a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010db20 con 0x7f24e0105190 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010d9a0 con 0x7f24e0102860 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010dca0 con 0x7f24e0107ac0 2024-04-20T11:22:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 0x7f24e019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 0x7f24e019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58720/0 (socket says 172.21.15.44:58720) 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e5800640 1 -- 172.21.15.44:0/3041679204 learned_addr learned my addr 172.21.15.44:0/3041679204 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e4e00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e0105190 0x7f24e019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.136+0000 7f24e6200640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0107ac0 0x7f24e01a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e5800640 1 -- 172.21.15.44:0/3041679204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0107ac0 msgr2=0x7f24e01a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e5800640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0107ac0 0x7f24e01a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e5800640 1 -- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e0105190 msgr2=0x7f24e019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e5800640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e0105190 0x7f24e019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e5800640 1 -- 172.21.15.44:0/3041679204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e01a2030 con 0x7f24e0102860 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e5800640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 0x7f24e019a410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24d400ea60 tx=0x7f24d400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d400cde0 con 0x7f24e0102860 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e01a2320 con 0x7f24e0102860 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f24d40035f0 con 0x7f24e0102860 2024-04-20T11:22:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4010720 con 0x7f24e0102860 2024-04-20T11:22:27.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.137+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01a2890 con 0x7f24e0102860 2024-04-20T11:22:27.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.139+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e01095b0 con 0x7f24e0102860 2024-04-20T11:22:27.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.140+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f24d40037a0 con 0x7f24e0102860 2024-04-20T11:22:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.142+0000 7f24de000640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 0x7f24c80a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.142+0000 7f24e4e00640 1 -- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 msgr2=0x7f24c80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.142+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f24d40d0760 con 0x7f24e0102860 2024-04-20T11:22:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.142+0000 7f24e4e00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 0x7f24c80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:22:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.146+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f24d409d140 con 0x7f24e0102860 2024-04-20T11:22:27.188 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:26.921+0000 7f3b642a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:22:27.230 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:26.824+0000 7faa73fcf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:22:27.231 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:26.925+0000 7faa73fcf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:22:27.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.342+0000 7f24e4e00640 1 -- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 msgr2=0x7f24c80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:27.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.342+0000 7f24e4e00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 0x7f24c80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:22:27.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.351+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f24e0062b60 con 0x7f24e0102860 2024-04-20T11:22:27.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.358+0000 7f24de000640 1 -- 172.21.15.44:0/3041679204 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f24d409ca40 con 0x7f24e0102860 2024-04-20T11:22:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 msgr2=0x7f24c80a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:22:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 0x7f24c80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 msgr2=0x7f24e019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 0x7f24e019a410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24d400ea60 tx=0x7f24d400ef20 comp rx=0 tx=0).stop 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 shutdown_connections 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f24c80a3bf0 0x7f24c80a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e0107ac0 0x7f24e01a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e0105190 0x7f24e019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 --2- 172.21.15.44:0/3041679204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24e0102860 0x7f24e019a410 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 >> 172.21.15.44:0/3041679204 conn(0x7f24e00fc250 msgr2=0x7f24e0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 shutdown_connections 2024-04-20T11:22:27.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:27.361+0000 7f24e6c00640 1 -- 172.21.15.44:0/3041679204 wait complete. 2024-04-20T11:22:27.377 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:27.058+0000 7f6256869200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:22:27.378 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:27.196+0000 7f6256869200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T11:22:27.379 INFO:teuthology.run_tasks:Running task kclient... 2024-04-20T11:22:27.404 INFO:tasks.kclient:Mounting kernel clients... 2024-04-20T11:22:27.405 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-20T11:22:27.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:27.405 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-20T11:22:27.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:27.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-20T11:22:27.536 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-04-20T11:22:27.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:22:27.540 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'rbytes': False, 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-20T11:22:27.540 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T11:22:27.540 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:27.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph fs ls 2024-04-20T11:22:27.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:27 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3569012289' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T11:22:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:27 smithi078 ceph-mon[21464]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T11:22:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:27 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' 2024-04-20T11:22:27.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:27 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' 2024-04-20T11:22:27.688 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:27.231+0000 7f3b642a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:22:27.688 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:27.316+0000 7f3b642a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:22:27.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:27.614+0000 7f6256869200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:22:27.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3569012289' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T11:22:27.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-mon[16855]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T11:22:27.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' 2024-04-20T11:22:27.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' 2024-04-20T11:22:27.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:27.235+0000 7faa73fcf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T11:22:27.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:27.320+0000 7faa73fcf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:22:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:27 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3569012289' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T11:22:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:27 smithi165 ceph-mon[21975]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T11:22:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:27 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' 2024-04-20T11:22:27.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:27 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' 2024-04-20T11:22:27.982 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:22:28.132 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:27.733+0000 7f6256869200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T11:22:28.817 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:28 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:28.485+0000 7f3b642a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:22:28.817 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:28 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:28.650+0000 7f3b642a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:22:28.817 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:28 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:28.816+0000 7f3b642a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:22:28.843 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:28.502+0000 7faa73fcf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:22:28.843 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:28.672+0000 7faa73fcf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:22:28.843 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:28.842+0000 7faa73fcf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:22:29.388 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:29 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:29.100+0000 7f3b642a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:22:29.389 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:29 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:29.191+0000 7f3b642a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:22:29.389 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:29 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:29.388+0000 7f3b642a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:22:29.427 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:29.132+0000 7faa73fcf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:22:29.427 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:29.224+0000 7faa73fcf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:22:29.605 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:29 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:29.362+0000 7f6256869200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T11:22:29.605 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:29 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:29.603+0000 7f6256869200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T11:22:29.687 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:29 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:29.579+0000 7f3b642a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:22:29.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:29.426+0000 7faa73fcf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:22:29.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:29.619+0000 7faa73fcf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:22:29.944 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:29 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:29.831+0000 7f6256869200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T11:22:30.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 -- 172.21.15.44:0/4288750513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 msgr2=0x7f3e48075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:30.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 --2- 172.21.15.44:0/4288750513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e48075260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009990 tx=0x7f3e3c009ca0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 -- 172.21.15.44:0/4288750513 shutdown_connections 2024-04-20T11:22:30.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 --2- 172.21.15.44:0/4288750513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4810f1c0 0x7f3e48111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 --2- 172.21.15.44:0/4288750513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e48075260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 --2- 172.21.15.44:0/4288750513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e480757a0 0x7f3e4810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 -- 172.21.15.44:0/4288750513 >> 172.21.15.44:0/4288750513 conn(0x7f3e480fe4d0 msgr2=0x7f3e481008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:30.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 -- 172.21.15.44:0/4288750513 shutdown_connections 2024-04-20T11:22:30.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.121+0000 7f3e4e400700 1 -- 172.21.15.44:0/4288750513 wait complete. 2024-04-20T11:22:30.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.122+0000 7f3e4e400700 1 Processor -- start 2024-04-20T11:22:30.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 -- start start 2024-04-20T11:22:30.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e4819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e480757a0 0x7f3e4819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4810f1c0 0x7f3e4819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48114ff0 con 0x7f3e4810f1c0 2024-04-20T11:22:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48114e70 con 0x7f3e48074e80 2024-04-20T11:22:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4e400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48115170 con 0x7f3e480757a0 2024-04-20T11:22:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4d000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e4819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4d000700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e4819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59320/0 (socket says 172.21.15.44:59320) 2024-04-20T11:22:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4d000700 1 -- 172.21.15.44:0/262455088 learned_addr learned my addr 172.21.15.44:0/262455088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e47e00700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e480757a0 0x7f3e4819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.123+0000 7f3e4da00700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4810f1c0 0x7f3e4819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4d000700 1 -- 172.21.15.44:0/262455088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e480757a0 msgr2=0x7f3e4819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:30.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4d000700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e480757a0 0x7f3e4819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4d000700 1 -- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4810f1c0 msgr2=0x7f3e4819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4d000700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4810f1c0 0x7f3e4819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4d000700 1 -- 172.21.15.44:0/262455088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481a0410 con 0x7f3e48074e80 2024-04-20T11:22:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4d000700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e4819b400 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0095b0 tx=0x7f3e3c005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c026070 con 0x7f3e48074e80 2024-04-20T11:22:30.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481a06a0 con 0x7f3e48074e80 2024-04-20T11:22:30.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e3c0056b0 con 0x7f3e48074e80 2024-04-20T11:22:30.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.124+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c021e60 con 0x7f3e48074e80 2024-04-20T11:22:30.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.125+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481a7f80 con 0x7f3e48074e80 2024-04-20T11:22:30.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.126+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e48069740 con 0x7f3e48074e80 2024-04-20T11:22:30.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.127+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3e3c029070 con 0x7f3e48074e80 2024-04-20T11:22:30.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.129+0000 7f3e45600700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 0x7f3e300a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:30.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.129+0000 7f3e47e00700 1 -- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 msgr2=0x7f3e300a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:30.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.129+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f3e3c0d0ff0 con 0x7f3e48074e80 2024-04-20T11:22:30.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.129+0000 7f3e47e00700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 0x7f3e300a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:22:30.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.131+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e3c013070 con 0x7f3e48074e80 2024-04-20T11:22:30.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.330+0000 7f3e47e00700 1 -- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 msgr2=0x7f3e300a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:30.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.330+0000 7f3e47e00700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 0x7f3e300a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:22:30.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.412+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e481a8150 con 0x7f3e48074e80 2024-04-20T11:22:30.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.413+0000 7f3e45600700 1 -- 172.21.15.44:0/262455088 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f3e3c09da80 con 0x7f3e48074e80 2024-04-20T11:22:30.414 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T11:22:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 msgr2=0x7f3e300a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:22:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 0x7f3e300a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 msgr2=0x7f3e4819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:30.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e4819b400 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0095b0 tx=0x7f3e3c005590 comp rx=0 tx=0).stop 2024-04-20T11:22:30.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 shutdown_connections 2024-04-20T11:22:30.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f3e300a3d10 0x7f3e300a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4810f1c0 0x7f3e4819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48074e80 0x7f3e4819b400 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 --2- 172.21.15.44:0/262455088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e480757a0 0x7f3e4819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:30.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.415+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 >> 172.21.15.44:0/262455088 conn(0x7f3e480fe4d0 msgr2=0x7f3e48111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:30.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.416+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 shutdown_connections 2024-04-20T11:22:30.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:30.416+0000 7f3e4e400700 1 -- 172.21.15.44:0/262455088 wait complete. 2024-04-20T11:22:30.437 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:30 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:30.168+0000 7f3b642a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:22:30.479 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:30.222+0000 7faa73fcf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:22:30.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:30 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/262455088' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:22:30.637 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:30 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/262455088' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:22:30.638 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:30 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:30.238+0000 7f6256869200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T11:22:30.638 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:30 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:30.366+0000 7f6256869200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T11:22:30.638 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:30 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:30.636+0000 7f6256869200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T11:22:30.880 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:30 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/262455088' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T11:22:30.880 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:30 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:30.584+0000 7f3b642a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:22:30.880 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:30 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:30.680+0000 7f3b642a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:22:30.942 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:30.644+0000 7faa73fcf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:22:30.942 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:30.739+0000 7faa73fcf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:22:30.942 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:30.941+0000 7faa73fcf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:22:31.187 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:30 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:30.880+0000 7f3b642a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:22:31.193 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:30 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:30.911+0000 7f6256869200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T11:22:31.264 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:22:31.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:31.265 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:22:31.317 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42027sec preferred_lft 42027sec 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-04-20T11:22:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-04-20T11:22:31.319 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T11:22:31.320 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-20T11:22:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:22:31.688 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:31.399+0000 7f3b642a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:22:31.689 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:31.491+0000 7f3b642a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:22:31.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:31.468+0000 7faa73fcf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:22:31.729 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:31.560+0000 7faa73fcf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:22:31.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:31.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:31.758 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T11:22:31.822 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-04-20T11:22:31.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:31.825 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-04-20T11:22:31.876 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-04-20T11:22:31.876 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T11:22:31.877 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T11:22:31.877 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T11:22:31.877 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T11:22:31.877 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:22:31.877 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-20T11:22:31.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T11:22:31.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T11:22:31.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T11:22:31.878 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-20T11:22:31.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:31 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:31.780+0000 7f6256869200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T11:22:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:22:32.072 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:31.732+0000 7f3b642a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:22:32.072 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:31.864+0000 7f3b642a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:22:32.072 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:31.958+0000 7f3b642a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:22:32.145 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:31.803+0000 7faa73fcf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:22:32.145 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:31.935+0000 7faa73fcf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:22:32.145 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:32.030+0000 7faa73fcf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:22:32.145 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:32.144+0000 7faa73fcf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:22:32.370 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:32.071+0000 7f3b642a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:22:32.517 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:32 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:32.377+0000 7f6256869200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T11:22:32.517 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:32 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:32.515+0000 7f6256869200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T11:22:32.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:32.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:32.589 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-20T11:22:32.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:32.635 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-04-20T11:22:32.687 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:32.370+0000 7f3b642a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:22:32.729 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:22:32.729 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-20T11:22:32.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:22:32.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T11:22:32.729 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-20T11:22:32.730 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:32.450+0000 7faa73fcf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:22:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:22:32.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:32.905 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T11:22:32.906 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-20T11:22:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:22:33.066 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:32 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:32.813+0000 7f6256869200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T11:22:33.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:33.198 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T11:22:33.199 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-20T11:22:33.199 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-04-20T11:22:33.199 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T11:22:33.199 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-20T11:22:33.254 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:32.969+0000 7f3b642a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:22:33.254 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y[22856]: 2024-04-20T11:22:33.186+0000 7f3b642a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:22:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:22:33.321 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-mon[16855]: Standby manager daemon y restarted 2024-04-20T11:22:33.321 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-mon[16855]: Standby manager daemon y started 2024-04-20T11:22:33.322 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T11:22:33.322 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:22:33.322 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T11:22:33.322 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:22:33.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:33.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:22:33.393 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:33.433 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T11:22:33.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T11:22:33.433 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:33.479 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:33.051+0000 7faa73fcf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:22:33.479 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z[23834]: 2024-04-20T11:22:33.272+0000 7faa73fcf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:22:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-mon[21975]: Standby manager daemon y restarted 2024-04-20T11:22:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-mon[21975]: Standby manager daemon y started 2024-04-20T11:22:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T11:22:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:22:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T11:22:33.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:33 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:22:33.522 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-20T11:22:33.522 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-20T11:22:33.522 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/2d5dd744-448e-43b3-845f-21ec1fa93e09 2024-04-20T11:22:33.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:33.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/2d5dd744-448e-43b3-845f-21ec1fa93e09 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T11:22:33.580 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:33.522+0000 7f6256869200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T11:22:33.639 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T11:22:33.640 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-20T11:22:33.640 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-04-20T11:22:33.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-mon[21464]: Standby manager daemon y restarted 2024-04-20T11:22:33.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-mon[21464]: Standby manager daemon y started 2024-04-20T11:22:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T11:22:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:22:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T11:22:33.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:33 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.78:0/58952625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:22:33.898 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:33 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:33.691+0000 7f6256869200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T11:22:34.374 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:34.067+0000 7f6256869200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T11:22:34.374 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:34.269+0000 7f6256869200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T11:22:34.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-20T11:22:34.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: Standby manager daemon z restarted 2024-04-20T11:22:34.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: Standby manager daemon z started 2024-04-20T11:22:34.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T11:22:34.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:22:34.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T11:22:34.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-mon[16855]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:22:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T11:22:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-20T11:22:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-04-20T11:22:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.78:3300,172.21.15.165:3300" 2024-04-20T11:22:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:3300,172.21.15.78:3300,172.21.15.165:3300:/volumes/qa/sv_1/2d5dd744-448e-43b3-845f-21ec1fa93e09 2024-04-20T11:22:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=759aac0c-ff06-11ee-bc92-c7b262605968" will pass to kernel 2024-04-20T11:22:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-20T11:22:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-04-20T11:22:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-20T11:22:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-04-20T11:22:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-20T11:22:34.403 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-20T11:22:34.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:34.403 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:34.469 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T11:22:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:22:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:34.575 INFO:teuthology.orchestra.run.smithi044.stdout:client25309 2024-04-20T11:22:34.575 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:22:34.575 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:22:34.626 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T11:22:34.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-20T11:22:34.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: Standby manager daemon z restarted 2024-04-20T11:22:34.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: Standby manager daemon z started 2024-04-20T11:22:34.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T11:22:34.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:22:34.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T11:22:34.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:34 smithi078 ceph-mon[21464]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:22:34.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:34.411+0000 7f6256869200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T11:22:34.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:34.573+0000 7f6256869200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: Standby manager daemon z restarted 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: Standby manager daemon z started 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T11:22:34.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:34 smithi165 ceph-mon[21975]: from='mgr.? 172.21.15.165:0/3189696133' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T11:22:34.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:22:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:34.781 INFO:teuthology.orchestra.run.smithi044.stdout:client25309 2024-04-20T11:22:34.781 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T11:22:34.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T11:22:34.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:34.945 INFO:teuthology.orchestra.run.smithi044.stdout:client25309 2024-04-20T11:22:34.946 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/759aac0c-ff06-11ee-bc92-c7b262605968.client25309/status') 2024-04-20T11:22:35.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T11:22:35.071 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.25309 (3)172.21.15.44:0/3102977493 2024-04-20T11:22:35.071 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-04-20T11:22:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-04-20T11:22:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-04-20T11:22:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:71 bytes copied, 7.9595e-05 s, 892 kB/s 2024-04-20T11:22:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T11:22:35.084 INFO:teuthology.run_tasks:Running task exec... 2024-04-20T11:22:35.093 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T11:22:35.094 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:22:35.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-20T11:22:35.269 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:34.960+0000 7f6256869200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 -- 172.21.15.44:0/2331647540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0102fd0 msgr2=0x7f5ba0103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 --2- 172.21.15.44:0/2331647540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0102fd0 0x7f5ba0103450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009900 tx=0x7f5b8c02bc50 comp rx=0 tx=0).stop 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 -- 172.21.15.44:0/2331647540 shutdown_connections 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 --2- 172.21.15.44:0/2331647540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ba0103990 0x7f5ba010a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 --2- 172.21.15.44:0/2331647540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0102fd0 0x7f5ba0103450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 --2- 172.21.15.44:0/2331647540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba01022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 -- 172.21.15.44:0/2331647540 >> 172.21.15.44:0/2331647540 conn(0x7f5ba00fd3f0 msgr2=0x7f5ba00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 -- 172.21.15.44:0/2331647540 shutdown_connections 2024-04-20T11:22:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.286+0000 7f5ba5400640 1 -- 172.21.15.44:0/2331647540 wait complete. 2024-04-20T11:22:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 Processor -- start 2024-04-20T11:22:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 -- start start 2024-04-20T11:22:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba0075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ba0102fd0 0x7f5ba0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0103990 0x7f5ba0074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba010dbe0 con 0x7f5ba0103990 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba010da60 con 0x7f5ba0101e80 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.288+0000 7f5ba5400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba010dd60 con 0x7f5ba0102fd0 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba0075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5ba4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0103990 0x7f5ba0074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba0075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59334/0 (socket says 172.21.15.44:59334) 2024-04-20T11:22:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 -- 172.21.15.44:0/897045520 learned_addr learned my addr 172.21.15.44:0/897045520 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9f400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ba0102fd0 0x7f5ba0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ba0102fd0 msgr2=0x7f5ba0074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ba0102fd0 0x7f5ba0074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0103990 msgr2=0x7f5ba0074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0103990 0x7f5ba0074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 -- 172.21.15.44:0/897045520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba0075120 con 0x7f5ba0101e80 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.289+0000 7f5b9fe00640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba0075d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000ea60 tx=0x7f5b9000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.290+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000cde0 con 0x7f5ba0101e80 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.290+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b900035f0 con 0x7f5ba0101e80 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.290+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90010720 con 0x7f5ba0101e80 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.290+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba01a2340 con 0x7f5ba0101e80 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.290+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba01a2860 con 0x7f5ba0101e80 2024-04-20T11:22:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.291+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba0050540 con 0x7f5ba0101e80 2024-04-20T11:22:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.293+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7f5b900037a0 con 0x7f5ba0101e80 2024-04-20T11:22:35.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.294+0000 7f5b9cc00640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 0x7f5b840a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:35.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.294+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f5b900d0860 con 0x7f5ba0101e80 2024-04-20T11:22:35.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.295+0000 7f5b9f400640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 msgr2=0x7f5b840a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:35.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.295+0000 7f5b9f400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 0x7f5b840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T11:22:35.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.299+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5b9009d240 con 0x7f5ba0101e80 2024-04-20T11:22:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.496+0000 7f5b9f400640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 msgr2=0x7f5b840a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.496+0000 7f5b9f400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 0x7f5b840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T11:22:35.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.543+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f5ba0109190 con 0x7f5ba0101e80 2024-04-20T11:22:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:35 smithi078 ceph-mon[21464]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-20T11:22:35.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:35 smithi044 ceph-mon[16855]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-20T11:22:35.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:35 smithi165 ceph-mon[21975]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-20T11:22:35.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.897+0000 7f5b9f400640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 msgr2=0x7f5b840a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:6800/2425002048 2024-04-20T11:22:35.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:35.897+0000 7f5b9f400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 0x7f5b840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T11:22:36.087 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:35 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:35.784+0000 7f6256869200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T11:22:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.109+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f5b900991b0 con 0x7f5ba0101e80 2024-04-20T11:22:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.109+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 msgr2=0x7f5b840a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T11:22:36.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.109+0000 7f5b9cc00640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 0x7f5b840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.274+0000 7f5b9cc00640 1 -- 172.21.15.44:0/897045520 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f5ba0109190 con 0x7f5ba0101e80 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 msgr2=0x7f5ba0075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba0075d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000ea60 tx=0x7f5b9000ef20 comp rx=0 tx=0).stop 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 shutdown_connections 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:6800/2425002048,v1:172.21.15.44:6801/2425002048] conn(0x7f5b840a3bf0 0x7f5b840a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba0103990 0x7f5ba0074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ba0102fd0 0x7f5ba0074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 --2- 172.21.15.44:0/897045520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ba0101e80 0x7f5ba0075d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.278+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 >> 172.21.15.44:0/897045520 conn(0x7f5ba00fd3f0 msgr2=0x7f5ba0106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:36.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.280+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 shutdown_connections 2024-04-20T11:22:36.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.280+0000 7f5ba5400640 1 -- 172.21.15.44:0/897045520 wait complete. 2024-04-20T11:22:36.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-20T11:22:36.372 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.085+0000 7f6256869200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T11:22:36.373 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/897045520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T11:22:36.373 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T11:22:36.373 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: Active manager daemon x restarted 2024-04-20T11:22:36.373 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: Activating manager daemon x 2024-04-20T11:22:36.373 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: mgrmap e22: x(active, starting, since 0.0157185s), standbys: z, y 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:22:36.374 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:22:36.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: Manager daemon x is now available 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:22:36.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T11:22:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 -- 172.21.15.44:0/3119984593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102f70 msgr2=0x7ff9a81033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 --2- 172.21.15.44:0/3119984593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102f70 0x7ff9a81033f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff994009900 tx=0x7ff99402bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 -- 172.21.15.44:0/3119984593 shutdown_connections 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 --2- 172.21.15.44:0/3119984593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a8103930 0x7ff9a810a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 --2- 172.21.15.44:0/3119984593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102f70 0x7ff9a81033f0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff994009900 tx=0x7ff99402bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 --2- 172.21.15.44:0/3119984593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.521+0000 7ff9a7200640 1 -- 172.21.15.44:0/3119984593 >> 172.21.15.44:0/3119984593 conn(0x7ff9a80fd390 msgr2=0x7ff9a80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.522+0000 7ff9a7200640 1 -- 172.21.15.44:0/3119984593 shutdown_connections 2024-04-20T11:22:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.522+0000 7ff9a7200640 1 -- 172.21.15.44:0/3119984593 wait complete. 2024-04-20T11:22:36.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.523+0000 7ff9a7200640 1 Processor -- start 2024-04-20T11:22:36.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.523+0000 7ff9a7200640 1 -- start start 2024-04-20T11:22:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.523+0000 7ff9a7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.523+0000 7ff9a7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a8103930 0x7ff9a8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a819d980 0x7ff9a819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a810dba0 con 0x7ff9a8103930 2024-04-20T11:22:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a7200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a810da20 con 0x7ff9a819d980 2024-04-20T11:22:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a810dd20 con 0x7ff9a8101e20 2024-04-20T11:22:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48350/0 (socket says 172.21.15.44:48350) 2024-04-20T11:22:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a819d980 0x7ff9a819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 -- 172.21.15.44:0/2894952609 learned_addr learned my addr 172.21.15.44:0/2894952609 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5400640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a8103930 0x7ff9a8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 -- 172.21.15.44:0/2894952609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a819d980 msgr2=0x7ff9a819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a819d980 0x7ff9a819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 -- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a8103930 msgr2=0x7ff9a8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a8103930 0x7ff9a8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a5e00640 1 -- 172.21.15.44:0/2894952609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a81a03e0 con 0x7ff9a8101e20 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.524+0000 7ff9a6800640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a819d980 0x7ff9a819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.525+0000 7ff9a5e00640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a81983e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff99800e9f0 tx=0x7ff99800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.525+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99800cd80 con 0x7ff9a8101e20 2024-04-20T11:22:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.525+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a81a06d0 con 0x7ff9a8101e20 2024-04-20T11:22:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.525+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff998010930 con 0x7ff9a8101e20 2024-04-20T11:22:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.525+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998018b00 con 0x7ff9a8101e20 2024-04-20T11:22:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.525+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a81a0c40 con 0x7ff9a8101e20 2024-04-20T11:22:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.527+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a81091b0 con 0x7ff9a8101e20 2024-04-20T11:22:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.530+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7ff998018ca0 con 0x7ff9a8101e20 2024-04-20T11:22:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.531+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff9980d0290 con 0x7ff9a8101e20 2024-04-20T11:22:36.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:36.535+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff99809c950 con 0x7ff9a8101e20 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/897045520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: Active manager daemon x restarted 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: Activating manager daemon x 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: mgrmap e22: x(active, starting, since 0.0157185s), standbys: z, y 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:22:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T11:22:36.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:22:36.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:22:36.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:22:36.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:22:36.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: Manager daemon x is now available 2024-04-20T11:22:36.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:22:36.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:22:36.691 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:36 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T11:22:36.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.415+0000 7f61a8600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.415+0000 7f61a8600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.694 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.415+0000 7f61a8600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.415+0000 7f61a8600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.415+0000 7f61a8600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.418+0000 7f61acc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.418+0000 7f61acc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.418+0000 7f61acc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.418+0000 7f61acc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:36.418+0000 7f61acc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/897045520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: Active manager daemon x restarted 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: Activating manager daemon x 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: mgrmap e22: x(active, starting, since 0.0157185s), standbys: z, y 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T11:22:36.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T11:22:36.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T11:22:36.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T11:22:36.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: Manager daemon x is now available 2024-04-20T11:22:36.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:22:36.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T11:22:36.733 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:36 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T11:22:37.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.117+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7ff9980986a0 con 0x7ff9a8101e20 2024-04-20T11:22:37.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.118+0000 7ff99ea00640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff98c0aa7a0 0x7ff98c0acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.118+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ff99801ee50 con 0x7ff98c0aa7a0 2024-04-20T11:22:37.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.129+0000 7ff9a5400640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff98c0aa7a0 0x7ff98c0acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.135+0000 7ff9a5400640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff98c0aa7a0 0x7ff98c0acb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff994009230 tx=0x7ff994005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:37.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.170+0000 7ff99ea00640 1 -- 172.21.15.44:0/2894952609 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7ff99801ee50 con 0x7ff98c0aa7a0 2024-04-20T11:22:37.170 INFO:teuthology.orchestra.run.smithi044.stdout:{} 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff98c0aa7a0 msgr2=0x7ff98c0acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff98c0aa7a0 0x7ff98c0acb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff994009230 tx=0x7ff994005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 msgr2=0x7ff9a81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a81983e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff99800e9f0 tx=0x7ff99800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 shutdown_connections 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a819d980 0x7ff9a819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff98c0aa7a0 0x7ff98c0acb90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9a8103930 0x7ff9a8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 --2- 172.21.15.44:0/2894952609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a8101e20 0x7ff9a81983e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 >> 172.21.15.44:0/2894952609 conn(0x7ff9a80fd390 msgr2=0x7ff9a8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 shutdown_connections 2024-04-20T11:22:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.172+0000 7ff9a7200640 1 -- 172.21.15.44:0/2894952609 wait complete. 2024-04-20T11:22:37.191 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-20T11:22:37.202 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-20T11:22:37.227 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:37.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.407+0000 7fe6cca00640 1 -- 172.21.15.44:0/4086133232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81083e0 msgr2=0x7fe6c8108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.407+0000 7fe6cca00640 1 --2- 172.21.15.44:0/4086133232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81083e0 0x7fe6c8108800 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009900 tx=0x7fe6b802bc50 comp rx=0 tx=0).stop 2024-04-20T11:22:37.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 -- 172.21.15.44:0/4086133232 shutdown_connections 2024-04-20T11:22:37.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 --2- 172.21.15.44:0/4086133232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c8110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 --2- 172.21.15.44:0/4086133232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c81095d0 0x7fe6c8109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 --2- 172.21.15.44:0/4086133232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81083e0 0x7fe6c8108800 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 -- 172.21.15.44:0/4086133232 >> 172.21.15.44:0/4086133232 conn(0x7fe6c80773a0 msgr2=0x7fe6c80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:37.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 -- 172.21.15.44:0/4086133232 shutdown_connections 2024-04-20T11:22:37.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.408+0000 7fe6cca00640 1 -- 172.21.15.44:0/4086133232 wait complete. 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.409+0000 7fe6cca00640 1 Processor -- start 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.409+0000 7fe6cca00640 1 -- start start 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c81083e0 0x7fe6c819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6cca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81095d0 0x7fe6c819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6cca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c81a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6cca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8114180 con 0x7fe6c81083e0 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6cca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8114000 con 0x7fe6c8109f90 2024-04-20T11:22:37.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6cca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8114300 con 0x7fe6c81095d0 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81095d0 0x7fe6c819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c81a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c81083e0 0x7fe6c819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81095d0 0x7fe6c819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48396/0 (socket says 172.21.15.44:48396) 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c81a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59508/0 (socket says 172.21.15.44:59508) 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c5e00640 1 -- 172.21.15.44:0/2116417971 learned_addr learned my addr 172.21.15.44:0/2116417971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 -- 172.21.15.44:0/2116417971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81095d0 msgr2=0x7fe6c819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81095d0 0x7fe6c819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 -- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c81083e0 msgr2=0x7fe6c819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c81083e0 0x7fe6c819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.410+0000 7fe6c7200640 1 -- 172.21.15.44:0/2116417971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c81a63e0 con 0x7fe6c8109f90 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.411+0000 7fe6c7200640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c81a5d70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009fd0 tx=0x7fe6c000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.411+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000ce20 con 0x7fe6c8109f90 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.411+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6c0004b20 con 0x7fe6c8109f90 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.411+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c81a66d0 con 0x7fe6c8109f90 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.411+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0010790 con 0x7fe6c8109f90 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.411+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c81a6bd0 con 0x7fe6c8109f90 2024-04-20T11:22:37.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.412+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c810f700 con 0x7fe6c8109f90 2024-04-20T11:22:37.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.414+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fe6c0005020 con 0x7fe6c8109f90 2024-04-20T11:22:37.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.415+0000 7fe6bf400640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe6ac0a39b0 0x7fe6ac0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.415+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe6c00d0870 con 0x7fe6c8109f90 2024-04-20T11:22:37.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.415+0000 7fe6c6800640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe6ac0a39b0 0x7fe6ac0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.416+0000 7fe6c6800640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe6ac0a39b0 0x7fe6ac0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8000c00 tx=0x7fe6b8005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:37.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.419+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe6c009d450 con 0x7fe6c8109f90 2024-04-20T11:22:37.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:37.151+0000 7f61a5400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:37.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:37.151+0000 7f61a5400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:37.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:37.151+0000 7f61a5400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:37.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:37.151+0000 7f61a5400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:37.445 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:22:37.151+0000 7f61a5400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:22:37.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-20T11:22:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:22:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: mgrmap e23: x(active, since 1.02772s), standbys: z, y 2024-04-20T11:22:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.446 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:37 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: mgrmap e23: x(active, since 1.02772s), standbys: z, y 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.589 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:37 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.631+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6c8062b60 con 0x7fe6c8109f90 2024-04-20T11:22:37.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.632+0000 7fe6bf400640 1 -- 172.21.15.44:0/2116417971 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22887 (secure 0 0 0) 0x7fe6c009cd50 con 0x7fe6c8109f90 2024-04-20T11:22:37.633 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:37.633 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:22:36.090257+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:37.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe6ac0a39b0 msgr2=0x7fe6ac0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe6ac0a39b0 0x7fe6ac0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8000c00 tx=0x7fe6b8005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 msgr2=0x7fe6c81a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c81a5d70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009fd0 tx=0x7fe6c000ef10 comp rx=0 tx=0).stop 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 shutdown_connections 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6c8109f90 0x7fe6c81a5d70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6c81095d0 0x7fe6c819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe6ac0a39b0 0x7fe6ac0a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 --2- 172.21.15.44:0/2116417971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c81083e0 0x7fe6c819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 >> 172.21.15.44:0/2116417971 conn(0x7fe6c80773a0 msgr2=0x7fe6c810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 shutdown_connections 2024-04-20T11:22:37.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.635+0000 7fe6cca00640 1 -- 172.21.15.44:0/2116417971 wait complete. 2024-04-20T11:22:37.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:22:37.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:22:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-20T11:22:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:22:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: mgrmap e23: x(active, since 1.02772s), standbys: z, y 2024-04-20T11:22:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:37 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 -- 172.21.15.44:0/230193008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 msgr2=0x7f4628103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 --2- 172.21.15.44:0/230193008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f4628103410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f461402bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 -- 172.21.15.44:0/230193008 shutdown_connections 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 --2- 172.21.15.44:0/230193008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628103950 0x7f462810a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 --2- 172.21.15.44:0/230193008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f4628103410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 --2- 172.21.15.44:0/230193008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4628101da0 0x7f46281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 -- 172.21.15.44:0/230193008 >> 172.21.15.44:0/230193008 conn(0x7f46280fd310 msgr2=0x7f46280ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 -- 172.21.15.44:0/230193008 shutdown_connections 2024-04-20T11:22:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.864+0000 7f4627e00640 1 -- 172.21.15.44:0/230193008 wait complete. 2024-04-20T11:22:37.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 Processor -- start 2024-04-20T11:22:37.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 -- start start 2024-04-20T11:22:37.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 0x7f4628198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f46281986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4628103950 0x7f462819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f462810db00 con 0x7f4628102f90 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f462810d980 con 0x7f4628101da0 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.866+0000 7f4627e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f462810dc80 con 0x7f4628103950 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f46281986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 0x7f4628198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4627400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4628103950 0x7f462819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 0x7f4628198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59522/0 (socket says 172.21.15.44:59522) 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f46281986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34404/0 (socket says 172.21.15.44:34404) 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 -- 172.21.15.44:0/951876272 learned_addr learned my addr 172.21.15.44:0/951876272 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 -- 172.21.15.44:0/951876272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4628103950 msgr2=0x7f462819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4628103950 0x7f462819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 -- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 msgr2=0x7f46281986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f46281986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 -- 172.21.15.44:0/951876272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462819fd90 con 0x7f4628101da0 2024-04-20T11:22:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.867+0000 7f4626a00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 0x7f4628198170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f461800e9c0 tx=0x7f461800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.868+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800ccc0 con 0x7f4628101da0 2024-04-20T11:22:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.868+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f46180035f0 con 0x7f4628101da0 2024-04-20T11:22:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.868+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800ce60 con 0x7f4628101da0 2024-04-20T11:22:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.868+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46281a0080 con 0x7f4628101da0 2024-04-20T11:22:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.868+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46281a05a0 con 0x7f4628101da0 2024-04-20T11:22:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.869+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46281090c0 con 0x7f4628101da0 2024-04-20T11:22:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.873+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f46180037a0 con 0x7f4628101da0 2024-04-20T11:22:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.874+0000 7f461f400640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f460c0a3a30 0x7f460c0a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.874+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f46180d0540 con 0x7f4628101da0 2024-04-20T11:22:37.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.874+0000 7f4626000640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f460c0a3a30 0x7f460c0a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:37.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.875+0000 7f4626000640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f460c0a3a30 0x7f460c0a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4614000c00 tx=0x7f4614005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:37.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:37.878+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f461809d160 con 0x7f4628101da0 2024-04-20T11:22:38.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.077+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4628062b60 con 0x7f4628101da0 2024-04-20T11:22:38.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.078+0000 7f461f400640 1 -- 172.21.15.44:0/951876272 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f461809ca60 con 0x7f4628101da0 2024-04-20T11:22:38.079 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f460c0a3a30 msgr2=0x7f460c0a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f460c0a3a30 0x7f460c0a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4614000c00 tx=0x7f4614005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 msgr2=0x7f4628198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 0x7f4628198170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f461800e9c0 tx=0x7f461800edc0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 shutdown_connections 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4628103950 0x7f462819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f460c0a3a30 0x7f460c0a5ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.080+0000 7f4627e00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4628102f90 0x7f46281986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.081+0000 7f4627e00640 1 --2- 172.21.15.44:0/951876272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4628101da0 0x7f4628198170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.081+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 >> 172.21.15.44:0/951876272 conn(0x7f46280fd310 msgr2=0x7f4628106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.081+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 shutdown_connections 2024-04-20T11:22:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.081+0000 7f4627e00640 1 -- 172.21.15.44:0/951876272 wait complete. 2024-04-20T11:22:38.098 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 -- 172.21.15.44:0/562386947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 msgr2=0x7fc68c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 --2- 172.21.15.44:0/562386947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c1045e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009900 tx=0x7fc67c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 -- 172.21.15.44:0/562386947 shutdown_connections 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 --2- 172.21.15.44:0/562386947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc68c105d70 0x7fc68c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 --2- 172.21.15.44:0/562386947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c1053b0 0x7fc68c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 --2- 172.21.15.44:0/562386947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c1045e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 -- 172.21.15.44:0/562386947 >> 172.21.15.44:0/562386947 conn(0x7fc68c0ff710 msgr2=0x7fc68c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:38.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 -- 172.21.15.44:0/562386947 shutdown_connections 2024-04-20T11:22:38.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.295+0000 7fc692600640 1 -- 172.21.15.44:0/562386947 wait complete. 2024-04-20T11:22:38.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.296+0000 7fc692600640 1 Processor -- start 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.296+0000 7fc692600640 1 -- start start 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.296+0000 7fc692600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc692600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c1053b0 0x7fc68c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc692600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc68c105d70 0x7fc68c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc692600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10ff10 con 0x7fc68c105d70 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc692600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10fd90 con 0x7fc68c1041c0 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc692600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c110090 con 0x7fc68c1053b0 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59566/0 (socket says 172.21.15.44:59566) 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 -- 172.21.15.44:0/3667347091 learned_addr learned my addr 172.21.15.44:0/3667347091 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68be00640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc68c105d70 0x7fc68c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68aa00640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c1053b0 0x7fc68c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 -- 172.21.15.44:0/3667347091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c1053b0 msgr2=0x7fc68c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c1053b0 0x7fc68c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 -- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc68c105d70 msgr2=0x7fc68c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc68c105d70 0x7fc68c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.297+0000 7fc68b400640 1 -- 172.21.15.44:0/3667347091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1a21b0 con 0x7fc68c1041c0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.298+0000 7fc68b400640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c19a590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000c00 tx=0x7fc67c004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.298+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c043020 con 0x7fc68c1041c0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.298+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc67c004250 con 0x7fc68c1041c0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.298+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1a2440 con 0x7fc68c1041c0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.298+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c003c00 con 0x7fc68c1041c0 2024-04-20T11:22:38.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.298+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a2990 con 0x7fc68c1041c0 2024-04-20T11:22:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.299+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c10b4e0 con 0x7fc68c1041c0 2024-04-20T11:22:38.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.301+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fc67c046020 con 0x7fc68c1041c0 2024-04-20T11:22:38.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.303+0000 7fc683e00640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6700a39b0 0x7fc6700a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.303+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc67c0f2130 con 0x7fc68c1041c0 2024-04-20T11:22:38.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.303+0000 7fc68aa00640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6700a39b0 0x7fc6700a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.304+0000 7fc68aa00640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6700a39b0 0x7fc6700a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6780095f0 tx=0x7fc678005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.307+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc67c04d020 con 0x7fc68c1041c0 2024-04-20T11:22:38.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.453+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7fc67c0bacc0 con 0x7fc68c1041c0 2024-04-20T11:22:38.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.526+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc68c062b60 con 0x7fc68c1041c0 2024-04-20T11:22:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.527+0000 7fc683e00640 1 -- 172.21.15.44:0/3667347091 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fc67c0bed50 con 0x7fc68c1041c0 2024-04-20T11:22:38.527 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:22:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6700a39b0 msgr2=0x7fc6700a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6700a39b0 0x7fc6700a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6780095f0 tx=0x7fc678005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 msgr2=0x7fc68c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c19a590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000c00 tx=0x7fc67c004a30 comp rx=0 tx=0).stop 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 shutdown_connections 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6700a39b0 0x7fc6700a5e70 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6780095f0 tx=0x7fc678005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc68c105d70 0x7fc68c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c1053b0 0x7fc68c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 --2- 172.21.15.44:0/3667347091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc68c1041c0 0x7fc68c19a590 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.529+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 >> 172.21.15.44:0/3667347091 conn(0x7fc68c0ff710 msgr2=0x7fc68c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.530+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 shutdown_connections 2024-04-20T11:22:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.530+0000 7fc692600640 1 -- 172.21.15.44:0/3667347091 wait complete. 2024-04-20T11:22:38.550 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: [20/Apr/2024:11:22:36] ENGINE Bus STARTING 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: [20/Apr/2024:11:22:37] ENGINE Serving on http://172.21.15.44:8765 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: from='client.25354 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: [20/Apr/2024:11:22:37] ENGINE Serving on https://172.21.15.44:7150 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: [20/Apr/2024:11:22:37] ENGINE Bus STARTED 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2116417971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:38.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:38 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/951876272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: [20/Apr/2024:11:22:36] ENGINE Bus STARTING 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: [20/Apr/2024:11:22:37] ENGINE Serving on http://172.21.15.44:8765 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: from='client.25354 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: [20/Apr/2024:11:22:37] ENGINE Serving on https://172.21.15.44:7150 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: [20/Apr/2024:11:22:37] ENGINE Bus STARTED 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:38.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:38.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2116417971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:38.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:38 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/951876272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:38.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: [20/Apr/2024:11:22:36] ENGINE Bus STARTING 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: [20/Apr/2024:11:22:37] ENGINE Serving on http://172.21.15.44:8765 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: from='client.25354 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: [20/Apr/2024:11:22:37] ENGINE Serving on https://172.21.15.44:7150 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: [20/Apr/2024:11:22:37] ENGINE Bus STARTED 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2116417971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:38.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:38 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/951876272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 -- 172.21.15.44:0/4101469944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c075fa0 msgr2=0x7fb48c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 --2- 172.21.15.44:0/4101469944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c075fa0 0x7fb48c0743f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009900 tx=0x7fb47c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 -- 172.21.15.44:0/4101469944 shutdown_connections 2024-04-20T11:22:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 --2- 172.21.15.44:0/4101469944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 0x7fb48c10c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 --2- 172.21.15.44:0/4101469944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb48c0749c0 0x7fb48c074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 --2- 172.21.15.44:0/4101469944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c075fa0 0x7fb48c0743f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 -- 172.21.15.44:0/4101469944 >> 172.21.15.44:0/4101469944 conn(0x7fb48c0fd310 msgr2=0x7fb48c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 -- 172.21.15.44:0/4101469944 shutdown_connections 2024-04-20T11:22:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.768+0000 7fb491200640 1 -- 172.21.15.44:0/4101469944 wait complete. 2024-04-20T11:22:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.769+0000 7fb491200640 1 Processor -- start 2024-04-20T11:22:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 -- start start 2024-04-20T11:22:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 0x7fb48c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb48c075fa0 0x7fb48c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 0x7fb48c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c10fcd0 con 0x7fb48c075fa0 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c10fb50 con 0x7fb48c109c60 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb491200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c10fe50 con 0x7fb48c0749c0 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb48b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 0x7fb48c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb48b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 0x7fb48c19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48410/0 (socket says 172.21.15.44:48410) 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb48b400640 1 -- 172.21.15.44:0/621902805 learned_addr learned my addr 172.21.15.44:0/621902805 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb48aa00640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb48c075fa0 0x7fb48c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.770+0000 7fb48be00640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 0x7fb48c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48b400640 1 -- 172.21.15.44:0/621902805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 msgr2=0x7fb48c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48b400640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 0x7fb48c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48b400640 1 -- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb48c075fa0 msgr2=0x7fb48c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48b400640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb48c075fa0 0x7fb48c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48b400640 1 -- 172.21.15.44:0/621902805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c1a1f60 con 0x7fb48c0749c0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48be00640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 0x7fb48c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb48b400640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 0x7fb48c19a340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005ab0 tx=0x7fb47c004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.771+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c043020 con 0x7fb48c0749c0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.772+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c1a21f0 con 0x7fb48c0749c0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.772+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb47c004250 con 0x7fb48c0749c0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.772+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c003b30 con 0x7fb48c0749c0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.772+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c1a2790 con 0x7fb48c0749c0 2024-04-20T11:22:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.773+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c108990 con 0x7fb48c0749c0 2024-04-20T11:22:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.775+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7fb47c046020 con 0x7fb48c0749c0 2024-04-20T11:22:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.776+0000 7fb483e00640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb4700a3db0 0x7fb4700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.776+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fb47c0f28d0 con 0x7fb48c0749c0 2024-04-20T11:22:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.776+0000 7fb48aa00640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb4700a3db0 0x7fb4700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.777+0000 7fb48aa00640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb4700a3db0 0x7fb4700a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb48c19b850 tx=0x7fb478005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.780+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb47c04d020 con 0x7fb48c0749c0 2024-04-20T11:22:38.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:38.997+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb48c062b60 con 0x7fb48c0749c0 2024-04-20T11:22:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.001+0000 7fb483e00640 1 -- 172.21.15.44:0/621902805 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fb47c0bef20 con 0x7fb48c0749c0 2024-04-20T11:22:39.002 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:22:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.004+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb4700a3db0 msgr2=0x7fb4700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.004+0000 7fb491200640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb4700a3db0 0x7fb4700a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb48c19b850 tx=0x7fb478005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.004+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 msgr2=0x7fb48c19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.004+0000 7fb491200640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 0x7fb48c19a340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005ab0 tx=0x7fb47c004a50 comp rx=0 tx=0).stop 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 shutdown_connections 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb48c109c60 0x7fb48c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb4700a3db0 0x7fb4700a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb48c075fa0 0x7fb48c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 --2- 172.21.15.44:0/621902805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb48c0749c0 0x7fb48c19a340 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 >> 172.21.15.44:0/621902805 conn(0x7fb48c0fd310 msgr2=0x7fb48c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 shutdown_connections 2024-04-20T11:22:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.005+0000 7fb491200640 1 -- 172.21.15.44:0/621902805 wait complete. 2024-04-20T11:22:39.028 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 -- 172.21.15.44:0/1475430764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 msgr2=0x7f17dc10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 --2- 172.21.15.44:0/1475430764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 0x7f17dc10c620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009ca0 tx=0x7f17d402c100 comp rx=0 tx=0).stop 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 -- 172.21.15.44:0/1475430764 shutdown_connections 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 --2- 172.21.15.44:0/1475430764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 0x7f17dc10c620 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 --2- 172.21.15.44:0/1475430764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1053d0 0x7f17dc105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 --2- 172.21.15.44:0/1475430764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17dc1041e0 0x7f17dc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 -- 172.21.15.44:0/1475430764 >> 172.21.15.44:0/1475430764 conn(0x7f17dc0ff750 msgr2=0x7f17dc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 -- 172.21.15.44:0/1475430764 shutdown_connections 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.239+0000 7f17e3800640 1 -- 172.21.15.44:0/1475430764 wait complete. 2024-04-20T11:22:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.240+0000 7f17e3800640 1 Processor -- start 2024-04-20T11:22:39.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 -- start start 2024-04-20T11:22:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 0x7f17dc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17dc1053d0 0x7f17dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 0x7f17dc1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10ff90 con 0x7f17dc1053d0 2024-04-20T11:22:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10fe10 con 0x7f17dc105d90 2024-04-20T11:22:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e3800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc110110 con 0x7f17dc1041e0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 0x7f17dc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17dc1053d0 0x7f17dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 0x7f17dc19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48432/0 (socket says 172.21.15.44:48432) 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e0e00640 1 -- 172.21.15.44:0/2786302891 learned_addr learned my addr 172.21.15.44:0/2786302891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.241+0000 7f17e1800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 0x7f17dc1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e0e00640 1 -- 172.21.15.44:0/2786302891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 msgr2=0x7f17dc1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e0e00640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 0x7f17dc1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e0e00640 1 -- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17dc1053d0 msgr2=0x7f17dc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e0e00640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17dc1053d0 0x7f17dc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e0e00640 1 -- 172.21.15.44:0/2786302891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc1a2140 con 0x7f17dc1041e0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e0e00640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 0x7f17dc19a520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9c0 tx=0x7f17d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000cd80 con 0x7f17dc1041e0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17d00035f0 con 0x7f17dc1041e0 2024-04-20T11:22:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0010690 con 0x7f17dc1041e0 2024-04-20T11:22:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.242+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1a2430 con 0x7f17dc1041e0 2024-04-20T11:22:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.243+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a29d0 con 0x7f17dc1041e0 2024-04-20T11:22:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.244+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17dc10b4a0 con 0x7f17dc1041e0 2024-04-20T11:22:39.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.246+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f17d00037a0 con 0x7f17dc1041e0 2024-04-20T11:22:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.248+0000 7f17d9600640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17c40a3e30 0x7f17c40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.248+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17d00d0cd0 con 0x7f17dc1041e0 2024-04-20T11:22:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.248+0000 7f17dbe00640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17c40a3e30 0x7f17c40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.249+0000 7f17dbe00640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17c40a3e30 0x7f17c40a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17c80095f0 tx=0x7f17c8005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:39.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.252+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17d009d270 con 0x7f17dc1041e0 2024-04-20T11:22:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.495+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17dc062b60 con 0x7f17dc1041e0 2024-04-20T11:22:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.496+0000 7f17d9600640 1 -- 172.21.15.44:0/2786302891 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22887 (secure 0 0 0) 0x7f17d009cb70 con 0x7f17dc1041e0 2024-04-20T11:22:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:39.498 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:22:36.090257+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17c40a3e30 msgr2=0x7f17c40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17c40a3e30 0x7f17c40a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17c80095f0 tx=0x7f17c8005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 msgr2=0x7f17dc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 0x7f17dc19a520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9c0 tx=0x7f17d000ee80 comp rx=0 tx=0).stop 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 shutdown_connections 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17dc105d90 0x7f17dc1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17c40a3e30 0x7f17c40a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17dc1053d0 0x7f17dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 --2- 172.21.15.44:0/2786302891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17dc1041e0 0x7f17dc19a520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 >> 172.21.15.44:0/2786302891 conn(0x7f17dc0ff750 msgr2=0x7f17dc108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.499+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 shutdown_connections 2024-04-20T11:22:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.500+0000 7f17e3800640 1 -- 172.21.15.44:0/2786302891 wait complete. 2024-04-20T11:22:39.518 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:22:39.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:22:39.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 -- 172.21.15.44:0/668090675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1053b0 msgr2=0x7f504c105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 --2- 172.21.15.44:0/668090675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1053b0 0x7f504c105830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5038009900 tx=0x7f503802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:39.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 -- 172.21.15.44:0/668090675 shutdown_connections 2024-04-20T11:22:39.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 --2- 172.21.15.44:0/668090675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c105d70 0x7f504c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 --2- 172.21.15.44:0/668090675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1053b0 0x7f504c105830 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 --2- 172.21.15.44:0/668090675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c1041c0 0x7f504c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 -- 172.21.15.44:0/668090675 >> 172.21.15.44:0/668090675 conn(0x7f504c0ff750 msgr2=0x7f504c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:39.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 -- 172.21.15.44:0/668090675 shutdown_connections 2024-04-20T11:22:39.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.701+0000 7f5054000640 1 -- 172.21.15.44:0/668090675 wait complete. 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.702+0000 7f5054000640 1 Processor -- start 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.702+0000 7f5054000640 1 -- start start 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5054000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 0x7f504c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5054000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 0x7f504c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5054000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c105d70 0x7f504c1a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5054000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10ff20 con 0x7f504c105d70 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5054000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10fda0 con 0x7f504c1053b0 2024-04-20T11:22:39.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5054000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1100a0 con 0x7f504c1041c0 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 0x7f504c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5051600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 0x7f504c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5051600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 0x7f504c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48444/0 (socket says 172.21.15.44:48444) 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5051600640 1 -- 172.21.15.44:0/4114585215 learned_addr learned my addr 172.21.15.44:0/4114585215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 0x7f504c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36706/0 (socket says 172.21.15.44:36706) 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5052000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c105d70 0x7f504c1a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 -- 172.21.15.44:0/4114585215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 msgr2=0x7f504c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 0x7f504c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 -- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c105d70 msgr2=0x7f504c1a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c105d70 0x7f504c1a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.703+0000 7f5050c00640 1 -- 172.21.15.44:0/4114585215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c1a21d0 con 0x7f504c1053b0 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f5051600640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 0x7f504c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f5052000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c105d70 0x7f504c1a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f5050c00640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 0x7f504c19aaf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5038000c00 tx=0x7f5038004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038043020 con 0x7f504c1053b0 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5038004250 con 0x7f504c1053b0 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c1a2460 con 0x7f504c1053b0 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003b90 con 0x7f504c1053b0 2024-04-20T11:22:39.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.704+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c1a2a00 con 0x7f504c1053b0 2024-04-20T11:22:39.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.706+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c10b480 con 0x7f504c1053b0 2024-04-20T11:22:39.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.710+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f5038046020 con 0x7f504c1053b0 2024-04-20T11:22:39.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.711+0000 7f504a000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f50340a3e30 0x7f50340a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:39.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.711+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f50380f2e90 con 0x7f504c1053b0 2024-04-20T11:22:39.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.711+0000 7f5051600640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f50340a3e30 0x7f50340a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:39.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.711+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f503804d020 con 0x7f504c1053b0 2024-04-20T11:22:39.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.712+0000 7f5051600640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f50340a3e30 0x7f50340a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5040005c40 tx=0x7f5040009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-20T11:22:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3667347091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/621902805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:39.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:39 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3667347091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/621902805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:39.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:39 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.938+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f504c02d760 con 0x7f504c1053b0 2024-04-20T11:22:39.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.939+0000 7f504a000640 1 -- 172.21.15.44:0/4114585215 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f50380bf4e0 con 0x7f504c1053b0 2024-04-20T11:22:39.939 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f50340a3e30 msgr2=0x7f50340a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f50340a3e30 0x7f50340a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5040005c40 tx=0x7f5040009500 comp rx=0 tx=0).stop 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 msgr2=0x7f504c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 0x7f504c19aaf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5038000c00 tx=0x7f5038004a30 comp rx=0 tx=0).stop 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 shutdown_connections 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f50340a3e30 0x7f50340a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f504c105d70 0x7f504c1a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f504c1053b0 0x7f504c19aaf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 --2- 172.21.15.44:0/4114585215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f504c1041c0 0x7f504c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 >> 172.21.15.44:0/4114585215 conn(0x7f504c0ff750 msgr2=0x7f504c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 shutdown_connections 2024-04-20T11:22:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:39.941+0000 7f5054000640 1 -- 172.21.15.44:0/4114585215 wait complete. 2024-04-20T11:22:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-20T11:22:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3667347091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/621902805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:39 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:39.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 -- 172.21.15.44:0/654553795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49881011d0 msgr2=0x7f4988101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 --2- 172.21.15.44:0/654553795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49881011d0 0x7f4988101630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4974009860 tx=0x7f497402bb70 comp rx=0 tx=0).stop 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 -- 172.21.15.44:0/654553795 shutdown_connections 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 --2- 172.21.15.44:0/654553795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4988101b70 0x7f4988101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 --2- 172.21.15.44:0/654553795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49881011d0 0x7f4988101630 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 --2- 172.21.15.44:0/654553795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4988100870 0x7f4988100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 -- 172.21.15.44:0/654553795 >> 172.21.15.44:0/654553795 conn(0x7f49880fc410 msgr2=0x7f49880fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 -- 172.21.15.44:0/654553795 shutdown_connections 2024-04-20T11:22:40.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.156+0000 7f498fe00640 1 -- 172.21.15.44:0/654553795 wait complete. 2024-04-20T11:22:40.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.157+0000 7f498fe00640 1 Processor -- start 2024-04-20T11:22:40.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.157+0000 7f498fe00640 1 -- start start 2024-04-20T11:22:40.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.157+0000 7f498fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4988100870 0x7f498819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.157+0000 7f498fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 0x7f498819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.157+0000 7f498fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4988101b70 0x7f49881a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f498810ff20 con 0x7f4988100870 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f498810fda0 con 0x7f4988101b70 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49881100a0 con 0x7f49881011d0 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 0x7f498819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 0x7f498819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48464/0 (socket says 172.21.15.44:48464) 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 -- 172.21.15.44:0/2380040627 learned_addr learned my addr 172.21.15.44:0/2380040627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498d400640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4988100870 0x7f498819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498de00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4988101b70 0x7f49881a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 -- 172.21.15.44:0/2380040627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4988101b70 msgr2=0x7f49881a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4988101b70 0x7f49881a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 -- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4988100870 msgr2=0x7f498819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4988100870 0x7f498819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.158+0000 7f498ca00640 1 -- 172.21.15.44:0/2380040627 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881a21c0 con 0x7f49881011d0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.159+0000 7f498ca00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 0x7f498819aae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4974034bb0 tx=0x7f49740349a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.159+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974041020 con 0x7f49881011d0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.159+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f497402ca10 con 0x7f49881011d0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.159+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49881a2450 con 0x7f49881011d0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.159+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974039040 con 0x7f49881011d0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.159+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a2990 con 0x7f49881011d0 2024-04-20T11:22:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.161+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498810b770 con 0x7f49881011d0 2024-04-20T11:22:40.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.162+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f4974044020 con 0x7f49881011d0 2024-04-20T11:22:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.164+0000 7f4986000640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f49700a3e30 0x7f49700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.165+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f49740f2140 con 0x7f49881011d0 2024-04-20T11:22:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.165+0000 7f498d400640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f49700a3e30 0x7f49700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.165+0000 7f498d400640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f49700a3e30 0x7f49700a62f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f497c009530 tx=0x7f497c009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.167+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f497404b020 con 0x7f49881011d0 2024-04-20T11:22:40.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.386+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f498802d780 con 0x7f49881011d0 2024-04-20T11:22:40.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.387+0000 7f4986000640 1 -- 172.21.15.44:0/2380040627 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f49740be790 con 0x7f49881011d0 2024-04-20T11:22:40.388 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:22:40.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f49700a3e30 msgr2=0x7f49700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f49700a3e30 0x7f49700a62f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f497c009530 tx=0x7f497c009260 comp rx=0 tx=0).stop 2024-04-20T11:22:40.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 msgr2=0x7f498819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 0x7f498819aae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4974034bb0 tx=0x7f49740349a0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 shutdown_connections 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4988101b70 0x7f49881a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49881011d0 0x7f498819aae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f49700a3e30 0x7f49700a62f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 --2- 172.21.15.44:0/2380040627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4988100870 0x7f498819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 >> 172.21.15.44:0/2380040627 conn(0x7f49880fc410 msgr2=0x7f49880fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 shutdown_connections 2024-04-20T11:22:40.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.390+0000 7f498fe00640 1 -- 172.21.15.44:0/2380040627 wait complete. 2024-04-20T11:22:40.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:22:40.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 -- 172.21.15.44:0/3907878666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 msgr2=0x7f9cc0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 --2- 172.21.15.44:0/3907878666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc0105810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009900 tx=0x7f9cac02bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:40.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 -- 172.21.15.44:0/3907878666 shutdown_connections 2024-04-20T11:22:40.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 --2- 172.21.15.44:0/3907878666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc0105d50 0x7f9cc010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 --2- 172.21.15.44:0/3907878666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc0105810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 --2- 172.21.15.44:0/3907878666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc01041a0 0x7f9cc01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 -- 172.21.15.44:0/3907878666 >> 172.21.15.44:0/3907878666 conn(0x7f9cc00ff750 msgr2=0x7f9cc0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:40.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 -- 172.21.15.44:0/3907878666 shutdown_connections 2024-04-20T11:22:40.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.598+0000 7f9cc5200640 1 -- 172.21.15.44:0/3907878666 wait complete. 2024-04-20T11:22:40.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.600+0000 7f9cc5200640 1 Processor -- start 2024-04-20T11:22:40.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.600+0000 7f9cc5200640 1 -- start start 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.600+0000 7f9cc5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc01041a0 0x7f9cc019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.600+0000 7f9cc5200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cc5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc0105d50 0x7f9cc01a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cc5200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc010ff00 con 0x7f9cc01041a0 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cc5200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc010fd80 con 0x7f9cc0105390 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cc5200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0110080 con 0x7f9cc0105d50 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc01041a0 0x7f9cc019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36750/0 (socket says 172.21.15.44:36750) 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 -- 172.21.15.44:0/2504682206 learned_addr learned my addr 172.21.15.44:0/2504682206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbe800640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc0105d50 0x7f9cc01a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 -- 172.21.15.44:0/2504682206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc0105d50 msgr2=0x7f9cc01a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc0105d50 0x7f9cc01a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 -- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc01041a0 msgr2=0x7f9cc019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc01041a0 0x7f9cc019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.601+0000 7f9cbd400640 1 -- 172.21.15.44:0/2504682206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc01a2170 con 0x7f9cc0105390 2024-04-20T11:22:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.602+0000 7f9cbd400640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc019aa90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009230 tx=0x7f9cac004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.602+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac043020 con 0x7f9cc0105390 2024-04-20T11:22:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.602+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9cac004250 con 0x7f9cc0105390 2024-04-20T11:22:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.602+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac003b50 con 0x7f9cc0105390 2024-04-20T11:22:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.602+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc01a2400 con 0x7f9cc0105390 2024-04-20T11:22:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.602+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01a29a0 con 0x7f9cc0105390 2024-04-20T11:22:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.603+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc010b460 con 0x7f9cc0105390 2024-04-20T11:22:40.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.605+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f9cac046020 con 0x7f9cc0105390 2024-04-20T11:22:40.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.606+0000 7f9cb6a00640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9ca40a3ec0 0x7f9ca40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:40.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.606+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9cac0f29d0 con 0x7f9cc0105390 2024-04-20T11:22:40.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.607+0000 7f9cbde00640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9ca40a3ec0 0x7f9ca40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:40.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.607+0000 7f9cbde00640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9ca40a3ec0 0x7f9ca40a6380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9cc01051f0 tx=0x7f9cb0005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:40.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.610+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9cac04d020 con 0x7f9cc0105390 2024-04-20T11:22:40.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2786302891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4114585215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2380040627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:40.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2786302891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:40.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4114585215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2380040627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:40.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.821+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9cc002d780 con 0x7f9cc0105390 2024-04-20T11:22:40.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.822+0000 7f9cb6a00640 1 -- 172.21.15.44:0/2504682206 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f9cac0bef40 con 0x7f9cc0105390 2024-04-20T11:22:40.823 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9ca40a3ec0 msgr2=0x7f9ca40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9ca40a3ec0 0x7f9ca40a6380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9cc01051f0 tx=0x7f9cb0005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 msgr2=0x7f9cc019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc019aa90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009230 tx=0x7f9cac004ab0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 shutdown_connections 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc0105d50 0x7f9cc01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc0105390 0x7f9cc019aa90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9ca40a3ec0 0x7f9ca40a6380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 --2- 172.21.15.44:0/2504682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc01041a0 0x7f9cc019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:40.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 >> 172.21.15.44:0/2504682206 conn(0x7f9cc00ff750 msgr2=0x7f9cc0108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:40.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.825+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 shutdown_connections 2024-04-20T11:22:40.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:40.826+0000 7f9cc5200640 1 -- 172.21.15.44:0/2504682206 wait complete. 2024-04-20T11:22:40.847 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-20T11:22:40.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2786302891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4114585215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:40.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:40.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2380040627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 -- 172.21.15.44:0/987447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c103fe0 msgr2=0x7fd05c104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 --2- 172.21.15.44:0/987447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c103fe0 0x7fd05c104400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd050009900 tx=0x7fd05002bc40 comp rx=0 tx=0).stop 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 -- 172.21.15.44:0/987447 shutdown_connections 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 --2- 172.21.15.44:0/987447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c105b90 0x7fd05c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 --2- 172.21.15.44:0/987447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c1051d0 0x7fd05c105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 --2- 172.21.15.44:0/987447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c103fe0 0x7fd05c104400 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 -- 172.21.15.44:0/987447 >> 172.21.15.44:0/987447 conn(0x7fd05c0ff570 msgr2=0x7fd05c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 -- 172.21.15.44:0/987447 shutdown_connections 2024-04-20T11:22:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.035+0000 7fd062e00640 1 -- 172.21.15.44:0/987447 wait complete. 2024-04-20T11:22:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.036+0000 7fd062e00640 1 Processor -- start 2024-04-20T11:22:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 -- start start 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c103fe0 0x7fd05c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c1051d0 0x7fd05c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c105b90 0x7fd05c1a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c10fd90 con 0x7fd05c105b90 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c10fc10 con 0x7fd05c1051d0 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.037+0000 7fd062e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c10ff10 con 0x7fd05c103fe0 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c103fe0 0x7fd05c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c1051d0 0x7fd05c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c103fe0 0x7fd05c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48500/0 (socket says 172.21.15.44:48500) 2024-04-20T11:22:41.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061a00640 1 -- 172.21.15.44:0/1449451789 learned_addr learned my addr 172.21.15.44:0/1449451789 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:41.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd062400640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c105b90 0x7fd05c1a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 -- 172.21.15.44:0/1449451789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c103fe0 msgr2=0x7fd05c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c103fe0 0x7fd05c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 -- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c105b90 msgr2=0x7fd05c1a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c105b90 0x7fd05c1a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 -- 172.21.15.44:0/1449451789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c1a2030 con 0x7fd05c1051d0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.038+0000 7fd061000640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c1051d0 0x7fd05c19a950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.039+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04800ccc0 con 0x7fd05c1051d0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.039+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c1a2320 con 0x7fd05c1051d0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.039+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd048005020 con 0x7fd05c1051d0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.039+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480049c0 con 0x7fd05c1051d0 2024-04-20T11:22:41.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.039+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1a2820 con 0x7fd05c1051d0 2024-04-20T11:22:41.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.040+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c10b300 con 0x7fd05c1051d0 2024-04-20T11:22:41.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.042+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fd0480051d0 con 0x7fd05c1051d0 2024-04-20T11:22:41.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.044+0000 7fd05a000640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0440a3e40 0x7fd0440a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.044+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fd0480d0a00 con 0x7fd05c1051d0 2024-04-20T11:22:41.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.045+0000 7fd061a00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0440a3e40 0x7fd0440a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.045+0000 7fd061a00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0440a3e40 0x7fd0440a6300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd05002ca40 tx=0x7fd050009560 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:41.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.046+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd04809cf70 con 0x7fd05c1051d0 2024-04-20T11:22:41.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.296+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd05c02d820 con 0x7fd05c1051d0 2024-04-20T11:22:41.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.297+0000 7fd05a000640 1 -- 172.21.15.44:0/1449451789 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fd04809c870 con 0x7fd05c1051d0 2024-04-20T11:22:41.297 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:41.297 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:22:35.546711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:22:41.298 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-04-20T11:22:41.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0440a3e40 msgr2=0x7fd0440a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0440a3e40 0x7fd0440a6300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd05002ca40 tx=0x7fd050009560 comp rx=0 tx=0).stop 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c1051d0 msgr2=0x7fd05c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c1051d0 0x7fd05c19a950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 shutdown_connections 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0440a3e40 0x7fd0440a6300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd05c105b90 0x7fd05c1a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd05c1051d0 0x7fd05c19a950 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 --2- 172.21.15.44:0/1449451789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd05c103fe0 0x7fd05c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.299+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 >> 172.21.15.44:0/1449451789 conn(0x7fd05c0ff570 msgr2=0x7fd05c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.300+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 shutdown_connections 2024-04-20T11:22:41.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.300+0000 7fd062e00640 1 -- 172.21.15.44:0/1449451789 wait complete. 2024-04-20T11:22:41.319 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-20T11:22:41.319 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-20T11:22:41.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:22:41.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 -- 172.21.15.44:0/1211229179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c107b30 msgr2=0x7f3e8c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 --2- 172.21.15.44:0/1211229179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c107b30 0x7f3e8c109f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009ca0 tx=0x7f3e8402c100 comp rx=0 tx=0).stop 2024-04-20T11:22:41.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 -- 172.21.15.44:0/1211229179 shutdown_connections 2024-04-20T11:22:41.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 --2- 172.21.15.44:0/1211229179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c107b30 0x7f3e8c109f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 --2- 172.21.15.44:0/1211229179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 0x7f3e8c1075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 --2- 172.21.15.44:0/1211229179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c1028d0 0x7f3e8c104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 -- 172.21.15.44:0/1211229179 >> 172.21.15.44:0/1211229179 conn(0x7f3e8c0fc300 msgr2=0x7f3e8c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:41.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 -- 172.21.15.44:0/1211229179 shutdown_connections 2024-04-20T11:22:41.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.526+0000 7f3e8b200640 1 -- 172.21.15.44:0/1211229179 wait complete. 2024-04-20T11:22:41.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.527+0000 7f3e8b200640 1 Processor -- start 2024-04-20T11:22:41.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 -- start start 2024-04-20T11:22:41.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c1028d0 0x7f3e8c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 0x7f3e8c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c107b30 0x7f3e8c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c10db90 con 0x7f3e8c107b30 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c10da10 con 0x7f3e8c1028d0 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c10dd10 con 0x7f3e8c105200 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e89400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 0x7f3e8c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e89e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c1028d0 0x7f3e8c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e89400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 0x7f3e8c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48522/0 (socket says 172.21.15.44:48522) 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e89400640 1 -- 172.21.15.44:0/2780379837 learned_addr learned my addr 172.21.15.44:0/2780379837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.528+0000 7f3e8a800640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c107b30 0x7f3e8c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e89e00640 1 -- 172.21.15.44:0/2780379837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 msgr2=0x7f3e8c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e89e00640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 0x7f3e8c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e89e00640 1 -- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c107b30 msgr2=0x7f3e8c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e89e00640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c107b30 0x7f3e8c1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e89e00640 1 -- 172.21.15.44:0/2780379837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c1a1fe0 con 0x7f3e8c1028d0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e8a800640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c107b30 0x7f3e8c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e89e00640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c1028d0 0x7f3e8c19a3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005fd0 tx=0x7f3e7c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c009c30 con 0x7f3e8c1028d0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e7c009dd0 con 0x7f3e8c1028d0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.529+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c1a22d0 con 0x7f3e8c1028d0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.530+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c010460 con 0x7f3e8c1028d0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.530+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1a2870 con 0x7f3e8c1028d0 2024-04-20T11:22:41.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.531+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8c109620 con 0x7f3e8c1028d0 2024-04-20T11:22:41.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.533+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f3e7c004000 con 0x7f3e8c1028d0 2024-04-20T11:22:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.534+0000 7f3e82a00640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e700a3ec0 0x7f3e700a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:41.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.534+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3e7c0d09a0 con 0x7f3e8c1028d0 2024-04-20T11:22:41.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.534+0000 7f3e89400640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e700a3ec0 0x7f3e700a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:41.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.535+0000 7f3e89400640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e700a3ec0 0x7f3e700a6380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c19b8d0 tx=0x7f3e7800b480 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:41.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.538+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e7c017020 con 0x7f3e8c1028d0 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2504682206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:41 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1449451789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-20T11:22:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T11:22:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-20T11:22:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-20T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-20T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2504682206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:41 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1449451789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:41.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.749+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e8c0623a0 con 0x7f3e700a3ec0 2024-04-20T11:22:41.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.774+0000 7f3e82a00640 1 -- 172.21.15.44:0/2780379837 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7f3e8c0623a0 con 0x7f3e700a3ec0 2024-04-20T11:22:41.779 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:41.781 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:22:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e700a3ec0 msgr2=0x7f3e700a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e700a3ec0 0x7f3e700a6380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c19b8d0 tx=0x7f3e7800b480 comp rx=0 tx=0).stop 2024-04-20T11:22:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c1028d0 msgr2=0x7f3e8c19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c1028d0 0x7f3e8c19a3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005fd0 tx=0x7f3e7c00dde0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 shutdown_connections 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e700a3ec0 0x7f3e700a6380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e8c107b30 0x7f3e8c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e8c105200 0x7f3e8c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 --2- 172.21.15.44:0/2780379837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8c1028d0 0x7f3e8c19a3c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 >> 172.21.15.44:0/2780379837 conn(0x7f3e8c0fc300 msgr2=0x7f3e8c101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.781+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 shutdown_connections 2024-04-20T11:22:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:41.782+0000 7f3e8b200640 1 -- 172.21.15.44:0/2780379837 wait complete. 2024-04-20T11:22:41.804 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-20T11:22:40.121527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189632,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1760,"num_read_kb":1044,"num_write":551,"num_write_kb":4401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":936,"ondisk_log_size":936,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361172,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580652,"statfs":{"total":1151940427776,"available":1151570587648,"internally_reserved":0,"allocated":12275712,"data_stored":9140321,"data_compressed":223938,"data_compressed_allocated":294912,"data_compressed_original":3440640,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.969006"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102181+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:22:36.102181+0000","last_peered":"2024-04-20T11:22:36.102181+0000","last_clean":"2024-04-20T11:22:36.102181+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:22:36.102181+0000","last_undegraded":"2024-04-20T11:22:36.102181+0000","last_fullsized":"2024-04-20T11:22:36.102181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169564+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:22:36.169564+0000","last_peered":"2024-04-20T11:22:36.169564+0000","last_clean":"2024-04-20T11:22:36.169564+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:22:36.169564+0000","last_undegraded":"2024-04-20T11:22:36.169564+0000","last_fullsized":"2024-04-20T11:22:36.169564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169452+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:22:36.169452+0000","last_peered":"2024-04-20T11:22:36.169452+0000","last_clean":"2024-04-20T11:22:36.169452+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:22:36.169452+0000","last_undegraded":"2024-04-20T11:22:36.169452+0000","last_fullsized":"2024-04-20T11:22:36.169452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102312+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:22:36.102312+0000","last_peered":"2024-04-20T11:22:36.102312+0000","last_clean":"2024-04-20T11:22:36.102312+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:22:36.102312+0000","last_undegraded":"2024-04-20T11:22:36.102312+0000","last_fullsized":"2024-04-20T11:22:36.102312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169679+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:22:36.169679+0000","last_peered":"2024-04-20T11:22:36.169679+0000","last_clean":"2024-04-20T11:22:36.169679+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:22:36.169679+0000","last_undegraded":"2024-04-20T11:22:36.169679+0000","last_fullsized":"2024-04-20T11:22:36.169679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151691+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:22:36.151691+0000","last_peered":"2024-04-20T11:22:36.151691+0000","last_clean":"2024-04-20T11:22:36.151691+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:22:36.151691+0000","last_undegraded":"2024-04-20T11:22:36.151691+0000","last_fullsized":"2024-04-20T11:22:36.151691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114218+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:22:36.114218+0000","last_peered":"2024-04-20T11:22:36.114218+0000","last_clean":"2024-04-20T11:22:36.114218+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:22:36.114218+0000","last_undegraded":"2024-04-20T11:22:36.114218+0000","last_fullsized":"2024-04-20T11:22:36.114218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137124+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:22:36.137124+0000","last_peered":"2024-04-20T11:22:36.137124+0000","last_clean":"2024-04-20T11:22:36.137124+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:22:36.137124+0000","last_undegraded":"2024-04-20T11:22:36.137124+0000","last_fullsized":"2024-04-20T11:22:36.137124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169305+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:22:36.169305+0000","last_peered":"2024-04-20T11:22:36.169305+0000","last_clean":"2024-04-20T11:22:36.169305+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:22:36.169305+0000","last_undegraded":"2024-04-20T11:22:36.169305+0000","last_fullsized":"2024-04-20T11:22:36.169305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151953+0000","last_change":"2024-04-20T11:20:51.223729+0000","last_active":"2024-04-20T11:22:36.151953+0000","last_peered":"2024-04-20T11:22:36.151953+0000","last_clean":"2024-04-20T11:22:36.151953+0000","last_became_active":"2024-04-20T11:20:51.223340+0000","last_became_peered":"2024-04-20T11:20:51.223340+0000","last_unstale":"2024-04-20T11:22:36.151953+0000","last_undegraded":"2024-04-20T11:22:36.151953+0000","last_fullsized":"2024-04-20T11:22:36.151953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:13.728111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114424+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:22:36.114424+0000","last_peered":"2024-04-20T11:22:36.114424+0000","last_clean":"2024-04-20T11:22:36.114424+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:22:36.114424+0000","last_undegraded":"2024-04-20T11:22:36.114424+0000","last_fullsized":"2024-04-20T11:22:36.114424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102257+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:22:36.102257+0000","last_peered":"2024-04-20T11:22:36.102257+0000","last_clean":"2024-04-20T11:22:36.102257+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:22:36.102257+0000","last_undegraded":"2024-04-20T11:22:36.102257+0000","last_fullsized":"2024-04-20T11:22:36.102257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134653+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:22:36.134653+0000","last_peered":"2024-04-20T11:22:36.134653+0000","last_clean":"2024-04-20T11:22:36.134653+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:22:36.134653+0000","last_undegraded":"2024-04-20T11:22:36.134653+0000","last_fullsized":"2024-04-20T11:22:36.134653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169165+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:22:36.169165+0000","last_peered":"2024-04-20T11:22:36.169165+0000","last_clean":"2024-04-20T11:22:36.169165+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:22:36.169165+0000","last_undegraded":"2024-04-20T11:22:36.169165+0000","last_fullsized":"2024-04-20T11:22:36.169165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169119+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:22:36.169119+0000","last_peered":"2024-04-20T11:22:36.169119+0000","last_clean":"2024-04-20T11:22:36.169119+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:22:36.169119+0000","last_undegraded":"2024-04-20T11:22:36.169119+0000","last_fullsized":"2024-04-20T11:22:36.169119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168825+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:22:36.168825+0000","last_peered":"2024-04-20T11:22:36.168825+0000","last_clean":"2024-04-20T11:22:36.168825+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:22:36.168825+0000","last_undegraded":"2024-04-20T11:22:36.168825+0000","last_fullsized":"2024-04-20T11:22:36.168825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102334+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:22:36.102334+0000","last_peered":"2024-04-20T11:22:36.102334+0000","last_clean":"2024-04-20T11:22:36.102334+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:22:36.102334+0000","last_undegraded":"2024-04-20T11:22:36.102334+0000","last_fullsized":"2024-04-20T11:22:36.102334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.104636+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:22:36.104636+0000","last_peered":"2024-04-20T11:22:36.104636+0000","last_clean":"2024-04-20T11:22:36.104636+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:22:36.104636+0000","last_undegraded":"2024-04-20T11:22:36.104636+0000","last_fullsized":"2024-04-20T11:22:36.104636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154198+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:22:36.154198+0000","last_peered":"2024-04-20T11:22:36.154198+0000","last_clean":"2024-04-20T11:22:36.154198+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:22:36.154198+0000","last_undegraded":"2024-04-20T11:22:36.154198+0000","last_fullsized":"2024-04-20T11:22:36.154198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":185,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.429408+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:22:36.429408+0000","last_peered":"2024-04-20T11:22:36.429408+0000","last_clean":"2024-04-20T11:22:36.429408+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:22:36.429408+0000","last_undegraded":"2024-04-20T11:22:36.429408+0000","last_fullsized":"2024-04-20T11:22:36.429408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137554+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:22:36.137554+0000","last_peered":"2024-04-20T11:22:36.137554+0000","last_clean":"2024-04-20T11:22:36.137554+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:22:36.137554+0000","last_undegraded":"2024-04-20T11:22:36.137554+0000","last_fullsized":"2024-04-20T11:22:36.137554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137621+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:22:36.137621+0000","last_peered":"2024-04-20T11:22:36.137621+0000","last_clean":"2024-04-20T11:22:36.137621+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:22:36.137621+0000","last_undegraded":"2024-04-20T11:22:36.137621+0000","last_fullsized":"2024-04-20T11:22:36.137621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169732+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:22:36.169732+0000","last_peered":"2024-04-20T11:22:36.169732+0000","last_clean":"2024-04-20T11:22:36.169732+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:22:36.169732+0000","last_undegraded":"2024-04-20T11:22:36.169732+0000","last_fullsized":"2024-04-20T11:22:36.169732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'13","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.870292+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:22:36.870292+0000","last_peered":"2024-04-20T11:22:36.870292+0000","last_clean":"2024-04-20T11:22:36.870292+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:22:36.870292+0000","last_undegraded":"2024-04-20T11:22:36.870292+0000","last_fullsized":"2024-04-20T11:22:36.870292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169025+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:22:36.169025+0000","last_peered":"2024-04-20T11:22:36.169025+0000","last_clean":"2024-04-20T11:22:36.169025+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:22:36.169025+0000","last_undegraded":"2024-04-20T11:22:36.169025+0000","last_fullsized":"2024-04-20T11:22:36.169025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.170006+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:22:36.170006+0000","last_peered":"2024-04-20T11:22:36.170006+0000","last_clean":"2024-04-20T11:22:36.170006+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:22:36.170006+0000","last_undegraded":"2024-04-20T11:22:36.170006+0000","last_fullsized":"2024-04-20T11:22:36.170006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169858+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:22:36.169858+0000","last_peered":"2024-04-20T11:22:36.169858+0000","last_clean":"2024-04-20T11:22:36.169858+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:22:36.169858+0000","last_undegraded":"2024-04-20T11:22:36.169858+0000","last_fullsized":"2024-04-20T11:22:36.169858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":179,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.008672+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:22:37.008672+0000","last_peered":"2024-04-20T11:22:37.008672+0000","last_clean":"2024-04-20T11:22:37.008672+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:22:37.008672+0000","last_undegraded":"2024-04-20T11:22:37.008672+0000","last_fullsized":"2024-04-20T11:22:37.008672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169420+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:22:36.169420+0000","last_peered":"2024-04-20T11:22:36.169420+0000","last_clean":"2024-04-20T11:22:36.169420+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:22:36.169420+0000","last_undegraded":"2024-04-20T11:22:36.169420+0000","last_fullsized":"2024-04-20T11:22:36.169420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151597+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:22:36.151597+0000","last_peered":"2024-04-20T11:22:36.151597+0000","last_clean":"2024-04-20T11:22:36.151597+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:22:36.151597+0000","last_undegraded":"2024-04-20T11:22:36.151597+0000","last_fullsized":"2024-04-20T11:22:36.151597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169629+0000","last_change":"2024-04-20T11:20:51.505789+0000","last_active":"2024-04-20T11:22:36.169629+0000","last_peered":"2024-04-20T11:22:36.169629+0000","last_clean":"2024-04-20T11:22:36.169629+0000","last_became_active":"2024-04-20T11:20:51.505449+0000","last_became_peered":"2024-04-20T11:20:51.505449+0000","last_unstale":"2024-04-20T11:22:36.169629+0000","last_undegraded":"2024-04-20T11:22:36.169629+0000","last_fullsized":"2024-04-20T11:22:36.169629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:30.003670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134458+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:22:36.134458+0000","last_peered":"2024-04-20T11:22:36.134458+0000","last_clean":"2024-04-20T11:22:36.134458+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:22:36.134458+0000","last_undegraded":"2024-04-20T11:22:36.134458+0000","last_fullsized":"2024-04-20T11:22:36.134458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137061+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:22:36.137061+0000","last_peered":"2024-04-20T11:22:36.137061+0000","last_clean":"2024-04-20T11:22:36.137061+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:22:36.137061+0000","last_undegraded":"2024-04-20T11:22:36.137061+0000","last_fullsized":"2024-04-20T11:22:36.137061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168971+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:22:36.168971+0000","last_peered":"2024-04-20T11:22:36.168971+0000","last_clean":"2024-04-20T11:22:36.168971+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:22:36.168971+0000","last_undegraded":"2024-04-20T11:22:36.168971+0000","last_fullsized":"2024-04-20T11:22:36.168971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.103961+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:22:36.103961+0000","last_peered":"2024-04-20T11:22:36.103961+0000","last_clean":"2024-04-20T11:22:36.103961+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:22:36.103961+0000","last_undegraded":"2024-04-20T11:22:36.103961+0000","last_fullsized":"2024-04-20T11:22:36.103961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'6","reported_seq":325,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.427698+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:22:36.427698+0000","last_peered":"2024-04-20T11:22:36.427698+0000","last_clean":"2024-04-20T11:22:36.427698+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:22:36.427698+0000","last_undegraded":"2024-04-20T11:22:36.427698+0000","last_fullsized":"2024-04-20T11:22:36.427698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":297,"num_read_kb":297,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169290+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:22:36.169290+0000","last_peered":"2024-04-20T11:22:36.169290+0000","last_clean":"2024-04-20T11:22:36.169290+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:22:36.169290+0000","last_undegraded":"2024-04-20T11:22:36.169290+0000","last_fullsized":"2024-04-20T11:22:36.169290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114461+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:22:36.114461+0000","last_peered":"2024-04-20T11:22:36.114461+0000","last_clean":"2024-04-20T11:22:36.114461+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:22:36.114461+0000","last_undegraded":"2024-04-20T11:22:36.114461+0000","last_fullsized":"2024-04-20T11:22:36.114461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169778+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:22:36.169778+0000","last_peered":"2024-04-20T11:22:36.169778+0000","last_clean":"2024-04-20T11:22:36.169778+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:22:36.169778+0000","last_undegraded":"2024-04-20T11:22:36.169778+0000","last_fullsized":"2024-04-20T11:22:36.169778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169923+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:22:36.169923+0000","last_peered":"2024-04-20T11:22:36.169923+0000","last_clean":"2024-04-20T11:22:36.169923+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:22:36.169923+0000","last_undegraded":"2024-04-20T11:22:36.169923+0000","last_fullsized":"2024-04-20T11:22:36.169923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169079+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:22:36.169079+0000","last_peered":"2024-04-20T11:22:36.169079+0000","last_clean":"2024-04-20T11:22:36.169079+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:22:36.169079+0000","last_undegraded":"2024-04-20T11:22:36.169079+0000","last_fullsized":"2024-04-20T11:22:36.169079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":366,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.008664+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:22:37.008664+0000","last_peered":"2024-04-20T11:22:37.008664+0000","last_clean":"2024-04-20T11:22:37.008664+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:22:37.008664+0000","last_undegraded":"2024-04-20T11:22:37.008664+0000","last_fullsized":"2024-04-20T11:22:37.008664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65164,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":310,"num_read_kb":218,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169762+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:22:36.169762+0000","last_peered":"2024-04-20T11:22:36.169762+0000","last_clean":"2024-04-20T11:22:36.169762+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:22:36.169762+0000","last_undegraded":"2024-04-20T11:22:36.169762+0000","last_fullsized":"2024-04-20T11:22:36.169762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154183+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:22:36.154183+0000","last_peered":"2024-04-20T11:22:36.154183+0000","last_clean":"2024-04-20T11:22:36.154183+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:22:36.154183+0000","last_undegraded":"2024-04-20T11:22:36.154183+0000","last_fullsized":"2024-04-20T11:22:36.154183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134837+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:22:36.134837+0000","last_peered":"2024-04-20T11:22:36.134837+0000","last_clean":"2024-04-20T11:22:36.134837+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:22:36.134837+0000","last_undegraded":"2024-04-20T11:22:36.134837+0000","last_fullsized":"2024-04-20T11:22:36.134837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169235+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:22:36.169235+0000","last_peered":"2024-04-20T11:22:36.169235+0000","last_clean":"2024-04-20T11:22:36.169235+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:22:36.169235+0000","last_undegraded":"2024-04-20T11:22:36.169235+0000","last_fullsized":"2024-04-20T11:22:36.169235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114321+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:22:36.114321+0000","last_peered":"2024-04-20T11:22:36.114321+0000","last_clean":"2024-04-20T11:22:36.114321+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:22:36.114321+0000","last_undegraded":"2024-04-20T11:22:36.114321+0000","last_fullsized":"2024-04-20T11:22:36.114321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'9","reported_seq":184,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.424523+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:22:36.424523+0000","last_peered":"2024-04-20T11:22:36.424523+0000","last_clean":"2024-04-20T11:22:36.424523+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:22:36.424523+0000","last_undegraded":"2024-04-20T11:22:36.424523+0000","last_fullsized":"2024-04-20T11:22:36.424523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":5,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114107+0000","last_change":"2024-04-20T11:20:51.223843+0000","last_active":"2024-04-20T11:22:36.114107+0000","last_peered":"2024-04-20T11:22:36.114107+0000","last_clean":"2024-04-20T11:22:36.114107+0000","last_became_active":"2024-04-20T11:20:51.223311+0000","last_became_peered":"2024-04-20T11:20:51.223311+0000","last_unstale":"2024-04-20T11:22:36.114107+0000","last_undegraded":"2024-04-20T11:22:36.114107+0000","last_fullsized":"2024-04-20T11:22:36.114107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:38.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101882+0000","last_change":"2024-04-20T11:20:51.236281+0000","last_active":"2024-04-20T11:22:36.101882+0000","last_peered":"2024-04-20T11:22:36.101882+0000","last_clean":"2024-04-20T11:22:36.101882+0000","last_became_active":"2024-04-20T11:20:51.222316+0000","last_became_peered":"2024-04-20T11:20:51.222316+0000","last_unstale":"2024-04-20T11:22:36.101882+0000","last_undegraded":"2024-04-20T11:22:36.101882+0000","last_fullsized":"2024-04-20T11:22:36.101882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169128+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:22:36.169128+0000","last_peered":"2024-04-20T11:22:36.169128+0000","last_clean":"2024-04-20T11:22:36.169128+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:22:36.169128+0000","last_undegraded":"2024-04-20T11:22:36.169128+0000","last_fullsized":"2024-04-20T11:22:36.169128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.170084+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:22:36.170084+0000","last_peered":"2024-04-20T11:22:36.170084+0000","last_clean":"2024-04-20T11:22:36.170084+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:22:36.170084+0000","last_undegraded":"2024-04-20T11:22:36.170084+0000","last_fullsized":"2024-04-20T11:22:36.170084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102064+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:22:36.102064+0000","last_peered":"2024-04-20T11:22:36.102064+0000","last_clean":"2024-04-20T11:22:36.102064+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:22:36.102064+0000","last_undegraded":"2024-04-20T11:22:36.102064+0000","last_fullsized":"2024-04-20T11:22:36.102064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.104397+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:22:36.104397+0000","last_peered":"2024-04-20T11:22:36.104397+0000","last_clean":"2024-04-20T11:22:36.104397+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:22:36.104397+0000","last_undegraded":"2024-04-20T11:22:36.104397+0000","last_fullsized":"2024-04-20T11:22:36.104397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151784+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:22:36.151784+0000","last_peered":"2024-04-20T11:22:36.151784+0000","last_clean":"2024-04-20T11:22:36.151784+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:22:36.151784+0000","last_undegraded":"2024-04-20T11:22:36.151784+0000","last_fullsized":"2024-04-20T11:22:36.151784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":178,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.120190+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:22:37.120190+0000","last_peered":"2024-04-20T11:22:37.120190+0000","last_clean":"2024-04-20T11:22:37.120190+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:22:37.120190+0000","last_undegraded":"2024-04-20T11:22:37.120190+0000","last_fullsized":"2024-04-20T11:22:37.120190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151591+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:22:36.151591+0000","last_peered":"2024-04-20T11:22:36.151591+0000","last_clean":"2024-04-20T11:22:36.151591+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:22:36.151591+0000","last_undegraded":"2024-04-20T11:22:36.151591+0000","last_fullsized":"2024-04-20T11:22:36.151591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169048+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:22:36.169048+0000","last_peered":"2024-04-20T11:22:36.169048+0000","last_clean":"2024-04-20T11:22:36.169048+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:22:36.169048+0000","last_undegraded":"2024-04-20T11:22:36.169048+0000","last_fullsized":"2024-04-20T11:22:36.169048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.426373+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:22:36.426373+0000","last_peered":"2024-04-20T11:22:36.426373+0000","last_clean":"2024-04-20T11:22:36.426373+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:22:36.426373+0000","last_undegraded":"2024-04-20T11:22:36.426373+0000","last_fullsized":"2024-04-20T11:22:36.426373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":151,"num_read_kb":151,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137260+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:22:36.137260+0000","last_peered":"2024-04-20T11:22:36.137260+0000","last_clean":"2024-04-20T11:22:36.137260+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:22:36.137260+0000","last_undegraded":"2024-04-20T11:22:36.137260+0000","last_fullsized":"2024-04-20T11:22:36.137260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102418+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:22:36.102418+0000","last_peered":"2024-04-20T11:22:36.102418+0000","last_clean":"2024-04-20T11:22:36.102418+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:22:36.102418+0000","last_undegraded":"2024-04-20T11:22:36.102418+0000","last_fullsized":"2024-04-20T11:22:36.102418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114180+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:22:36.114180+0000","last_peered":"2024-04-20T11:22:36.114180+0000","last_clean":"2024-04-20T11:22:36.114180+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:22:36.114180+0000","last_undegraded":"2024-04-20T11:22:36.114180+0000","last_fullsized":"2024-04-20T11:22:36.114180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134177+0000","last_change":"2024-04-20T11:20:51.224226+0000","last_active":"2024-04-20T11:22:36.134177+0000","last_peered":"2024-04-20T11:22:36.134177+0000","last_clean":"2024-04-20T11:22:36.134177+0000","last_became_active":"2024-04-20T11:20:51.223728+0000","last_became_peered":"2024-04-20T11:20:51.223728+0000","last_unstale":"2024-04-20T11:22:36.134177+0000","last_undegraded":"2024-04-20T11:22:36.134177+0000","last_fullsized":"2024-04-20T11:22:36.134177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:37.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114398+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:22:36.114398+0000","last_peered":"2024-04-20T11:22:36.114398+0000","last_clean":"2024-04-20T11:22:36.114398+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:22:36.114398+0000","last_undegraded":"2024-04-20T11:22:36.114398+0000","last_fullsized":"2024-04-20T11:22:36.114398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169785+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:22:36.169785+0000","last_peered":"2024-04-20T11:22:36.169785+0000","last_clean":"2024-04-20T11:22:36.169785+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:22:36.169785+0000","last_undegraded":"2024-04-20T11:22:36.169785+0000","last_fullsized":"2024-04-20T11:22:36.169785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134405+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:22:36.134405+0000","last_peered":"2024-04-20T11:22:36.134405+0000","last_clean":"2024-04-20T11:22:36.134405+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:22:36.134405+0000","last_undegraded":"2024-04-20T11:22:36.134405+0000","last_fullsized":"2024-04-20T11:22:36.134405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.366090+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:22:36.366090+0000","last_peered":"2024-04-20T11:22:36.366090+0000","last_clean":"2024-04-20T11:22:36.366090+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:22:36.366090+0000","last_undegraded":"2024-04-20T11:22:36.366090+0000","last_fullsized":"2024-04-20T11:22:36.366090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137464+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:22:36.137464+0000","last_peered":"2024-04-20T11:22:36.137464+0000","last_clean":"2024-04-20T11:22:36.137464+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:22:36.137464+0000","last_undegraded":"2024-04-20T11:22:36.137464+0000","last_fullsized":"2024-04-20T11:22:36.137464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102357+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:22:36.102357+0000","last_peered":"2024-04-20T11:22:36.102357+0000","last_clean":"2024-04-20T11:22:36.102357+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:22:36.102357+0000","last_undegraded":"2024-04-20T11:22:36.102357+0000","last_fullsized":"2024-04-20T11:22:36.102357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169611+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:22:36.169611+0000","last_peered":"2024-04-20T11:22:36.169611+0000","last_clean":"2024-04-20T11:22:36.169611+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:22:36.169611+0000","last_undegraded":"2024-04-20T11:22:36.169611+0000","last_fullsized":"2024-04-20T11:22:36.169611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168896+0000","last_change":"2024-04-20T11:20:51.225236+0000","last_active":"2024-04-20T11:22:36.168896+0000","last_peered":"2024-04-20T11:22:36.168896+0000","last_clean":"2024-04-20T11:22:36.168896+0000","last_became_active":"2024-04-20T11:20:51.221635+0000","last_became_peered":"2024-04-20T11:20:51.221635+0000","last_unstale":"2024-04-20T11:22:36.168896+0000","last_undegraded":"2024-04-20T11:22:36.168896+0000","last_fullsized":"2024-04-20T11:22:36.168896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:03.894361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169171+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:22:36.169171+0000","last_peered":"2024-04-20T11:22:36.169171+0000","last_clean":"2024-04-20T11:22:36.169171+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:22:36.169171+0000","last_undegraded":"2024-04-20T11:22:36.169171+0000","last_fullsized":"2024-04-20T11:22:36.169171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137112+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:22:36.137112+0000","last_peered":"2024-04-20T11:22:36.137112+0000","last_clean":"2024-04-20T11:22:36.137112+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:22:36.137112+0000","last_undegraded":"2024-04-20T11:22:36.137112+0000","last_fullsized":"2024-04-20T11:22:36.137112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101804+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:22:36.101804+0000","last_peered":"2024-04-20T11:22:36.101804+0000","last_clean":"2024-04-20T11:22:36.101804+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:22:36.101804+0000","last_undegraded":"2024-04-20T11:22:36.101804+0000","last_fullsized":"2024-04-20T11:22:36.101804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154030+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:22:36.154030+0000","last_peered":"2024-04-20T11:22:36.154030+0000","last_clean":"2024-04-20T11:22:36.154030+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:22:36.154030+0000","last_undegraded":"2024-04-20T11:22:36.154030+0000","last_fullsized":"2024-04-20T11:22:36.154030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137375+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:22:36.137375+0000","last_peered":"2024-04-20T11:22:36.137375+0000","last_clean":"2024-04-20T11:22:36.137375+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:22:36.137375+0000","last_undegraded":"2024-04-20T11:22:36.137375+0000","last_fullsized":"2024-04-20T11:22:36.137375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169506+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:22:36.169506+0000","last_peered":"2024-04-20T11:22:36.169506+0000","last_clean":"2024-04-20T11:22:36.169506+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:22:36.169506+0000","last_undegraded":"2024-04-20T11:22:36.169506+0000","last_fullsized":"2024-04-20T11:22:36.169506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114559+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:22:36.114559+0000","last_peered":"2024-04-20T11:22:36.114559+0000","last_clean":"2024-04-20T11:22:36.114559+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:22:36.114559+0000","last_undegraded":"2024-04-20T11:22:36.114559+0000","last_fullsized":"2024-04-20T11:22:36.114559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'8","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154545+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:22:36.154545+0000","last_peered":"2024-04-20T11:22:36.154545+0000","last_clean":"2024-04-20T11:22:36.154545+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:22:36.154545+0000","last_undegraded":"2024-04-20T11:22:36.154545+0000","last_fullsized":"2024-04-20T11:22:36.154545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114465+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:22:36.114465+0000","last_peered":"2024-04-20T11:22:36.114465+0000","last_clean":"2024-04-20T11:22:36.114465+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:22:36.114465+0000","last_undegraded":"2024-04-20T11:22:36.114465+0000","last_fullsized":"2024-04-20T11:22:36.114465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.363029+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:22:36.363029+0000","last_peered":"2024-04-20T11:22:36.363029+0000","last_clean":"2024-04-20T11:22:36.363029+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:22:36.363029+0000","last_undegraded":"2024-04-20T11:22:36.363029+0000","last_fullsized":"2024-04-20T11:22:36.363029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114342+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:22:36.114342+0000","last_peered":"2024-04-20T11:22:36.114342+0000","last_clean":"2024-04-20T11:22:36.114342+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:22:36.114342+0000","last_undegraded":"2024-04-20T11:22:36.114342+0000","last_fullsized":"2024-04-20T11:22:36.114342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134792+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:22:36.134792+0000","last_peered":"2024-04-20T11:22:36.134792+0000","last_clean":"2024-04-20T11:22:36.134792+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:22:36.134792+0000","last_undegraded":"2024-04-20T11:22:36.134792+0000","last_fullsized":"2024-04-20T11:22:36.134792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114458+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:22:36.114458+0000","last_peered":"2024-04-20T11:22:36.114458+0000","last_clean":"2024-04-20T11:22:36.114458+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:22:36.114458+0000","last_undegraded":"2024-04-20T11:22:36.114458+0000","last_fullsized":"2024-04-20T11:22:36.114458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101975+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:22:36.101975+0000","last_peered":"2024-04-20T11:22:36.101975+0000","last_clean":"2024-04-20T11:22:36.101975+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:22:36.101975+0000","last_undegraded":"2024-04-20T11:22:36.101975+0000","last_fullsized":"2024-04-20T11:22:36.101975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154063+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:22:36.154063+0000","last_peered":"2024-04-20T11:22:36.154063+0000","last_clean":"2024-04-20T11:22:36.154063+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:22:36.154063+0000","last_undegraded":"2024-04-20T11:22:36.154063+0000","last_fullsized":"2024-04-20T11:22:36.154063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154017+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:22:36.154017+0000","last_peered":"2024-04-20T11:22:36.154017+0000","last_clean":"2024-04-20T11:22:36.154017+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:22:36.154017+0000","last_undegraded":"2024-04-20T11:22:36.154017+0000","last_fullsized":"2024-04-20T11:22:36.154017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169323+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:22:36.169323+0000","last_peered":"2024-04-20T11:22:36.169323+0000","last_clean":"2024-04-20T11:22:36.169323+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:22:36.169323+0000","last_undegraded":"2024-04-20T11:22:36.169323+0000","last_fullsized":"2024-04-20T11:22:36.169323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134952+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:22:36.134952+0000","last_peered":"2024-04-20T11:22:36.134952+0000","last_clean":"2024-04-20T11:22:36.134952+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:22:36.134952+0000","last_undegraded":"2024-04-20T11:22:36.134952+0000","last_fullsized":"2024-04-20T11:22:36.134952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101932+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:22:36.101932+0000","last_peered":"2024-04-20T11:22:36.101932+0000","last_clean":"2024-04-20T11:22:36.101932+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:22:36.101932+0000","last_undegraded":"2024-04-20T11:22:36.101932+0000","last_fullsized":"2024-04-20T11:22:36.101932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154450+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:22:36.154450+0000","last_peered":"2024-04-20T11:22:36.154450+0000","last_clean":"2024-04-20T11:22:36.154450+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:22:36.154450+0000","last_undegraded":"2024-04-20T11:22:36.154450+0000","last_fullsized":"2024-04-20T11:22:36.154450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137439+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:22:36.137439+0000","last_peered":"2024-04-20T11:22:36.137439+0000","last_clean":"2024-04-20T11:22:36.137439+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:22:36.137439+0000","last_undegraded":"2024-04-20T11:22:36.137439+0000","last_fullsized":"2024-04-20T11:22:36.137439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169002+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:22:36.169002+0000","last_peered":"2024-04-20T11:22:36.169002+0000","last_clean":"2024-04-20T11:22:36.169002+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:22:36.169002+0000","last_undegraded":"2024-04-20T11:22:36.169002+0000","last_fullsized":"2024-04-20T11:22:36.169002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137123+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:22:36.137123+0000","last_peered":"2024-04-20T11:22:36.137123+0000","last_clean":"2024-04-20T11:22:36.137123+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:22:36.137123+0000","last_undegraded":"2024-04-20T11:22:36.137123+0000","last_fullsized":"2024-04-20T11:22:36.137123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168939+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:22:36.168939+0000","last_peered":"2024-04-20T11:22:36.168939+0000","last_clean":"2024-04-20T11:22:36.168939+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:22:36.168939+0000","last_undegraded":"2024-04-20T11:22:36.168939+0000","last_fullsized":"2024-04-20T11:22:36.168939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134401+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:22:36.134401+0000","last_peered":"2024-04-20T11:22:36.134401+0000","last_clean":"2024-04-20T11:22:36.134401+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:22:36.134401+0000","last_undegraded":"2024-04-20T11:22:36.134401+0000","last_fullsized":"2024-04-20T11:22:36.134401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101694+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:22:36.101694+0000","last_peered":"2024-04-20T11:22:36.101694+0000","last_clean":"2024-04-20T11:22:36.101694+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:22:36.101694+0000","last_undegraded":"2024-04-20T11:22:36.101694+0000","last_fullsized":"2024-04-20T11:22:36.101694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107634,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599030,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1624,"num_read_kb":925,"num_write":368,"num_write_kb":1461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":700416,"data_stored":1797090,"data_compressed":81420,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":466545,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645361,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1254617,"data_compressed":46779,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":858656,"data_compressed":22053,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":858922,"data_compressed":22053,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1059650,"data_compressed":38785,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":465157,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":531625,"data_compressed":4745,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":467419,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":532377,"data_compressed":4745,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":989798,"data_compressed":28209,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":466332,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1189223,"data_compressed":44941,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67307,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68059,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":199814,"data_compressed":7994,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:22:41.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2504682206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:41 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1449451789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 -- 172.21.15.44:0/2451885563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 msgr2=0x7fd628101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 --2- 172.21.15.44:0/2451885563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 0x7fd628101100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009900 tx=0x7fd61c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 -- 172.21.15.44:0/2451885563 shutdown_connections 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 --2- 172.21.15.44:0/2451885563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628102890 0x7fd628109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 --2- 172.21.15.44:0/2451885563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd628102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 --2- 172.21.15.44:0/2451885563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 0x7fd628101100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 -- 172.21.15.44:0/2451885563 >> 172.21.15.44:0/2451885563 conn(0x7fd6280fc230 msgr2=0x7fd6280fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 -- 172.21.15.44:0/2451885563 shutdown_connections 2024-04-20T11:22:42.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.054+0000 7fd62e800640 1 -- 172.21.15.44:0/2451885563 wait complete. 2024-04-20T11:22:42.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.055+0000 7fd62e800640 1 Processor -- start 2024-04-20T11:22:42.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 -- start start 2024-04-20T11:22:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 0x7fd62819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd62819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628102890 0x7fd6281a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810dba0 con 0x7fd628102890 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810da20 con 0x7fd628100ce0 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.056+0000 7fd62e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810dd20 con 0x7fd628101ed0 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd62819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628102890 0x7fd6281a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd62819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48544/0 (socket says 172.21.15.44:48544) 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 -- 172.21.15.44:0/1082990814 learned_addr learned my addr 172.21.15.44:0/1082990814 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62d400640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 0x7fd62819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 -- 172.21.15.44:0/1082990814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 msgr2=0x7fd62819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 0x7fd62819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 -- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628102890 msgr2=0x7fd6281a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628102890 0x7fd6281a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 -- 172.21.15.44:0/1082990814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281a20d0 con 0x7fd628101ed0 2024-04-20T11:22:42.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.057+0000 7fd62ca00640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd62819aa40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e9f0 tx=0x7fd61400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:42.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.058+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400cd80 con 0x7fd628101ed0 2024-04-20T11:22:42.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.058+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281a23c0 con 0x7fd628101ed0 2024-04-20T11:22:42.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.058+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd614005520 con 0x7fd628101ed0 2024-04-20T11:22:42.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.058+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614010690 con 0x7fd628101ed0 2024-04-20T11:22:42.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.058+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a2910 con 0x7fd628101ed0 2024-04-20T11:22:42.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.060+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6281a2bd0 con 0x7fd628101ed0 2024-04-20T11:22:42.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.063+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fd6140046c0 con 0x7fd628101ed0 2024-04-20T11:22:42.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.064+0000 7fd626000640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6100a3e40 0x7fd6100a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.065+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fd6140d0eb0 con 0x7fd628101ed0 2024-04-20T11:22:42.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.065+0000 7fd62d400640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6100a3e40 0x7fd6100a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.066+0000 7fd62d400640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6100a3e40 0x7fd6100a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd61c02c250 tx=0x7fd61c02c9e0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:42.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.069+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd61409d310 con 0x7fd628101ed0 2024-04-20T11:22:42.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.293+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd62810b200 con 0x7fd6100a3e40 2024-04-20T11:22:42.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.305+0000 7fd626000640 1 -- 172.21.15.44:0/1082990814 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7fd62810b200 con 0x7fd6100a3e40 2024-04-20T11:22:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:22:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6100a3e40 msgr2=0x7fd6100a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6100a3e40 0x7fd6100a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd61c02c250 tx=0x7fd61c02c9e0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 msgr2=0x7fd62819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd62819aa40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e9f0 tx=0x7fd61400edf0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 shutdown_connections 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6100a3e40 0x7fd6100a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd628102890 0x7fd6281a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd628101ed0 0x7fd62819aa40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 --2- 172.21.15.44:0/1082990814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd628100ce0 0x7fd62819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 >> 172.21.15.44:0/1082990814 conn(0x7fd6280fc230 msgr2=0x7fd628109ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 shutdown_connections 2024-04-20T11:22:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.312+0000 7fd62e800640 1 -- 172.21.15.44:0/1082990814 wait complete. 2024-04-20T11:22:42.335 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-20T11:22:42.122373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189632,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1760,"num_read_kb":1044,"num_write":551,"num_write_kb":4401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":936,"ondisk_log_size":936,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361172,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580652,"statfs":{"total":1151940427776,"available":1151570587648,"internally_reserved":0,"allocated":12275712,"data_stored":9140321,"data_compressed":223938,"data_compressed_allocated":294912,"data_compressed_original":3440640,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.969852"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102181+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:22:36.102181+0000","last_peered":"2024-04-20T11:22:36.102181+0000","last_clean":"2024-04-20T11:22:36.102181+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:22:36.102181+0000","last_undegraded":"2024-04-20T11:22:36.102181+0000","last_fullsized":"2024-04-20T11:22:36.102181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169564+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:22:36.169564+0000","last_peered":"2024-04-20T11:22:36.169564+0000","last_clean":"2024-04-20T11:22:36.169564+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:22:36.169564+0000","last_undegraded":"2024-04-20T11:22:36.169564+0000","last_fullsized":"2024-04-20T11:22:36.169564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169452+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:22:36.169452+0000","last_peered":"2024-04-20T11:22:36.169452+0000","last_clean":"2024-04-20T11:22:36.169452+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:22:36.169452+0000","last_undegraded":"2024-04-20T11:22:36.169452+0000","last_fullsized":"2024-04-20T11:22:36.169452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102312+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:22:36.102312+0000","last_peered":"2024-04-20T11:22:36.102312+0000","last_clean":"2024-04-20T11:22:36.102312+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:22:36.102312+0000","last_undegraded":"2024-04-20T11:22:36.102312+0000","last_fullsized":"2024-04-20T11:22:36.102312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169679+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:22:36.169679+0000","last_peered":"2024-04-20T11:22:36.169679+0000","last_clean":"2024-04-20T11:22:36.169679+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:22:36.169679+0000","last_undegraded":"2024-04-20T11:22:36.169679+0000","last_fullsized":"2024-04-20T11:22:36.169679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151691+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:22:36.151691+0000","last_peered":"2024-04-20T11:22:36.151691+0000","last_clean":"2024-04-20T11:22:36.151691+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:22:36.151691+0000","last_undegraded":"2024-04-20T11:22:36.151691+0000","last_fullsized":"2024-04-20T11:22:36.151691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114218+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:22:36.114218+0000","last_peered":"2024-04-20T11:22:36.114218+0000","last_clean":"2024-04-20T11:22:36.114218+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:22:36.114218+0000","last_undegraded":"2024-04-20T11:22:36.114218+0000","last_fullsized":"2024-04-20T11:22:36.114218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137124+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:22:36.137124+0000","last_peered":"2024-04-20T11:22:36.137124+0000","last_clean":"2024-04-20T11:22:36.137124+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:22:36.137124+0000","last_undegraded":"2024-04-20T11:22:36.137124+0000","last_fullsized":"2024-04-20T11:22:36.137124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169305+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:22:36.169305+0000","last_peered":"2024-04-20T11:22:36.169305+0000","last_clean":"2024-04-20T11:22:36.169305+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:22:36.169305+0000","last_undegraded":"2024-04-20T11:22:36.169305+0000","last_fullsized":"2024-04-20T11:22:36.169305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151953+0000","last_change":"2024-04-20T11:20:51.223729+0000","last_active":"2024-04-20T11:22:36.151953+0000","last_peered":"2024-04-20T11:22:36.151953+0000","last_clean":"2024-04-20T11:22:36.151953+0000","last_became_active":"2024-04-20T11:20:51.223340+0000","last_became_peered":"2024-04-20T11:20:51.223340+0000","last_unstale":"2024-04-20T11:22:36.151953+0000","last_undegraded":"2024-04-20T11:22:36.151953+0000","last_fullsized":"2024-04-20T11:22:36.151953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:13.728111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114424+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:22:36.114424+0000","last_peered":"2024-04-20T11:22:36.114424+0000","last_clean":"2024-04-20T11:22:36.114424+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:22:36.114424+0000","last_undegraded":"2024-04-20T11:22:36.114424+0000","last_fullsized":"2024-04-20T11:22:36.114424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102257+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:22:36.102257+0000","last_peered":"2024-04-20T11:22:36.102257+0000","last_clean":"2024-04-20T11:22:36.102257+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:22:36.102257+0000","last_undegraded":"2024-04-20T11:22:36.102257+0000","last_fullsized":"2024-04-20T11:22:36.102257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134653+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:22:36.134653+0000","last_peered":"2024-04-20T11:22:36.134653+0000","last_clean":"2024-04-20T11:22:36.134653+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:22:36.134653+0000","last_undegraded":"2024-04-20T11:22:36.134653+0000","last_fullsized":"2024-04-20T11:22:36.134653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169165+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:22:36.169165+0000","last_peered":"2024-04-20T11:22:36.169165+0000","last_clean":"2024-04-20T11:22:36.169165+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:22:36.169165+0000","last_undegraded":"2024-04-20T11:22:36.169165+0000","last_fullsized":"2024-04-20T11:22:36.169165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169119+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:22:36.169119+0000","last_peered":"2024-04-20T11:22:36.169119+0000","last_clean":"2024-04-20T11:22:36.169119+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:22:36.169119+0000","last_undegraded":"2024-04-20T11:22:36.169119+0000","last_fullsized":"2024-04-20T11:22:36.169119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168825+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:22:36.168825+0000","last_peered":"2024-04-20T11:22:36.168825+0000","last_clean":"2024-04-20T11:22:36.168825+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:22:36.168825+0000","last_undegraded":"2024-04-20T11:22:36.168825+0000","last_fullsized":"2024-04-20T11:22:36.168825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102334+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:22:36.102334+0000","last_peered":"2024-04-20T11:22:36.102334+0000","last_clean":"2024-04-20T11:22:36.102334+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:22:36.102334+0000","last_undegraded":"2024-04-20T11:22:36.102334+0000","last_fullsized":"2024-04-20T11:22:36.102334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.104636+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:22:36.104636+0000","last_peered":"2024-04-20T11:22:36.104636+0000","last_clean":"2024-04-20T11:22:36.104636+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:22:36.104636+0000","last_undegraded":"2024-04-20T11:22:36.104636+0000","last_fullsized":"2024-04-20T11:22:36.104636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154198+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:22:36.154198+0000","last_peered":"2024-04-20T11:22:36.154198+0000","last_clean":"2024-04-20T11:22:36.154198+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:22:36.154198+0000","last_undegraded":"2024-04-20T11:22:36.154198+0000","last_fullsized":"2024-04-20T11:22:36.154198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":185,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.429408+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:22:36.429408+0000","last_peered":"2024-04-20T11:22:36.429408+0000","last_clean":"2024-04-20T11:22:36.429408+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:22:36.429408+0000","last_undegraded":"2024-04-20T11:22:36.429408+0000","last_fullsized":"2024-04-20T11:22:36.429408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137554+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:22:36.137554+0000","last_peered":"2024-04-20T11:22:36.137554+0000","last_clean":"2024-04-20T11:22:36.137554+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:22:36.137554+0000","last_undegraded":"2024-04-20T11:22:36.137554+0000","last_fullsized":"2024-04-20T11:22:36.137554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137621+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:22:36.137621+0000","last_peered":"2024-04-20T11:22:36.137621+0000","last_clean":"2024-04-20T11:22:36.137621+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:22:36.137621+0000","last_undegraded":"2024-04-20T11:22:36.137621+0000","last_fullsized":"2024-04-20T11:22:36.137621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169732+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:22:36.169732+0000","last_peered":"2024-04-20T11:22:36.169732+0000","last_clean":"2024-04-20T11:22:36.169732+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:22:36.169732+0000","last_undegraded":"2024-04-20T11:22:36.169732+0000","last_fullsized":"2024-04-20T11:22:36.169732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'13","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.870292+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:22:36.870292+0000","last_peered":"2024-04-20T11:22:36.870292+0000","last_clean":"2024-04-20T11:22:36.870292+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:22:36.870292+0000","last_undegraded":"2024-04-20T11:22:36.870292+0000","last_fullsized":"2024-04-20T11:22:36.870292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169025+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:22:36.169025+0000","last_peered":"2024-04-20T11:22:36.169025+0000","last_clean":"2024-04-20T11:22:36.169025+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:22:36.169025+0000","last_undegraded":"2024-04-20T11:22:36.169025+0000","last_fullsized":"2024-04-20T11:22:36.169025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.170006+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:22:36.170006+0000","last_peered":"2024-04-20T11:22:36.170006+0000","last_clean":"2024-04-20T11:22:36.170006+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:22:36.170006+0000","last_undegraded":"2024-04-20T11:22:36.170006+0000","last_fullsized":"2024-04-20T11:22:36.170006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169858+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:22:36.169858+0000","last_peered":"2024-04-20T11:22:36.169858+0000","last_clean":"2024-04-20T11:22:36.169858+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:22:36.169858+0000","last_undegraded":"2024-04-20T11:22:36.169858+0000","last_fullsized":"2024-04-20T11:22:36.169858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":179,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.008672+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:22:37.008672+0000","last_peered":"2024-04-20T11:22:37.008672+0000","last_clean":"2024-04-20T11:22:37.008672+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:22:37.008672+0000","last_undegraded":"2024-04-20T11:22:37.008672+0000","last_fullsized":"2024-04-20T11:22:37.008672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169420+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:22:36.169420+0000","last_peered":"2024-04-20T11:22:36.169420+0000","last_clean":"2024-04-20T11:22:36.169420+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:22:36.169420+0000","last_undegraded":"2024-04-20T11:22:36.169420+0000","last_fullsized":"2024-04-20T11:22:36.169420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151597+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:22:36.151597+0000","last_peered":"2024-04-20T11:22:36.151597+0000","last_clean":"2024-04-20T11:22:36.151597+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:22:36.151597+0000","last_undegraded":"2024-04-20T11:22:36.151597+0000","last_fullsized":"2024-04-20T11:22:36.151597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169629+0000","last_change":"2024-04-20T11:20:51.505789+0000","last_active":"2024-04-20T11:22:36.169629+0000","last_peered":"2024-04-20T11:22:36.169629+0000","last_clean":"2024-04-20T11:22:36.169629+0000","last_became_active":"2024-04-20T11:20:51.505449+0000","last_became_peered":"2024-04-20T11:20:51.505449+0000","last_unstale":"2024-04-20T11:22:36.169629+0000","last_undegraded":"2024-04-20T11:22:36.169629+0000","last_fullsized":"2024-04-20T11:22:36.169629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:30.003670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134458+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:22:36.134458+0000","last_peered":"2024-04-20T11:22:36.134458+0000","last_clean":"2024-04-20T11:22:36.134458+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:22:36.134458+0000","last_undegraded":"2024-04-20T11:22:36.134458+0000","last_fullsized":"2024-04-20T11:22:36.134458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137061+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:22:36.137061+0000","last_peered":"2024-04-20T11:22:36.137061+0000","last_clean":"2024-04-20T11:22:36.137061+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:22:36.137061+0000","last_undegraded":"2024-04-20T11:22:36.137061+0000","last_fullsized":"2024-04-20T11:22:36.137061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168971+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:22:36.168971+0000","last_peered":"2024-04-20T11:22:36.168971+0000","last_clean":"2024-04-20T11:22:36.168971+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:22:36.168971+0000","last_undegraded":"2024-04-20T11:22:36.168971+0000","last_fullsized":"2024-04-20T11:22:36.168971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.103961+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:22:36.103961+0000","last_peered":"2024-04-20T11:22:36.103961+0000","last_clean":"2024-04-20T11:22:36.103961+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:22:36.103961+0000","last_undegraded":"2024-04-20T11:22:36.103961+0000","last_fullsized":"2024-04-20T11:22:36.103961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'6","reported_seq":325,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.427698+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:22:36.427698+0000","last_peered":"2024-04-20T11:22:36.427698+0000","last_clean":"2024-04-20T11:22:36.427698+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:22:36.427698+0000","last_undegraded":"2024-04-20T11:22:36.427698+0000","last_fullsized":"2024-04-20T11:22:36.427698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":297,"num_read_kb":297,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169290+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:22:36.169290+0000","last_peered":"2024-04-20T11:22:36.169290+0000","last_clean":"2024-04-20T11:22:36.169290+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:22:36.169290+0000","last_undegraded":"2024-04-20T11:22:36.169290+0000","last_fullsized":"2024-04-20T11:22:36.169290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114461+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:22:36.114461+0000","last_peered":"2024-04-20T11:22:36.114461+0000","last_clean":"2024-04-20T11:22:36.114461+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:22:36.114461+0000","last_undegraded":"2024-04-20T11:22:36.114461+0000","last_fullsized":"2024-04-20T11:22:36.114461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169778+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:22:36.169778+0000","last_peered":"2024-04-20T11:22:36.169778+0000","last_clean":"2024-04-20T11:22:36.169778+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:22:36.169778+0000","last_undegraded":"2024-04-20T11:22:36.169778+0000","last_fullsized":"2024-04-20T11:22:36.169778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169923+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:22:36.169923+0000","last_peered":"2024-04-20T11:22:36.169923+0000","last_clean":"2024-04-20T11:22:36.169923+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:22:36.169923+0000","last_undegraded":"2024-04-20T11:22:36.169923+0000","last_fullsized":"2024-04-20T11:22:36.169923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169079+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:22:36.169079+0000","last_peered":"2024-04-20T11:22:36.169079+0000","last_clean":"2024-04-20T11:22:36.169079+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:22:36.169079+0000","last_undegraded":"2024-04-20T11:22:36.169079+0000","last_fullsized":"2024-04-20T11:22:36.169079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":366,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.008664+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:22:37.008664+0000","last_peered":"2024-04-20T11:22:37.008664+0000","last_clean":"2024-04-20T11:22:37.008664+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:22:37.008664+0000","last_undegraded":"2024-04-20T11:22:37.008664+0000","last_fullsized":"2024-04-20T11:22:37.008664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65164,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":310,"num_read_kb":218,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169762+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:22:36.169762+0000","last_peered":"2024-04-20T11:22:36.169762+0000","last_clean":"2024-04-20T11:22:36.169762+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:22:36.169762+0000","last_undegraded":"2024-04-20T11:22:36.169762+0000","last_fullsized":"2024-04-20T11:22:36.169762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154183+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:22:36.154183+0000","last_peered":"2024-04-20T11:22:36.154183+0000","last_clean":"2024-04-20T11:22:36.154183+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:22:36.154183+0000","last_undegraded":"2024-04-20T11:22:36.154183+0000","last_fullsized":"2024-04-20T11:22:36.154183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134837+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:22:36.134837+0000","last_peered":"2024-04-20T11:22:36.134837+0000","last_clean":"2024-04-20T11:22:36.134837+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:22:36.134837+0000","last_undegraded":"2024-04-20T11:22:36.134837+0000","last_fullsized":"2024-04-20T11:22:36.134837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169235+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:22:36.169235+0000","last_peered":"2024-04-20T11:22:36.169235+0000","last_clean":"2024-04-20T11:22:36.169235+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:22:36.169235+0000","last_undegraded":"2024-04-20T11:22:36.169235+0000","last_fullsized":"2024-04-20T11:22:36.169235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114321+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:22:36.114321+0000","last_peered":"2024-04-20T11:22:36.114321+0000","last_clean":"2024-04-20T11:22:36.114321+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:22:36.114321+0000","last_undegraded":"2024-04-20T11:22:36.114321+0000","last_fullsized":"2024-04-20T11:22:36.114321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'9","reported_seq":184,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.424523+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:22:36.424523+0000","last_peered":"2024-04-20T11:22:36.424523+0000","last_clean":"2024-04-20T11:22:36.424523+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:22:36.424523+0000","last_undegraded":"2024-04-20T11:22:36.424523+0000","last_fullsized":"2024-04-20T11:22:36.424523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":5,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114107+0000","last_change":"2024-04-20T11:20:51.223843+0000","last_active":"2024-04-20T11:22:36.114107+0000","last_peered":"2024-04-20T11:22:36.114107+0000","last_clean":"2024-04-20T11:22:36.114107+0000","last_became_active":"2024-04-20T11:20:51.223311+0000","last_became_peered":"2024-04-20T11:20:51.223311+0000","last_unstale":"2024-04-20T11:22:36.114107+0000","last_undegraded":"2024-04-20T11:22:36.114107+0000","last_fullsized":"2024-04-20T11:22:36.114107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:38.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101882+0000","last_change":"2024-04-20T11:20:51.236281+0000","last_active":"2024-04-20T11:22:36.101882+0000","last_peered":"2024-04-20T11:22:36.101882+0000","last_clean":"2024-04-20T11:22:36.101882+0000","last_became_active":"2024-04-20T11:20:51.222316+0000","last_became_peered":"2024-04-20T11:20:51.222316+0000","last_unstale":"2024-04-20T11:22:36.101882+0000","last_undegraded":"2024-04-20T11:22:36.101882+0000","last_fullsized":"2024-04-20T11:22:36.101882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169128+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:22:36.169128+0000","last_peered":"2024-04-20T11:22:36.169128+0000","last_clean":"2024-04-20T11:22:36.169128+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:22:36.169128+0000","last_undegraded":"2024-04-20T11:22:36.169128+0000","last_fullsized":"2024-04-20T11:22:36.169128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.170084+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:22:36.170084+0000","last_peered":"2024-04-20T11:22:36.170084+0000","last_clean":"2024-04-20T11:22:36.170084+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:22:36.170084+0000","last_undegraded":"2024-04-20T11:22:36.170084+0000","last_fullsized":"2024-04-20T11:22:36.170084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102064+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:22:36.102064+0000","last_peered":"2024-04-20T11:22:36.102064+0000","last_clean":"2024-04-20T11:22:36.102064+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:22:36.102064+0000","last_undegraded":"2024-04-20T11:22:36.102064+0000","last_fullsized":"2024-04-20T11:22:36.102064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.104397+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:22:36.104397+0000","last_peered":"2024-04-20T11:22:36.104397+0000","last_clean":"2024-04-20T11:22:36.104397+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:22:36.104397+0000","last_undegraded":"2024-04-20T11:22:36.104397+0000","last_fullsized":"2024-04-20T11:22:36.104397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151784+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:22:36.151784+0000","last_peered":"2024-04-20T11:22:36.151784+0000","last_clean":"2024-04-20T11:22:36.151784+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:22:36.151784+0000","last_undegraded":"2024-04-20T11:22:36.151784+0000","last_fullsized":"2024-04-20T11:22:36.151784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":178,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.120190+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:22:37.120190+0000","last_peered":"2024-04-20T11:22:37.120190+0000","last_clean":"2024-04-20T11:22:37.120190+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:22:37.120190+0000","last_undegraded":"2024-04-20T11:22:37.120190+0000","last_fullsized":"2024-04-20T11:22:37.120190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151591+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:22:36.151591+0000","last_peered":"2024-04-20T11:22:36.151591+0000","last_clean":"2024-04-20T11:22:36.151591+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:22:36.151591+0000","last_undegraded":"2024-04-20T11:22:36.151591+0000","last_fullsized":"2024-04-20T11:22:36.151591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169048+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:22:36.169048+0000","last_peered":"2024-04-20T11:22:36.169048+0000","last_clean":"2024-04-20T11:22:36.169048+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:22:36.169048+0000","last_undegraded":"2024-04-20T11:22:36.169048+0000","last_fullsized":"2024-04-20T11:22:36.169048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.426373+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:22:36.426373+0000","last_peered":"2024-04-20T11:22:36.426373+0000","last_clean":"2024-04-20T11:22:36.426373+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:22:36.426373+0000","last_undegraded":"2024-04-20T11:22:36.426373+0000","last_fullsized":"2024-04-20T11:22:36.426373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":151,"num_read_kb":151,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137260+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:22:36.137260+0000","last_peered":"2024-04-20T11:22:36.137260+0000","last_clean":"2024-04-20T11:22:36.137260+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:22:36.137260+0000","last_undegraded":"2024-04-20T11:22:36.137260+0000","last_fullsized":"2024-04-20T11:22:36.137260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102418+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:22:36.102418+0000","last_peered":"2024-04-20T11:22:36.102418+0000","last_clean":"2024-04-20T11:22:36.102418+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:22:36.102418+0000","last_undegraded":"2024-04-20T11:22:36.102418+0000","last_fullsized":"2024-04-20T11:22:36.102418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114180+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:22:36.114180+0000","last_peered":"2024-04-20T11:22:36.114180+0000","last_clean":"2024-04-20T11:22:36.114180+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:22:36.114180+0000","last_undegraded":"2024-04-20T11:22:36.114180+0000","last_fullsized":"2024-04-20T11:22:36.114180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134177+0000","last_change":"2024-04-20T11:20:51.224226+0000","last_active":"2024-04-20T11:22:36.134177+0000","last_peered":"2024-04-20T11:22:36.134177+0000","last_clean":"2024-04-20T11:22:36.134177+0000","last_became_active":"2024-04-20T11:20:51.223728+0000","last_became_peered":"2024-04-20T11:20:51.223728+0000","last_unstale":"2024-04-20T11:22:36.134177+0000","last_undegraded":"2024-04-20T11:22:36.134177+0000","last_fullsized":"2024-04-20T11:22:36.134177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:37.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114398+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:22:36.114398+0000","last_peered":"2024-04-20T11:22:36.114398+0000","last_clean":"2024-04-20T11:22:36.114398+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:22:36.114398+0000","last_undegraded":"2024-04-20T11:22:36.114398+0000","last_fullsized":"2024-04-20T11:22:36.114398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169785+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:22:36.169785+0000","last_peered":"2024-04-20T11:22:36.169785+0000","last_clean":"2024-04-20T11:22:36.169785+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:22:36.169785+0000","last_undegraded":"2024-04-20T11:22:36.169785+0000","last_fullsized":"2024-04-20T11:22:36.169785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134405+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:22:36.134405+0000","last_peered":"2024-04-20T11:22:36.134405+0000","last_clean":"2024-04-20T11:22:36.134405+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:22:36.134405+0000","last_undegraded":"2024-04-20T11:22:36.134405+0000","last_fullsized":"2024-04-20T11:22:36.134405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.366090+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:22:36.366090+0000","last_peered":"2024-04-20T11:22:36.366090+0000","last_clean":"2024-04-20T11:22:36.366090+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:22:36.366090+0000","last_undegraded":"2024-04-20T11:22:36.366090+0000","last_fullsized":"2024-04-20T11:22:36.366090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137464+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:22:36.137464+0000","last_peered":"2024-04-20T11:22:36.137464+0000","last_clean":"2024-04-20T11:22:36.137464+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:22:36.137464+0000","last_undegraded":"2024-04-20T11:22:36.137464+0000","last_fullsized":"2024-04-20T11:22:36.137464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102357+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:22:36.102357+0000","last_peered":"2024-04-20T11:22:36.102357+0000","last_clean":"2024-04-20T11:22:36.102357+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:22:36.102357+0000","last_undegraded":"2024-04-20T11:22:36.102357+0000","last_fullsized":"2024-04-20T11:22:36.102357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169611+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:22:36.169611+0000","last_peered":"2024-04-20T11:22:36.169611+0000","last_clean":"2024-04-20T11:22:36.169611+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:22:36.169611+0000","last_undegraded":"2024-04-20T11:22:36.169611+0000","last_fullsized":"2024-04-20T11:22:36.169611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168896+0000","last_change":"2024-04-20T11:20:51.225236+0000","last_active":"2024-04-20T11:22:36.168896+0000","last_peered":"2024-04-20T11:22:36.168896+0000","last_clean":"2024-04-20T11:22:36.168896+0000","last_became_active":"2024-04-20T11:20:51.221635+0000","last_became_peered":"2024-04-20T11:20:51.221635+0000","last_unstale":"2024-04-20T11:22:36.168896+0000","last_undegraded":"2024-04-20T11:22:36.168896+0000","last_fullsized":"2024-04-20T11:22:36.168896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:03.894361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169171+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:22:36.169171+0000","last_peered":"2024-04-20T11:22:36.169171+0000","last_clean":"2024-04-20T11:22:36.169171+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:22:36.169171+0000","last_undegraded":"2024-04-20T11:22:36.169171+0000","last_fullsized":"2024-04-20T11:22:36.169171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137112+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:22:36.137112+0000","last_peered":"2024-04-20T11:22:36.137112+0000","last_clean":"2024-04-20T11:22:36.137112+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:22:36.137112+0000","last_undegraded":"2024-04-20T11:22:36.137112+0000","last_fullsized":"2024-04-20T11:22:36.137112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101804+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:22:36.101804+0000","last_peered":"2024-04-20T11:22:36.101804+0000","last_clean":"2024-04-20T11:22:36.101804+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:22:36.101804+0000","last_undegraded":"2024-04-20T11:22:36.101804+0000","last_fullsized":"2024-04-20T11:22:36.101804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154030+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:22:36.154030+0000","last_peered":"2024-04-20T11:22:36.154030+0000","last_clean":"2024-04-20T11:22:36.154030+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:22:36.154030+0000","last_undegraded":"2024-04-20T11:22:36.154030+0000","last_fullsized":"2024-04-20T11:22:36.154030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137375+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:22:36.137375+0000","last_peered":"2024-04-20T11:22:36.137375+0000","last_clean":"2024-04-20T11:22:36.137375+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:22:36.137375+0000","last_undegraded":"2024-04-20T11:22:36.137375+0000","last_fullsized":"2024-04-20T11:22:36.137375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169506+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:22:36.169506+0000","last_peered":"2024-04-20T11:22:36.169506+0000","last_clean":"2024-04-20T11:22:36.169506+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:22:36.169506+0000","last_undegraded":"2024-04-20T11:22:36.169506+0000","last_fullsized":"2024-04-20T11:22:36.169506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114559+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:22:36.114559+0000","last_peered":"2024-04-20T11:22:36.114559+0000","last_clean":"2024-04-20T11:22:36.114559+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:22:36.114559+0000","last_undegraded":"2024-04-20T11:22:36.114559+0000","last_fullsized":"2024-04-20T11:22:36.114559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'8","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154545+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:22:36.154545+0000","last_peered":"2024-04-20T11:22:36.154545+0000","last_clean":"2024-04-20T11:22:36.154545+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:22:36.154545+0000","last_undegraded":"2024-04-20T11:22:36.154545+0000","last_fullsized":"2024-04-20T11:22:36.154545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114465+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:22:36.114465+0000","last_peered":"2024-04-20T11:22:36.114465+0000","last_clean":"2024-04-20T11:22:36.114465+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:22:36.114465+0000","last_undegraded":"2024-04-20T11:22:36.114465+0000","last_fullsized":"2024-04-20T11:22:36.114465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.363029+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:22:36.363029+0000","last_peered":"2024-04-20T11:22:36.363029+0000","last_clean":"2024-04-20T11:22:36.363029+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:22:36.363029+0000","last_undegraded":"2024-04-20T11:22:36.363029+0000","last_fullsized":"2024-04-20T11:22:36.363029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114342+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:22:36.114342+0000","last_peered":"2024-04-20T11:22:36.114342+0000","last_clean":"2024-04-20T11:22:36.114342+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:22:36.114342+0000","last_undegraded":"2024-04-20T11:22:36.114342+0000","last_fullsized":"2024-04-20T11:22:36.114342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134792+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:22:36.134792+0000","last_peered":"2024-04-20T11:22:36.134792+0000","last_clean":"2024-04-20T11:22:36.134792+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:22:36.134792+0000","last_undegraded":"2024-04-20T11:22:36.134792+0000","last_fullsized":"2024-04-20T11:22:36.134792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114458+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:22:36.114458+0000","last_peered":"2024-04-20T11:22:36.114458+0000","last_clean":"2024-04-20T11:22:36.114458+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:22:36.114458+0000","last_undegraded":"2024-04-20T11:22:36.114458+0000","last_fullsized":"2024-04-20T11:22:36.114458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101975+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:22:36.101975+0000","last_peered":"2024-04-20T11:22:36.101975+0000","last_clean":"2024-04-20T11:22:36.101975+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:22:36.101975+0000","last_undegraded":"2024-04-20T11:22:36.101975+0000","last_fullsized":"2024-04-20T11:22:36.101975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154063+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:22:36.154063+0000","last_peered":"2024-04-20T11:22:36.154063+0000","last_clean":"2024-04-20T11:22:36.154063+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:22:36.154063+0000","last_undegraded":"2024-04-20T11:22:36.154063+0000","last_fullsized":"2024-04-20T11:22:36.154063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154017+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:22:36.154017+0000","last_peered":"2024-04-20T11:22:36.154017+0000","last_clean":"2024-04-20T11:22:36.154017+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:22:36.154017+0000","last_undegraded":"2024-04-20T11:22:36.154017+0000","last_fullsized":"2024-04-20T11:22:36.154017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169323+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:22:36.169323+0000","last_peered":"2024-04-20T11:22:36.169323+0000","last_clean":"2024-04-20T11:22:36.169323+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:22:36.169323+0000","last_undegraded":"2024-04-20T11:22:36.169323+0000","last_fullsized":"2024-04-20T11:22:36.169323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134952+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:22:36.134952+0000","last_peered":"2024-04-20T11:22:36.134952+0000","last_clean":"2024-04-20T11:22:36.134952+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:22:36.134952+0000","last_undegraded":"2024-04-20T11:22:36.134952+0000","last_fullsized":"2024-04-20T11:22:36.134952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101932+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:22:36.101932+0000","last_peered":"2024-04-20T11:22:36.101932+0000","last_clean":"2024-04-20T11:22:36.101932+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:22:36.101932+0000","last_undegraded":"2024-04-20T11:22:36.101932+0000","last_fullsized":"2024-04-20T11:22:36.101932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154450+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:22:36.154450+0000","last_peered":"2024-04-20T11:22:36.154450+0000","last_clean":"2024-04-20T11:22:36.154450+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:22:36.154450+0000","last_undegraded":"2024-04-20T11:22:36.154450+0000","last_fullsized":"2024-04-20T11:22:36.154450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137439+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:22:36.137439+0000","last_peered":"2024-04-20T11:22:36.137439+0000","last_clean":"2024-04-20T11:22:36.137439+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:22:36.137439+0000","last_undegraded":"2024-04-20T11:22:36.137439+0000","last_fullsized":"2024-04-20T11:22:36.137439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169002+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:22:36.169002+0000","last_peered":"2024-04-20T11:22:36.169002+0000","last_clean":"2024-04-20T11:22:36.169002+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:22:36.169002+0000","last_undegraded":"2024-04-20T11:22:36.169002+0000","last_fullsized":"2024-04-20T11:22:36.169002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137123+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:22:36.137123+0000","last_peered":"2024-04-20T11:22:36.137123+0000","last_clean":"2024-04-20T11:22:36.137123+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:22:36.137123+0000","last_undegraded":"2024-04-20T11:22:36.137123+0000","last_fullsized":"2024-04-20T11:22:36.137123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168939+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:22:36.168939+0000","last_peered":"2024-04-20T11:22:36.168939+0000","last_clean":"2024-04-20T11:22:36.168939+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:22:36.168939+0000","last_undegraded":"2024-04-20T11:22:36.168939+0000","last_fullsized":"2024-04-20T11:22:36.168939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134401+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:22:36.134401+0000","last_peered":"2024-04-20T11:22:36.134401+0000","last_clean":"2024-04-20T11:22:36.134401+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:22:36.134401+0000","last_undegraded":"2024-04-20T11:22:36.134401+0000","last_fullsized":"2024-04-20T11:22:36.134401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101694+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:22:36.101694+0000","last_peered":"2024-04-20T11:22:36.101694+0000","last_clean":"2024-04-20T11:22:36.101694+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:22:36.101694+0000","last_undegraded":"2024-04-20T11:22:36.101694+0000","last_fullsized":"2024-04-20T11:22:36.101694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107634,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599030,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1624,"num_read_kb":925,"num_write":368,"num_write_kb":1461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":700416,"data_stored":1797090,"data_compressed":81420,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":466545,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645361,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1254617,"data_compressed":46779,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":858656,"data_compressed":22053,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":858922,"data_compressed":22053,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1059650,"data_compressed":38785,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":465157,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":531625,"data_compressed":4745,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":467419,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":532377,"data_compressed":4745,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":989798,"data_compressed":28209,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":466332,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1189223,"data_compressed":44941,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67307,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68059,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":199814,"data_compressed":7994,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:22:42.339 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-20T11:22:42.340 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:22:42.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 -- 172.21.15.44:0/465090983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181041e0 msgr2=0x7ff318104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 --2- 172.21.15.44:0/465090983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181041e0 0x7ff318104600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff308009900 tx=0x7ff30802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:42.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 -- 172.21.15.44:0/465090983 shutdown_connections 2024-04-20T11:22:42.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 --2- 172.21.15.44:0/465090983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff318105d90 0x7ff31810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 --2- 172.21.15.44:0/465090983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181053d0 0x7ff318105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 --2- 172.21.15.44:0/465090983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181041e0 0x7ff318104600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.585+0000 7ff31d400640 1 -- 172.21.15.44:0/465090983 >> 172.21.15.44:0/465090983 conn(0x7ff3180ff750 msgr2=0x7ff318101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:42.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.586+0000 7ff31d400640 1 -- 172.21.15.44:0/465090983 shutdown_connections 2024-04-20T11:22:42.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.586+0000 7ff31d400640 1 -- 172.21.15.44:0/465090983 wait complete. 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.587+0000 7ff31d400640 1 Processor -- start 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.587+0000 7ff31d400640 1 -- start start 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.587+0000 7ff31d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 0x7ff31819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.587+0000 7ff31d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 0x7ff31819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff31d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff318105d90 0x7ff3181a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff31d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff31810ff40 con 0x7ff3181041e0 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff31d400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff31810fdc0 con 0x7ff3181053d0 2024-04-20T11:22:42.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff31d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181100c0 con 0x7ff318105d90 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff30fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 0x7ff31819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 0x7ff31819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 0x7ff31819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50184/0 (socket says 172.21.15.44:50184) 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff30fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 0x7ff31819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36814/0 (socket says 172.21.15.44:36814) 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 -- 172.21.15.44:0/3325397932 learned_addr learned my addr 172.21.15.44:0/3325397932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff317200640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff318105d90 0x7ff3181a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 -- 172.21.15.44:0/3325397932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff318105d90 msgr2=0x7ff3181a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff318105d90 0x7ff3181a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 -- 172.21.15.44:0/3325397932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 msgr2=0x7ff31819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 0x7ff31819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff316800640 1 -- 172.21.15.44:0/3325397932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3181a21d0 con 0x7ff3181041e0 2024-04-20T11:22:42.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.588+0000 7ff30fe00640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 0x7ff31819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:42.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.589+0000 7ff316800640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 0x7ff31819a5b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff308000c00 tx=0x7ff308004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:42.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.589+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308043020 con 0x7ff3181041e0 2024-04-20T11:22:42.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.589+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff308004250 con 0x7ff3181041e0 2024-04-20T11:22:42.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.589+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308003b90 con 0x7ff3181041e0 2024-04-20T11:22:42.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.589+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181a2460 con 0x7ff3181041e0 2024-04-20T11:22:42.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.589+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a2960 con 0x7ff3181041e0 2024-04-20T11:22:42.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.590+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31810b500 con 0x7ff3181041e0 2024-04-20T11:22:42.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.593+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7ff308046020 con 0x7ff3181041e0 2024-04-20T11:22:42.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.594+0000 7ff30f400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff2fc0a3ec0 0x7ff2fc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:42.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.594+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff3080f2980 con 0x7ff3181041e0 2024-04-20T11:22:42.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.595+0000 7ff30fe00640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff2fc0a3ec0 0x7ff2fc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:42.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.595+0000 7ff30fe00640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff2fc0a3ec0 0x7ff2fc0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff3000095f0 tx=0x7ff300005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.599+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff30804d020 con 0x7ff3181041e0 2024-04-20T11:22:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.807+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3180623a0 con 0x7ff2fc0a3ec0 2024-04-20T11:22:42.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.815+0000 7ff30f400640 1 -- 172.21.15.44:0/3325397932 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7ff3180623a0 con 0x7ff2fc0a3ec0 2024-04-20T11:22:42.819 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:42.820 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T11:22:42.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff2fc0a3ec0 msgr2=0x7ff2fc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff2fc0a3ec0 0x7ff2fc0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff3000095f0 tx=0x7ff300005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 msgr2=0x7ff31819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 0x7ff31819a5b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff308000c00 tx=0x7ff308004a30 comp rx=0 tx=0).stop 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 shutdown_connections 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff318105d90 0x7ff3181a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3181053d0 0x7ff31819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff2fc0a3ec0 0x7ff2fc0a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 --2- 172.21.15.44:0/3325397932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3181041e0 0x7ff31819a5b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 >> 172.21.15.44:0/3325397932 conn(0x7ff3180ff750 msgr2=0x7ff3181086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 shutdown_connections 2024-04-20T11:22:42.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:42.820+0000 7ff31d400640 1 -- 172.21.15.44:0/3325397932 wait complete. 2024-04-20T11:22:42.838 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-20T11:22:42.122373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189632,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1760,"num_read_kb":1044,"num_write":551,"num_write_kb":4401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":936,"ondisk_log_size":936,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361172,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580652,"statfs":{"total":1151940427776,"available":1151570587648,"internally_reserved":0,"allocated":12275712,"data_stored":9140321,"data_compressed":223938,"data_compressed_allocated":294912,"data_compressed_original":3440640,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.969852"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102181+0000","last_change":"2024-04-20T11:20:02.710006+0000","last_active":"2024-04-20T11:22:36.102181+0000","last_peered":"2024-04-20T11:22:36.102181+0000","last_clean":"2024-04-20T11:22:36.102181+0000","last_became_active":"2024-04-20T11:20:02.709610+0000","last_became_peered":"2024-04-20T11:20:02.709610+0000","last_unstale":"2024-04-20T11:22:36.102181+0000","last_undegraded":"2024-04-20T11:22:36.102181+0000","last_fullsized":"2024-04-20T11:22:36.102181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:14:27.221681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169564+0000","last_change":"2024-04-20T11:20:02.704096+0000","last_active":"2024-04-20T11:22:36.169564+0000","last_peered":"2024-04-20T11:22:36.169564+0000","last_clean":"2024-04-20T11:22:36.169564+0000","last_became_active":"2024-04-20T11:20:02.702413+0000","last_became_peered":"2024-04-20T11:20:02.702413+0000","last_unstale":"2024-04-20T11:22:36.169564+0000","last_undegraded":"2024-04-20T11:22:36.169564+0000","last_fullsized":"2024-04-20T11:22:36.169564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:08.744690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169452+0000","last_change":"2024-04-20T11:20:02.706663+0000","last_active":"2024-04-20T11:22:36.169452+0000","last_peered":"2024-04-20T11:22:36.169452+0000","last_clean":"2024-04-20T11:22:36.169452+0000","last_became_active":"2024-04-20T11:20:02.706263+0000","last_became_peered":"2024-04-20T11:20:02.706263+0000","last_unstale":"2024-04-20T11:22:36.169452+0000","last_undegraded":"2024-04-20T11:22:36.169452+0000","last_fullsized":"2024-04-20T11:22:36.169452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:39.973092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102312+0000","last_change":"2024-04-20T11:20:02.692600+0000","last_active":"2024-04-20T11:22:36.102312+0000","last_peered":"2024-04-20T11:22:36.102312+0000","last_clean":"2024-04-20T11:22:36.102312+0000","last_became_active":"2024-04-20T11:20:02.692279+0000","last_became_peered":"2024-04-20T11:20:02.692279+0000","last_unstale":"2024-04-20T11:22:36.102312+0000","last_undegraded":"2024-04-20T11:22:36.102312+0000","last_fullsized":"2024-04-20T11:22:36.102312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:42.270203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169679+0000","last_change":"2024-04-20T11:20:02.715239+0000","last_active":"2024-04-20T11:22:36.169679+0000","last_peered":"2024-04-20T11:22:36.169679+0000","last_clean":"2024-04-20T11:22:36.169679+0000","last_became_active":"2024-04-20T11:20:02.714802+0000","last_became_peered":"2024-04-20T11:20:02.714802+0000","last_unstale":"2024-04-20T11:22:36.169679+0000","last_undegraded":"2024-04-20T11:22:36.169679+0000","last_fullsized":"2024-04-20T11:22:36.169679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:51.437848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151691+0000","last_change":"2024-04-20T11:20:02.708401+0000","last_active":"2024-04-20T11:22:36.151691+0000","last_peered":"2024-04-20T11:22:36.151691+0000","last_clean":"2024-04-20T11:22:36.151691+0000","last_became_active":"2024-04-20T11:20:02.706828+0000","last_became_peered":"2024-04-20T11:20:02.706828+0000","last_unstale":"2024-04-20T11:22:36.151691+0000","last_undegraded":"2024-04-20T11:22:36.151691+0000","last_fullsized":"2024-04-20T11:22:36.151691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:40:13.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114218+0000","last_change":"2024-04-20T11:20:02.695059+0000","last_active":"2024-04-20T11:22:36.114218+0000","last_peered":"2024-04-20T11:22:36.114218+0000","last_clean":"2024-04-20T11:22:36.114218+0000","last_became_active":"2024-04-20T11:20:02.694715+0000","last_became_peered":"2024-04-20T11:20:02.694715+0000","last_unstale":"2024-04-20T11:22:36.114218+0000","last_undegraded":"2024-04-20T11:22:36.114218+0000","last_fullsized":"2024-04-20T11:22:36.114218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:11:19.949707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137124+0000","last_change":"2024-04-20T11:20:02.700480+0000","last_active":"2024-04-20T11:22:36.137124+0000","last_peered":"2024-04-20T11:22:36.137124+0000","last_clean":"2024-04-20T11:22:36.137124+0000","last_became_active":"2024-04-20T11:20:02.700141+0000","last_became_peered":"2024-04-20T11:20:02.700141+0000","last_unstale":"2024-04-20T11:22:36.137124+0000","last_undegraded":"2024-04-20T11:22:36.137124+0000","last_fullsized":"2024-04-20T11:22:36.137124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:58.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169305+0000","last_change":"2024-04-20T11:20:02.687034+0000","last_active":"2024-04-20T11:22:36.169305+0000","last_peered":"2024-04-20T11:22:36.169305+0000","last_clean":"2024-04-20T11:22:36.169305+0000","last_became_active":"2024-04-20T11:20:02.686041+0000","last_became_peered":"2024-04-20T11:20:02.686041+0000","last_unstale":"2024-04-20T11:22:36.169305+0000","last_undegraded":"2024-04-20T11:22:36.169305+0000","last_fullsized":"2024-04-20T11:22:36.169305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:25.587777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151953+0000","last_change":"2024-04-20T11:20:51.223729+0000","last_active":"2024-04-20T11:22:36.151953+0000","last_peered":"2024-04-20T11:22:36.151953+0000","last_clean":"2024-04-20T11:22:36.151953+0000","last_became_active":"2024-04-20T11:20:51.223340+0000","last_became_peered":"2024-04-20T11:20:51.223340+0000","last_unstale":"2024-04-20T11:22:36.151953+0000","last_undegraded":"2024-04-20T11:22:36.151953+0000","last_fullsized":"2024-04-20T11:22:36.151953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:10:13.728111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114424+0000","last_change":"2024-04-20T11:20:02.693807+0000","last_active":"2024-04-20T11:22:36.114424+0000","last_peered":"2024-04-20T11:22:36.114424+0000","last_clean":"2024-04-20T11:22:36.114424+0000","last_became_active":"2024-04-20T11:20:02.693510+0000","last_became_peered":"2024-04-20T11:20:02.693510+0000","last_unstale":"2024-04-20T11:22:36.114424+0000","last_undegraded":"2024-04-20T11:22:36.114424+0000","last_fullsized":"2024-04-20T11:22:36.114424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:41:04.963576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102257+0000","last_change":"2024-04-20T11:20:02.695941+0000","last_active":"2024-04-20T11:22:36.102257+0000","last_peered":"2024-04-20T11:22:36.102257+0000","last_clean":"2024-04-20T11:22:36.102257+0000","last_became_active":"2024-04-20T11:20:02.695519+0000","last_became_peered":"2024-04-20T11:20:02.695519+0000","last_unstale":"2024-04-20T11:22:36.102257+0000","last_undegraded":"2024-04-20T11:22:36.102257+0000","last_fullsized":"2024-04-20T11:22:36.102257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:18.782956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134653+0000","last_change":"2024-04-20T11:20:02.705496+0000","last_active":"2024-04-20T11:22:36.134653+0000","last_peered":"2024-04-20T11:22:36.134653+0000","last_clean":"2024-04-20T11:22:36.134653+0000","last_became_active":"2024-04-20T11:20:02.696243+0000","last_became_peered":"2024-04-20T11:20:02.696243+0000","last_unstale":"2024-04-20T11:22:36.134653+0000","last_undegraded":"2024-04-20T11:22:36.134653+0000","last_fullsized":"2024-04-20T11:22:36.134653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:24.186318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169165+0000","last_change":"2024-04-20T11:20:02.718438+0000","last_active":"2024-04-20T11:22:36.169165+0000","last_peered":"2024-04-20T11:22:36.169165+0000","last_clean":"2024-04-20T11:22:36.169165+0000","last_became_active":"2024-04-20T11:20:02.709943+0000","last_became_peered":"2024-04-20T11:20:02.709943+0000","last_unstale":"2024-04-20T11:22:36.169165+0000","last_undegraded":"2024-04-20T11:22:36.169165+0000","last_fullsized":"2024-04-20T11:22:36.169165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:44.656336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169119+0000","last_change":"2024-04-20T11:20:02.712986+0000","last_active":"2024-04-20T11:22:36.169119+0000","last_peered":"2024-04-20T11:22:36.169119+0000","last_clean":"2024-04-20T11:22:36.169119+0000","last_became_active":"2024-04-20T11:20:02.711979+0000","last_became_peered":"2024-04-20T11:20:02.711979+0000","last_unstale":"2024-04-20T11:22:36.169119+0000","last_undegraded":"2024-04-20T11:22:36.169119+0000","last_fullsized":"2024-04-20T11:22:36.169119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:47.493088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168825+0000","last_change":"2024-04-20T11:20:02.708396+0000","last_active":"2024-04-20T11:22:36.168825+0000","last_peered":"2024-04-20T11:22:36.168825+0000","last_clean":"2024-04-20T11:22:36.168825+0000","last_became_active":"2024-04-20T11:20:02.708090+0000","last_became_peered":"2024-04-20T11:20:02.708090+0000","last_unstale":"2024-04-20T11:22:36.168825+0000","last_undegraded":"2024-04-20T11:22:36.168825+0000","last_fullsized":"2024-04-20T11:22:36.168825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:45.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102334+0000","last_change":"2024-04-20T11:20:02.709902+0000","last_active":"2024-04-20T11:22:36.102334+0000","last_peered":"2024-04-20T11:22:36.102334+0000","last_clean":"2024-04-20T11:22:36.102334+0000","last_became_active":"2024-04-20T11:20:02.709599+0000","last_became_peered":"2024-04-20T11:20:02.709599+0000","last_unstale":"2024-04-20T11:22:36.102334+0000","last_undegraded":"2024-04-20T11:22:36.102334+0000","last_fullsized":"2024-04-20T11:22:36.102334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:29:14.946393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.104636+0000","last_change":"2024-04-20T11:20:02.693188+0000","last_active":"2024-04-20T11:22:36.104636+0000","last_peered":"2024-04-20T11:22:36.104636+0000","last_clean":"2024-04-20T11:22:36.104636+0000","last_became_active":"2024-04-20T11:20:02.692774+0000","last_became_peered":"2024-04-20T11:20:02.692774+0000","last_unstale":"2024-04-20T11:22:36.104636+0000","last_undegraded":"2024-04-20T11:22:36.104636+0000","last_fullsized":"2024-04-20T11:22:36.104636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:14.183407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154198+0000","last_change":"2024-04-20T11:20:02.719819+0000","last_active":"2024-04-20T11:22:36.154198+0000","last_peered":"2024-04-20T11:22:36.154198+0000","last_clean":"2024-04-20T11:22:36.154198+0000","last_became_active":"2024-04-20T11:20:02.719556+0000","last_became_peered":"2024-04-20T11:20:02.719556+0000","last_unstale":"2024-04-20T11:22:36.154198+0000","last_undegraded":"2024-04-20T11:22:36.154198+0000","last_fullsized":"2024-04-20T11:22:36.154198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:57.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":185,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.429408+0000","last_change":"2024-04-20T11:20:01.670344+0000","last_active":"2024-04-20T11:22:36.429408+0000","last_peered":"2024-04-20T11:22:36.429408+0000","last_clean":"2024-04-20T11:22:36.429408+0000","last_became_active":"2024-04-20T11:20:01.669982+0000","last_became_peered":"2024-04-20T11:20:01.669982+0000","last_unstale":"2024-04-20T11:22:36.429408+0000","last_undegraded":"2024-04-20T11:22:36.429408+0000","last_fullsized":"2024-04-20T11:22:36.429408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:33:44.880495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137554+0000","last_change":"2024-04-20T11:20:02.721352+0000","last_active":"2024-04-20T11:22:36.137554+0000","last_peered":"2024-04-20T11:22:36.137554+0000","last_clean":"2024-04-20T11:22:36.137554+0000","last_became_active":"2024-04-20T11:20:02.720833+0000","last_became_peered":"2024-04-20T11:20:02.720833+0000","last_unstale":"2024-04-20T11:22:36.137554+0000","last_undegraded":"2024-04-20T11:22:36.137554+0000","last_fullsized":"2024-04-20T11:22:36.137554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:57.217947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137621+0000","last_change":"2024-04-20T11:20:01.681969+0000","last_active":"2024-04-20T11:22:36.137621+0000","last_peered":"2024-04-20T11:22:36.137621+0000","last_clean":"2024-04-20T11:22:36.137621+0000","last_became_active":"2024-04-20T11:20:01.681615+0000","last_became_peered":"2024-04-20T11:20:01.681615+0000","last_unstale":"2024-04-20T11:22:36.137621+0000","last_undegraded":"2024-04-20T11:22:36.137621+0000","last_fullsized":"2024-04-20T11:22:36.137621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:48:54.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169732+0000","last_change":"2024-04-20T11:20:02.710471+0000","last_active":"2024-04-20T11:22:36.169732+0000","last_peered":"2024-04-20T11:22:36.169732+0000","last_clean":"2024-04-20T11:22:36.169732+0000","last_became_active":"2024-04-20T11:20:02.710101+0000","last_became_peered":"2024-04-20T11:20:02.710101+0000","last_unstale":"2024-04-20T11:22:36.169732+0000","last_undegraded":"2024-04-20T11:22:36.169732+0000","last_fullsized":"2024-04-20T11:22:36.169732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:29:47.462005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'13","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.870292+0000","last_change":"2024-04-20T11:20:01.654026+0000","last_active":"2024-04-20T11:22:36.870292+0000","last_peered":"2024-04-20T11:22:36.870292+0000","last_clean":"2024-04-20T11:22:36.870292+0000","last_became_active":"2024-04-20T11:20:01.653110+0000","last_became_peered":"2024-04-20T11:20:01.653110+0000","last_unstale":"2024-04-20T11:22:36.870292+0000","last_undegraded":"2024-04-20T11:22:36.870292+0000","last_fullsized":"2024-04-20T11:22:36.870292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:20.163476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169025+0000","last_change":"2024-04-20T11:20:02.695781+0000","last_active":"2024-04-20T11:22:36.169025+0000","last_peered":"2024-04-20T11:22:36.169025+0000","last_clean":"2024-04-20T11:22:36.169025+0000","last_became_active":"2024-04-20T11:20:02.695485+0000","last_became_peered":"2024-04-20T11:20:02.695485+0000","last_unstale":"2024-04-20T11:22:36.169025+0000","last_undegraded":"2024-04-20T11:22:36.169025+0000","last_fullsized":"2024-04-20T11:22:36.169025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:46:59.272158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.170006+0000","last_change":"2024-04-20T11:20:01.653772+0000","last_active":"2024-04-20T11:22:36.170006+0000","last_peered":"2024-04-20T11:22:36.170006+0000","last_clean":"2024-04-20T11:22:36.170006+0000","last_became_active":"2024-04-20T11:20:01.653360+0000","last_became_peered":"2024-04-20T11:20:01.653360+0000","last_unstale":"2024-04-20T11:22:36.170006+0000","last_undegraded":"2024-04-20T11:22:36.170006+0000","last_fullsized":"2024-04-20T11:22:36.170006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:58.166689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169858+0000","last_change":"2024-04-20T11:20:02.688257+0000","last_active":"2024-04-20T11:22:36.169858+0000","last_peered":"2024-04-20T11:22:36.169858+0000","last_clean":"2024-04-20T11:22:36.169858+0000","last_became_active":"2024-04-20T11:20:02.687962+0000","last_became_peered":"2024-04-20T11:20:02.687962+0000","last_unstale":"2024-04-20T11:22:36.169858+0000","last_undegraded":"2024-04-20T11:22:36.169858+0000","last_fullsized":"2024-04-20T11:22:36.169858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":179,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.008672+0000","last_change":"2024-04-20T11:20:01.655796+0000","last_active":"2024-04-20T11:22:37.008672+0000","last_peered":"2024-04-20T11:22:37.008672+0000","last_clean":"2024-04-20T11:22:37.008672+0000","last_became_active":"2024-04-20T11:20:01.655479+0000","last_became_peered":"2024-04-20T11:20:01.655479+0000","last_unstale":"2024-04-20T11:22:37.008672+0000","last_undegraded":"2024-04-20T11:22:37.008672+0000","last_fullsized":"2024-04-20T11:22:37.008672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:38:09.521191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169420+0000","last_change":"2024-04-20T11:20:02.693254+0000","last_active":"2024-04-20T11:22:36.169420+0000","last_peered":"2024-04-20T11:22:36.169420+0000","last_clean":"2024-04-20T11:22:36.169420+0000","last_became_active":"2024-04-20T11:20:02.692933+0000","last_became_peered":"2024-04-20T11:20:02.692933+0000","last_unstale":"2024-04-20T11:22:36.169420+0000","last_undegraded":"2024-04-20T11:22:36.169420+0000","last_fullsized":"2024-04-20T11:22:36.169420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:42:44.112716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151597+0000","last_change":"2024-04-20T11:20:01.662411+0000","last_active":"2024-04-20T11:22:36.151597+0000","last_peered":"2024-04-20T11:22:36.151597+0000","last_clean":"2024-04-20T11:22:36.151597+0000","last_became_active":"2024-04-20T11:20:01.662060+0000","last_became_peered":"2024-04-20T11:20:01.662060+0000","last_unstale":"2024-04-20T11:22:36.151597+0000","last_undegraded":"2024-04-20T11:22:36.151597+0000","last_fullsized":"2024-04-20T11:22:36.151597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.834089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169629+0000","last_change":"2024-04-20T11:20:51.505789+0000","last_active":"2024-04-20T11:22:36.169629+0000","last_peered":"2024-04-20T11:22:36.169629+0000","last_clean":"2024-04-20T11:22:36.169629+0000","last_became_active":"2024-04-20T11:20:51.505449+0000","last_became_peered":"2024-04-20T11:20:51.505449+0000","last_unstale":"2024-04-20T11:22:36.169629+0000","last_undegraded":"2024-04-20T11:22:36.169629+0000","last_fullsized":"2024-04-20T11:22:36.169629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:30.003670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134458+0000","last_change":"2024-04-20T11:20:01.667297+0000","last_active":"2024-04-20T11:22:36.134458+0000","last_peered":"2024-04-20T11:22:36.134458+0000","last_clean":"2024-04-20T11:22:36.134458+0000","last_became_active":"2024-04-20T11:20:01.666437+0000","last_became_peered":"2024-04-20T11:20:01.666437+0000","last_unstale":"2024-04-20T11:22:36.134458+0000","last_undegraded":"2024-04-20T11:22:36.134458+0000","last_fullsized":"2024-04-20T11:22:36.134458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:45:08.284969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137061+0000","last_change":"2024-04-20T11:20:02.718584+0000","last_active":"2024-04-20T11:22:36.137061+0000","last_peered":"2024-04-20T11:22:36.137061+0000","last_clean":"2024-04-20T11:22:36.137061+0000","last_became_active":"2024-04-20T11:20:02.718184+0000","last_became_peered":"2024-04-20T11:20:02.718184+0000","last_unstale":"2024-04-20T11:22:36.137061+0000","last_undegraded":"2024-04-20T11:22:36.137061+0000","last_fullsized":"2024-04-20T11:22:36.137061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:41.963717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168971+0000","last_change":"2024-04-20T11:20:01.652411+0000","last_active":"2024-04-20T11:22:36.168971+0000","last_peered":"2024-04-20T11:22:36.168971+0000","last_clean":"2024-04-20T11:22:36.168971+0000","last_became_active":"2024-04-20T11:20:01.652098+0000","last_became_peered":"2024-04-20T11:20:01.652098+0000","last_unstale":"2024-04-20T11:22:36.168971+0000","last_undegraded":"2024-04-20T11:22:36.168971+0000","last_fullsized":"2024-04-20T11:22:36.168971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:26.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.103961+0000","last_change":"2024-04-20T11:20:02.712095+0000","last_active":"2024-04-20T11:22:36.103961+0000","last_peered":"2024-04-20T11:22:36.103961+0000","last_clean":"2024-04-20T11:22:36.103961+0000","last_became_active":"2024-04-20T11:20:02.711772+0000","last_became_peered":"2024-04-20T11:20:02.711772+0000","last_unstale":"2024-04-20T11:22:36.103961+0000","last_undegraded":"2024-04-20T11:22:36.103961+0000","last_fullsized":"2024-04-20T11:22:36.103961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:30.491946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'6","reported_seq":325,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.427698+0000","last_change":"2024-04-20T11:20:01.655233+0000","last_active":"2024-04-20T11:22:36.427698+0000","last_peered":"2024-04-20T11:22:36.427698+0000","last_clean":"2024-04-20T11:22:36.427698+0000","last_became_active":"2024-04-20T11:20:01.654864+0000","last_became_peered":"2024-04-20T11:20:01.654864+0000","last_unstale":"2024-04-20T11:22:36.427698+0000","last_undegraded":"2024-04-20T11:22:36.427698+0000","last_fullsized":"2024-04-20T11:22:36.427698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:37.492987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":297,"num_read_kb":297,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169290+0000","last_change":"2024-04-20T11:20:02.696816+0000","last_active":"2024-04-20T11:22:36.169290+0000","last_peered":"2024-04-20T11:22:36.169290+0000","last_clean":"2024-04-20T11:22:36.169290+0000","last_became_active":"2024-04-20T11:20:02.696348+0000","last_became_peered":"2024-04-20T11:20:02.696348+0000","last_unstale":"2024-04-20T11:22:36.169290+0000","last_undegraded":"2024-04-20T11:22:36.169290+0000","last_fullsized":"2024-04-20T11:22:36.169290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:22.190460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114461+0000","last_change":"2024-04-20T11:20:01.653396+0000","last_active":"2024-04-20T11:22:36.114461+0000","last_peered":"2024-04-20T11:22:36.114461+0000","last_clean":"2024-04-20T11:22:36.114461+0000","last_became_active":"2024-04-20T11:20:01.653070+0000","last_became_peered":"2024-04-20T11:20:01.653070+0000","last_unstale":"2024-04-20T11:22:36.114461+0000","last_undegraded":"2024-04-20T11:22:36.114461+0000","last_fullsized":"2024-04-20T11:22:36.114461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:55:09.106741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169778+0000","last_change":"2024-04-20T11:20:02.706493+0000","last_active":"2024-04-20T11:22:36.169778+0000","last_peered":"2024-04-20T11:22:36.169778+0000","last_clean":"2024-04-20T11:22:36.169778+0000","last_became_active":"2024-04-20T11:20:02.705133+0000","last_became_peered":"2024-04-20T11:20:02.705133+0000","last_unstale":"2024-04-20T11:22:36.169778+0000","last_undegraded":"2024-04-20T11:22:36.169778+0000","last_fullsized":"2024-04-20T11:22:36.169778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:03.191160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169923+0000","last_change":"2024-04-20T11:20:01.676094+0000","last_active":"2024-04-20T11:22:36.169923+0000","last_peered":"2024-04-20T11:22:36.169923+0000","last_clean":"2024-04-20T11:22:36.169923+0000","last_became_active":"2024-04-20T11:20:01.675656+0000","last_became_peered":"2024-04-20T11:20:01.675656+0000","last_unstale":"2024-04-20T11:22:36.169923+0000","last_undegraded":"2024-04-20T11:22:36.169923+0000","last_fullsized":"2024-04-20T11:22:36.169923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:57.057257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169079+0000","last_change":"2024-04-20T11:20:02.690816+0000","last_active":"2024-04-20T11:22:36.169079+0000","last_peered":"2024-04-20T11:22:36.169079+0000","last_clean":"2024-04-20T11:22:36.169079+0000","last_became_active":"2024-04-20T11:20:02.690460+0000","last_became_peered":"2024-04-20T11:20:02.690460+0000","last_unstale":"2024-04-20T11:22:36.169079+0000","last_undegraded":"2024-04-20T11:22:36.169079+0000","last_fullsized":"2024-04-20T11:22:36.169079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:29.290781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":366,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.008664+0000","last_change":"2024-04-20T11:20:01.648794+0000","last_active":"2024-04-20T11:22:37.008664+0000","last_peered":"2024-04-20T11:22:37.008664+0000","last_clean":"2024-04-20T11:22:37.008664+0000","last_became_active":"2024-04-20T11:20:01.648462+0000","last_became_peered":"2024-04-20T11:20:01.648462+0000","last_unstale":"2024-04-20T11:22:37.008664+0000","last_undegraded":"2024-04-20T11:22:37.008664+0000","last_fullsized":"2024-04-20T11:22:37.008664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:27.149001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65164,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":310,"num_read_kb":218,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169762+0000","last_change":"2024-04-20T11:20:02.694071+0000","last_active":"2024-04-20T11:22:36.169762+0000","last_peered":"2024-04-20T11:22:36.169762+0000","last_clean":"2024-04-20T11:22:36.169762+0000","last_became_active":"2024-04-20T11:20:02.691868+0000","last_became_peered":"2024-04-20T11:20:02.691868+0000","last_unstale":"2024-04-20T11:22:36.169762+0000","last_undegraded":"2024-04-20T11:22:36.169762+0000","last_fullsized":"2024-04-20T11:22:36.169762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:33:59.152521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154183+0000","last_change":"2024-04-20T11:20:01.669143+0000","last_active":"2024-04-20T11:22:36.154183+0000","last_peered":"2024-04-20T11:22:36.154183+0000","last_clean":"2024-04-20T11:22:36.154183+0000","last_became_active":"2024-04-20T11:20:01.668752+0000","last_became_peered":"2024-04-20T11:20:01.668752+0000","last_unstale":"2024-04-20T11:22:36.154183+0000","last_undegraded":"2024-04-20T11:22:36.154183+0000","last_fullsized":"2024-04-20T11:22:36.154183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:03.021613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134837+0000","last_change":"2024-04-20T11:20:02.716508+0000","last_active":"2024-04-20T11:22:36.134837+0000","last_peered":"2024-04-20T11:22:36.134837+0000","last_clean":"2024-04-20T11:22:36.134837+0000","last_became_active":"2024-04-20T11:20:02.716178+0000","last_became_peered":"2024-04-20T11:20:02.716178+0000","last_unstale":"2024-04-20T11:22:36.134837+0000","last_undegraded":"2024-04-20T11:22:36.134837+0000","last_fullsized":"2024-04-20T11:22:36.134837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:34.059640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169235+0000","last_change":"2024-04-20T11:20:01.669411+0000","last_active":"2024-04-20T11:22:36.169235+0000","last_peered":"2024-04-20T11:22:36.169235+0000","last_clean":"2024-04-20T11:22:36.169235+0000","last_became_active":"2024-04-20T11:20:01.669090+0000","last_became_peered":"2024-04-20T11:20:01.669090+0000","last_unstale":"2024-04-20T11:22:36.169235+0000","last_undegraded":"2024-04-20T11:22:36.169235+0000","last_fullsized":"2024-04-20T11:22:36.169235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:21.362693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114321+0000","last_change":"2024-04-20T11:20:02.707913+0000","last_active":"2024-04-20T11:22:36.114321+0000","last_peered":"2024-04-20T11:22:36.114321+0000","last_clean":"2024-04-20T11:22:36.114321+0000","last_became_active":"2024-04-20T11:20:02.707536+0000","last_became_peered":"2024-04-20T11:20:02.707536+0000","last_unstale":"2024-04-20T11:22:36.114321+0000","last_undegraded":"2024-04-20T11:22:36.114321+0000","last_fullsized":"2024-04-20T11:22:36.114321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:16.433183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'9","reported_seq":184,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.424523+0000","last_change":"2024-04-20T11:20:01.651632+0000","last_active":"2024-04-20T11:22:36.424523+0000","last_peered":"2024-04-20T11:22:36.424523+0000","last_clean":"2024-04-20T11:22:36.424523+0000","last_became_active":"2024-04-20T11:20:01.651300+0000","last_became_peered":"2024-04-20T11:20:01.651300+0000","last_unstale":"2024-04-20T11:22:36.424523+0000","last_undegraded":"2024-04-20T11:22:36.424523+0000","last_fullsized":"2024-04-20T11:22:36.424523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:28:30.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":5,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114107+0000","last_change":"2024-04-20T11:20:51.223843+0000","last_active":"2024-04-20T11:22:36.114107+0000","last_peered":"2024-04-20T11:22:36.114107+0000","last_clean":"2024-04-20T11:22:36.114107+0000","last_became_active":"2024-04-20T11:20:51.223311+0000","last_became_peered":"2024-04-20T11:20:51.223311+0000","last_unstale":"2024-04-20T11:22:36.114107+0000","last_undegraded":"2024-04-20T11:22:36.114107+0000","last_fullsized":"2024-04-20T11:22:36.114107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:38.072609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101882+0000","last_change":"2024-04-20T11:20:51.236281+0000","last_active":"2024-04-20T11:22:36.101882+0000","last_peered":"2024-04-20T11:22:36.101882+0000","last_clean":"2024-04-20T11:22:36.101882+0000","last_became_active":"2024-04-20T11:20:51.222316+0000","last_became_peered":"2024-04-20T11:20:51.222316+0000","last_unstale":"2024-04-20T11:22:36.101882+0000","last_undegraded":"2024-04-20T11:22:36.101882+0000","last_fullsized":"2024-04-20T11:22:36.101882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:31.933379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169128+0000","last_change":"2024-04-20T11:20:02.689478+0000","last_active":"2024-04-20T11:22:36.169128+0000","last_peered":"2024-04-20T11:22:36.169128+0000","last_clean":"2024-04-20T11:22:36.169128+0000","last_became_active":"2024-04-20T11:20:02.688426+0000","last_became_peered":"2024-04-20T11:20:02.688426+0000","last_unstale":"2024-04-20T11:22:36.169128+0000","last_undegraded":"2024-04-20T11:22:36.169128+0000","last_fullsized":"2024-04-20T11:22:36.169128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:19:47.047054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.170084+0000","last_change":"2024-04-20T11:20:01.654699+0000","last_active":"2024-04-20T11:22:36.170084+0000","last_peered":"2024-04-20T11:22:36.170084+0000","last_clean":"2024-04-20T11:22:36.170084+0000","last_became_active":"2024-04-20T11:20:01.654187+0000","last_became_peered":"2024-04-20T11:20:01.654187+0000","last_unstale":"2024-04-20T11:22:36.170084+0000","last_undegraded":"2024-04-20T11:22:36.170084+0000","last_fullsized":"2024-04-20T11:22:36.170084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:27.959414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102064+0000","last_change":"2024-04-20T11:20:02.686346+0000","last_active":"2024-04-20T11:22:36.102064+0000","last_peered":"2024-04-20T11:22:36.102064+0000","last_clean":"2024-04-20T11:22:36.102064+0000","last_became_active":"2024-04-20T11:20:02.685990+0000","last_became_peered":"2024-04-20T11:20:02.685990+0000","last_unstale":"2024-04-20T11:22:36.102064+0000","last_undegraded":"2024-04-20T11:22:36.102064+0000","last_fullsized":"2024-04-20T11:22:36.102064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:17.108810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.104397+0000","last_change":"2024-04-20T11:20:01.683235+0000","last_active":"2024-04-20T11:22:36.104397+0000","last_peered":"2024-04-20T11:22:36.104397+0000","last_clean":"2024-04-20T11:22:36.104397+0000","last_became_active":"2024-04-20T11:20:01.682782+0000","last_became_peered":"2024-04-20T11:20:01.682782+0000","last_unstale":"2024-04-20T11:22:36.104397+0000","last_undegraded":"2024-04-20T11:22:36.104397+0000","last_fullsized":"2024-04-20T11:22:36.104397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:31.528007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151784+0000","last_change":"2024-04-20T11:20:02.719356+0000","last_active":"2024-04-20T11:22:36.151784+0000","last_peered":"2024-04-20T11:22:36.151784+0000","last_clean":"2024-04-20T11:22:36.151784+0000","last_became_active":"2024-04-20T11:20:02.719023+0000","last_became_peered":"2024-04-20T11:20:02.719023+0000","last_unstale":"2024-04-20T11:22:36.151784+0000","last_undegraded":"2024-04-20T11:22:36.151784+0000","last_fullsized":"2024-04-20T11:22:36.151784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:11:39.881226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":178,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:37.120190+0000","last_change":"2024-04-20T11:20:01.665120+0000","last_active":"2024-04-20T11:22:37.120190+0000","last_peered":"2024-04-20T11:22:37.120190+0000","last_clean":"2024-04-20T11:22:37.120190+0000","last_became_active":"2024-04-20T11:20:01.664739+0000","last_became_peered":"2024-04-20T11:20:01.664739+0000","last_unstale":"2024-04-20T11:22:37.120190+0000","last_undegraded":"2024-04-20T11:22:37.120190+0000","last_fullsized":"2024-04-20T11:22:37.120190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:50.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.151591+0000","last_change":"2024-04-20T11:20:02.703173+0000","last_active":"2024-04-20T11:22:36.151591+0000","last_peered":"2024-04-20T11:22:36.151591+0000","last_clean":"2024-04-20T11:22:36.151591+0000","last_became_active":"2024-04-20T11:20:02.702816+0000","last_became_peered":"2024-04-20T11:20:02.702816+0000","last_unstale":"2024-04-20T11:22:36.151591+0000","last_undegraded":"2024-04-20T11:22:36.151591+0000","last_fullsized":"2024-04-20T11:22:36.151591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:00:39.673271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169048+0000","last_change":"2024-04-20T11:20:02.690420+0000","last_active":"2024-04-20T11:22:36.169048+0000","last_peered":"2024-04-20T11:22:36.169048+0000","last_clean":"2024-04-20T11:22:36.169048+0000","last_became_active":"2024-04-20T11:20:02.690127+0000","last_became_peered":"2024-04-20T11:20:02.690127+0000","last_unstale":"2024-04-20T11:22:36.169048+0000","last_undegraded":"2024-04-20T11:22:36.169048+0000","last_fullsized":"2024-04-20T11:22:36.169048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:54:36.957278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.426373+0000","last_change":"2024-04-20T11:20:01.679695+0000","last_active":"2024-04-20T11:22:36.426373+0000","last_peered":"2024-04-20T11:22:36.426373+0000","last_clean":"2024-04-20T11:22:36.426373+0000","last_became_active":"2024-04-20T11:20:01.679177+0000","last_became_peered":"2024-04-20T11:20:01.679177+0000","last_unstale":"2024-04-20T11:22:36.426373+0000","last_undegraded":"2024-04-20T11:22:36.426373+0000","last_fullsized":"2024-04-20T11:22:36.426373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:34.068510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":151,"num_read_kb":151,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137260+0000","last_change":"2024-04-20T11:20:02.716195+0000","last_active":"2024-04-20T11:22:36.137260+0000","last_peered":"2024-04-20T11:22:36.137260+0000","last_clean":"2024-04-20T11:22:36.137260+0000","last_became_active":"2024-04-20T11:20:02.706018+0000","last_became_peered":"2024-04-20T11:20:02.706018+0000","last_unstale":"2024-04-20T11:22:36.137260+0000","last_undegraded":"2024-04-20T11:22:36.137260+0000","last_fullsized":"2024-04-20T11:22:36.137260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:09.691775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102418+0000","last_change":"2024-04-20T11:20:01.662684+0000","last_active":"2024-04-20T11:22:36.102418+0000","last_peered":"2024-04-20T11:22:36.102418+0000","last_clean":"2024-04-20T11:22:36.102418+0000","last_became_active":"2024-04-20T11:20:01.662284+0000","last_became_peered":"2024-04-20T11:20:01.662284+0000","last_unstale":"2024-04-20T11:22:36.102418+0000","last_undegraded":"2024-04-20T11:22:36.102418+0000","last_fullsized":"2024-04-20T11:22:36.102418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:59.278057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114180+0000","last_change":"2024-04-20T11:18:40.112871+0000","last_active":"2024-04-20T11:22:36.114180+0000","last_peered":"2024-04-20T11:22:36.114180+0000","last_clean":"2024-04-20T11:22:36.114180+0000","last_became_active":"2024-04-20T11:18:40.112505+0000","last_became_peered":"2024-04-20T11:18:40.112505+0000","last_unstale":"2024-04-20T11:22:36.114180+0000","last_undegraded":"2024-04-20T11:22:36.114180+0000","last_fullsized":"2024-04-20T11:22:36.114180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:14:38.921947+0000","last_clean_scrub_stamp":"2024-04-20T11:14:38.921947+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:26.680779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134177+0000","last_change":"2024-04-20T11:20:51.224226+0000","last_active":"2024-04-20T11:22:36.134177+0000","last_peered":"2024-04-20T11:22:36.134177+0000","last_clean":"2024-04-20T11:22:36.134177+0000","last_became_active":"2024-04-20T11:20:51.223728+0000","last_became_peered":"2024-04-20T11:20:51.223728+0000","last_unstale":"2024-04-20T11:22:36.134177+0000","last_undegraded":"2024-04-20T11:22:36.134177+0000","last_fullsized":"2024-04-20T11:22:36.134177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:37.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114398+0000","last_change":"2024-04-20T11:20:01.656846+0000","last_active":"2024-04-20T11:22:36.114398+0000","last_peered":"2024-04-20T11:22:36.114398+0000","last_clean":"2024-04-20T11:22:36.114398+0000","last_became_active":"2024-04-20T11:20:01.656481+0000","last_became_peered":"2024-04-20T11:20:01.656481+0000","last_unstale":"2024-04-20T11:22:36.114398+0000","last_undegraded":"2024-04-20T11:22:36.114398+0000","last_fullsized":"2024-04-20T11:22:36.114398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:35.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169785+0000","last_change":"2024-04-20T11:20:01.668339+0000","last_active":"2024-04-20T11:22:36.169785+0000","last_peered":"2024-04-20T11:22:36.169785+0000","last_clean":"2024-04-20T11:22:36.169785+0000","last_became_active":"2024-04-20T11:20:01.668028+0000","last_became_peered":"2024-04-20T11:20:01.668028+0000","last_unstale":"2024-04-20T11:22:36.169785+0000","last_undegraded":"2024-04-20T11:22:36.169785+0000","last_fullsized":"2024-04-20T11:22:36.169785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:04.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134405+0000","last_change":"2024-04-20T11:20:02.694140+0000","last_active":"2024-04-20T11:22:36.134405+0000","last_peered":"2024-04-20T11:22:36.134405+0000","last_clean":"2024-04-20T11:22:36.134405+0000","last_became_active":"2024-04-20T11:20:02.693723+0000","last_became_peered":"2024-04-20T11:20:02.693723+0000","last_unstale":"2024-04-20T11:22:36.134405+0000","last_undegraded":"2024-04-20T11:22:36.134405+0000","last_fullsized":"2024-04-20T11:22:36.134405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:02:31.390557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.366090+0000","last_change":"2024-04-20T11:20:01.671623+0000","last_active":"2024-04-20T11:22:36.366090+0000","last_peered":"2024-04-20T11:22:36.366090+0000","last_clean":"2024-04-20T11:22:36.366090+0000","last_became_active":"2024-04-20T11:20:01.671201+0000","last_became_peered":"2024-04-20T11:20:01.671201+0000","last_unstale":"2024-04-20T11:22:36.366090+0000","last_undegraded":"2024-04-20T11:22:36.366090+0000","last_fullsized":"2024-04-20T11:22:36.366090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:53.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137464+0000","last_change":"2024-04-20T11:20:02.721498+0000","last_active":"2024-04-20T11:22:36.137464+0000","last_peered":"2024-04-20T11:22:36.137464+0000","last_clean":"2024-04-20T11:22:36.137464+0000","last_became_active":"2024-04-20T11:20:02.721076+0000","last_became_peered":"2024-04-20T11:20:02.721076+0000","last_unstale":"2024-04-20T11:22:36.137464+0000","last_undegraded":"2024-04-20T11:22:36.137464+0000","last_fullsized":"2024-04-20T11:22:36.137464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:00.298168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.102357+0000","last_change":"2024-04-20T11:20:01.657337+0000","last_active":"2024-04-20T11:22:36.102357+0000","last_peered":"2024-04-20T11:22:36.102357+0000","last_clean":"2024-04-20T11:22:36.102357+0000","last_became_active":"2024-04-20T11:20:01.656884+0000","last_became_peered":"2024-04-20T11:20:01.656884+0000","last_unstale":"2024-04-20T11:22:36.102357+0000","last_undegraded":"2024-04-20T11:22:36.102357+0000","last_fullsized":"2024-04-20T11:22:36.102357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:44.878147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169611+0000","last_change":"2024-04-20T11:20:02.720591+0000","last_active":"2024-04-20T11:22:36.169611+0000","last_peered":"2024-04-20T11:22:36.169611+0000","last_clean":"2024-04-20T11:22:36.169611+0000","last_became_active":"2024-04-20T11:20:02.720229+0000","last_became_peered":"2024-04-20T11:20:02.720229+0000","last_unstale":"2024-04-20T11:22:36.169611+0000","last_undegraded":"2024-04-20T11:22:36.169611+0000","last_fullsized":"2024-04-20T11:22:36.169611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:57:41.206298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168896+0000","last_change":"2024-04-20T11:20:51.225236+0000","last_active":"2024-04-20T11:22:36.168896+0000","last_peered":"2024-04-20T11:22:36.168896+0000","last_clean":"2024-04-20T11:22:36.168896+0000","last_became_active":"2024-04-20T11:20:51.221635+0000","last_became_peered":"2024-04-20T11:20:51.221635+0000","last_unstale":"2024-04-20T11:22:36.168896+0000","last_undegraded":"2024-04-20T11:22:36.168896+0000","last_fullsized":"2024-04-20T11:22:36.168896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:03.894361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169171+0000","last_change":"2024-04-20T11:20:02.692316+0000","last_active":"2024-04-20T11:22:36.169171+0000","last_peered":"2024-04-20T11:22:36.169171+0000","last_clean":"2024-04-20T11:22:36.169171+0000","last_became_active":"2024-04-20T11:20:02.691262+0000","last_became_peered":"2024-04-20T11:20:02.691262+0000","last_unstale":"2024-04-20T11:22:36.169171+0000","last_undegraded":"2024-04-20T11:22:36.169171+0000","last_fullsized":"2024-04-20T11:22:36.169171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:59:28.635012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137112+0000","last_change":"2024-04-20T11:20:01.677538+0000","last_active":"2024-04-20T11:22:36.137112+0000","last_peered":"2024-04-20T11:22:36.137112+0000","last_clean":"2024-04-20T11:22:36.137112+0000","last_became_active":"2024-04-20T11:20:01.677104+0000","last_became_peered":"2024-04-20T11:20:01.677104+0000","last_unstale":"2024-04-20T11:22:36.137112+0000","last_undegraded":"2024-04-20T11:22:36.137112+0000","last_fullsized":"2024-04-20T11:22:36.137112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:50.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101804+0000","last_change":"2024-04-20T11:20:02.706298+0000","last_active":"2024-04-20T11:22:36.101804+0000","last_peered":"2024-04-20T11:22:36.101804+0000","last_clean":"2024-04-20T11:22:36.101804+0000","last_became_active":"2024-04-20T11:20:02.705859+0000","last_became_peered":"2024-04-20T11:20:02.705859+0000","last_unstale":"2024-04-20T11:22:36.101804+0000","last_undegraded":"2024-04-20T11:22:36.101804+0000","last_fullsized":"2024-04-20T11:22:36.101804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:01:55.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154030+0000","last_change":"2024-04-20T11:20:01.664285+0000","last_active":"2024-04-20T11:22:36.154030+0000","last_peered":"2024-04-20T11:22:36.154030+0000","last_clean":"2024-04-20T11:22:36.154030+0000","last_became_active":"2024-04-20T11:20:01.663896+0000","last_became_peered":"2024-04-20T11:20:01.663896+0000","last_unstale":"2024-04-20T11:22:36.154030+0000","last_undegraded":"2024-04-20T11:22:36.154030+0000","last_fullsized":"2024-04-20T11:22:36.154030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:55.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137375+0000","last_change":"2024-04-20T11:20:02.717667+0000","last_active":"2024-04-20T11:22:36.137375+0000","last_peered":"2024-04-20T11:22:36.137375+0000","last_clean":"2024-04-20T11:22:36.137375+0000","last_became_active":"2024-04-20T11:20:02.714867+0000","last_became_peered":"2024-04-20T11:20:02.714867+0000","last_unstale":"2024-04-20T11:22:36.137375+0000","last_undegraded":"2024-04-20T11:22:36.137375+0000","last_fullsized":"2024-04-20T11:22:36.137375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:28.507170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169506+0000","last_change":"2024-04-20T11:20:01.652327+0000","last_active":"2024-04-20T11:22:36.169506+0000","last_peered":"2024-04-20T11:22:36.169506+0000","last_clean":"2024-04-20T11:22:36.169506+0000","last_became_active":"2024-04-20T11:20:01.651272+0000","last_became_peered":"2024-04-20T11:20:01.651272+0000","last_unstale":"2024-04-20T11:22:36.169506+0000","last_undegraded":"2024-04-20T11:22:36.169506+0000","last_fullsized":"2024-04-20T11:22:36.169506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:19.823521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114559+0000","last_change":"2024-04-20T11:20:02.695560+0000","last_active":"2024-04-20T11:22:36.114559+0000","last_peered":"2024-04-20T11:22:36.114559+0000","last_clean":"2024-04-20T11:22:36.114559+0000","last_became_active":"2024-04-20T11:20:02.695245+0000","last_became_peered":"2024-04-20T11:20:02.695245+0000","last_unstale":"2024-04-20T11:22:36.114559+0000","last_undegraded":"2024-04-20T11:22:36.114559+0000","last_fullsized":"2024-04-20T11:22:36.114559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:50:34.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'8","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154545+0000","last_change":"2024-04-20T11:20:01.676902+0000","last_active":"2024-04-20T11:22:36.154545+0000","last_peered":"2024-04-20T11:22:36.154545+0000","last_clean":"2024-04-20T11:22:36.154545+0000","last_became_active":"2024-04-20T11:20:01.676574+0000","last_became_peered":"2024-04-20T11:20:01.676574+0000","last_unstale":"2024-04-20T11:22:36.154545+0000","last_undegraded":"2024-04-20T11:22:36.154545+0000","last_fullsized":"2024-04-20T11:22:36.154545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:09.218928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114465+0000","last_change":"2024-04-20T11:20:02.692835+0000","last_active":"2024-04-20T11:22:36.114465+0000","last_peered":"2024-04-20T11:22:36.114465+0000","last_clean":"2024-04-20T11:22:36.114465+0000","last_became_active":"2024-04-20T11:20:02.692338+0000","last_became_peered":"2024-04-20T11:20:02.692338+0000","last_unstale":"2024-04-20T11:22:36.114465+0000","last_undegraded":"2024-04-20T11:22:36.114465+0000","last_fullsized":"2024-04-20T11:22:36.114465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:46:08.350714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.363029+0000","last_change":"2024-04-20T11:20:01.670083+0000","last_active":"2024-04-20T11:22:36.363029+0000","last_peered":"2024-04-20T11:22:36.363029+0000","last_clean":"2024-04-20T11:22:36.363029+0000","last_became_active":"2024-04-20T11:20:01.669730+0000","last_became_peered":"2024-04-20T11:20:01.669730+0000","last_unstale":"2024-04-20T11:22:36.363029+0000","last_undegraded":"2024-04-20T11:22:36.363029+0000","last_fullsized":"2024-04-20T11:22:36.363029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:34.856060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114342+0000","last_change":"2024-04-20T11:20:02.709962+0000","last_active":"2024-04-20T11:22:36.114342+0000","last_peered":"2024-04-20T11:22:36.114342+0000","last_clean":"2024-04-20T11:22:36.114342+0000","last_became_active":"2024-04-20T11:20:02.709639+0000","last_became_peered":"2024-04-20T11:20:02.709639+0000","last_unstale":"2024-04-20T11:22:36.114342+0000","last_undegraded":"2024-04-20T11:22:36.114342+0000","last_fullsized":"2024-04-20T11:22:36.114342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:55.067570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134792+0000","last_change":"2024-04-20T11:20:01.670000+0000","last_active":"2024-04-20T11:22:36.134792+0000","last_peered":"2024-04-20T11:22:36.134792+0000","last_clean":"2024-04-20T11:22:36.134792+0000","last_became_active":"2024-04-20T11:20:01.669673+0000","last_became_peered":"2024-04-20T11:20:01.669673+0000","last_unstale":"2024-04-20T11:22:36.134792+0000","last_undegraded":"2024-04-20T11:22:36.134792+0000","last_fullsized":"2024-04-20T11:22:36.134792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:00.628490+0000","last_clean_scrub_stamp":"2024-04-20T11:20:00.628490+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:34.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.114458+0000","last_change":"2024-04-20T11:20:02.688430+0000","last_active":"2024-04-20T11:22:36.114458+0000","last_peered":"2024-04-20T11:22:36.114458+0000","last_clean":"2024-04-20T11:22:36.114458+0000","last_became_active":"2024-04-20T11:20:02.688130+0000","last_became_peered":"2024-04-20T11:20:02.688130+0000","last_unstale":"2024-04-20T11:22:36.114458+0000","last_undegraded":"2024-04-20T11:22:36.114458+0000","last_fullsized":"2024-04-20T11:22:36.114458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:03.731921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101975+0000","last_change":"2024-04-20T11:20:02.687680+0000","last_active":"2024-04-20T11:22:36.101975+0000","last_peered":"2024-04-20T11:22:36.101975+0000","last_clean":"2024-04-20T11:22:36.101975+0000","last_became_active":"2024-04-20T11:20:02.687304+0000","last_became_peered":"2024-04-20T11:20:02.687304+0000","last_unstale":"2024-04-20T11:22:36.101975+0000","last_undegraded":"2024-04-20T11:22:36.101975+0000","last_fullsized":"2024-04-20T11:22:36.101975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:58:03.243024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154063+0000","last_change":"2024-04-20T11:20:02.695675+0000","last_active":"2024-04-20T11:22:36.154063+0000","last_peered":"2024-04-20T11:22:36.154063+0000","last_clean":"2024-04-20T11:22:36.154063+0000","last_became_active":"2024-04-20T11:20:02.695414+0000","last_became_peered":"2024-04-20T11:20:02.695414+0000","last_unstale":"2024-04-20T11:22:36.154063+0000","last_undegraded":"2024-04-20T11:22:36.154063+0000","last_fullsized":"2024-04-20T11:22:36.154063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:09.157993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154017+0000","last_change":"2024-04-20T11:20:02.711872+0000","last_active":"2024-04-20T11:22:36.154017+0000","last_peered":"2024-04-20T11:22:36.154017+0000","last_clean":"2024-04-20T11:22:36.154017+0000","last_became_active":"2024-04-20T11:20:02.711551+0000","last_became_peered":"2024-04-20T11:20:02.711551+0000","last_unstale":"2024-04-20T11:22:36.154017+0000","last_undegraded":"2024-04-20T11:22:36.154017+0000","last_fullsized":"2024-04-20T11:22:36.154017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:28.681014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169323+0000","last_change":"2024-04-20T11:20:02.716658+0000","last_active":"2024-04-20T11:22:36.169323+0000","last_peered":"2024-04-20T11:22:36.169323+0000","last_clean":"2024-04-20T11:22:36.169323+0000","last_became_active":"2024-04-20T11:20:02.716315+0000","last_became_peered":"2024-04-20T11:20:02.716315+0000","last_unstale":"2024-04-20T11:22:36.169323+0000","last_undegraded":"2024-04-20T11:22:36.169323+0000","last_fullsized":"2024-04-20T11:22:36.169323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:59:34.689028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134952+0000","last_change":"2024-04-20T11:20:02.711596+0000","last_active":"2024-04-20T11:22:36.134952+0000","last_peered":"2024-04-20T11:22:36.134952+0000","last_clean":"2024-04-20T11:22:36.134952+0000","last_became_active":"2024-04-20T11:20:02.709711+0000","last_became_peered":"2024-04-20T11:20:02.709711+0000","last_unstale":"2024-04-20T11:22:36.134952+0000","last_undegraded":"2024-04-20T11:22:36.134952+0000","last_fullsized":"2024-04-20T11:22:36.134952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:15.178394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101932+0000","last_change":"2024-04-20T11:20:02.687403+0000","last_active":"2024-04-20T11:22:36.101932+0000","last_peered":"2024-04-20T11:22:36.101932+0000","last_clean":"2024-04-20T11:22:36.101932+0000","last_became_active":"2024-04-20T11:20:02.687083+0000","last_became_peered":"2024-04-20T11:20:02.687083+0000","last_unstale":"2024-04-20T11:22:36.101932+0000","last_undegraded":"2024-04-20T11:22:36.101932+0000","last_fullsized":"2024-04-20T11:22:36.101932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:04:22.278411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.154450+0000","last_change":"2024-04-20T11:20:02.688118+0000","last_active":"2024-04-20T11:22:36.154450+0000","last_peered":"2024-04-20T11:22:36.154450+0000","last_clean":"2024-04-20T11:22:36.154450+0000","last_became_active":"2024-04-20T11:20:02.687781+0000","last_became_peered":"2024-04-20T11:20:02.687781+0000","last_unstale":"2024-04-20T11:22:36.154450+0000","last_undegraded":"2024-04-20T11:22:36.154450+0000","last_fullsized":"2024-04-20T11:22:36.154450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:30:18.167022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137439+0000","last_change":"2024-04-20T11:20:02.718602+0000","last_active":"2024-04-20T11:22:36.137439+0000","last_peered":"2024-04-20T11:22:36.137439+0000","last_clean":"2024-04-20T11:22:36.137439+0000","last_became_active":"2024-04-20T11:20:02.718219+0000","last_became_peered":"2024-04-20T11:20:02.718219+0000","last_unstale":"2024-04-20T11:22:36.137439+0000","last_undegraded":"2024-04-20T11:22:36.137439+0000","last_fullsized":"2024-04-20T11:22:36.137439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:17.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.169002+0000","last_change":"2024-04-20T11:20:02.721578+0000","last_active":"2024-04-20T11:22:36.169002+0000","last_peered":"2024-04-20T11:22:36.169002+0000","last_clean":"2024-04-20T11:22:36.169002+0000","last_became_active":"2024-04-20T11:20:02.721245+0000","last_became_peered":"2024-04-20T11:20:02.721245+0000","last_unstale":"2024-04-20T11:22:36.169002+0000","last_undegraded":"2024-04-20T11:22:36.169002+0000","last_fullsized":"2024-04-20T11:22:36.169002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:15.125279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.137123+0000","last_change":"2024-04-20T11:20:02.713860+0000","last_active":"2024-04-20T11:22:36.137123+0000","last_peered":"2024-04-20T11:22:36.137123+0000","last_clean":"2024-04-20T11:22:36.137123+0000","last_became_active":"2024-04-20T11:20:02.708242+0000","last_became_peered":"2024-04-20T11:20:02.708242+0000","last_unstale":"2024-04-20T11:22:36.137123+0000","last_undegraded":"2024-04-20T11:22:36.137123+0000","last_fullsized":"2024-04-20T11:22:36.137123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:30.213524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.168939+0000","last_change":"2024-04-20T11:20:02.712041+0000","last_active":"2024-04-20T11:22:36.168939+0000","last_peered":"2024-04-20T11:22:36.168939+0000","last_clean":"2024-04-20T11:22:36.168939+0000","last_became_active":"2024-04-20T11:20:02.711699+0000","last_became_peered":"2024-04-20T11:20:02.711699+0000","last_unstale":"2024-04-20T11:22:36.168939+0000","last_undegraded":"2024-04-20T11:22:36.168939+0000","last_fullsized":"2024-04-20T11:22:36.168939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:04:38.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.134401+0000","last_change":"2024-04-20T11:20:02.713853+0000","last_active":"2024-04-20T11:22:36.134401+0000","last_peered":"2024-04-20T11:22:36.134401+0000","last_clean":"2024-04-20T11:22:36.134401+0000","last_became_active":"2024-04-20T11:20:02.713478+0000","last_became_peered":"2024-04-20T11:20:02.713478+0000","last_unstale":"2024-04-20T11:22:36.134401+0000","last_undegraded":"2024-04-20T11:22:36.134401+0000","last_fullsized":"2024-04-20T11:22:36.134401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:44.865669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T11:22:36.101694+0000","last_change":"2024-04-20T11:20:02.690186+0000","last_active":"2024-04-20T11:22:36.101694+0000","last_peered":"2024-04-20T11:22:36.101694+0000","last_clean":"2024-04-20T11:22:36.101694+0000","last_became_active":"2024-04-20T11:20:02.689894+0000","last_became_peered":"2024-04-20T11:20:02.689894+0000","last_unstale":"2024-04-20T11:22:36.101694+0000","last_undegraded":"2024-04-20T11:22:36.101694+0000","last_fullsized":"2024-04-20T11:22:36.101694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T11:20:01.631212+0000","last_clean_scrub_stamp":"2024-04-20T11:20:01.631212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:40.348879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107634,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599030,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1624,"num_read_kb":925,"num_write":368,"num_write_kb":1461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":700416,"data_stored":1797090,"data_compressed":81420,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":466545,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645361,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1254617,"data_compressed":46779,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":858656,"data_compressed":22053,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":858922,"data_compressed":22053,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1059650,"data_compressed":38785,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":465157,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":531625,"data_compressed":4745,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":467419,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":532377,"data_compressed":4745,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":989798,"data_compressed":28209,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":466332,"data_compressed":2907,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1189223,"data_compressed":44941,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35878,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67307,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68059,"data_compressed":1838,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":199814,"data_compressed":7994,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:22:42.843 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 -- 172.21.15.44:0/1993217290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 msgr2=0x7f780c074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 --2- 172.21.15.44:0/1993217290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c074e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009900 tx=0x7f77f802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 -- 172.21.15.44:0/1993217290 shutdown_connections 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 --2- 172.21.15.44:0/1993217290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 0x7f780c1107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 --2- 172.21.15.44:0/1993217290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c074e40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 --2- 172.21.15.44:0/1993217290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780c075fa0 0x7f780c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 -- 172.21.15.44:0/1993217290 >> 172.21.15.44:0/1993217290 conn(0x7f780c0fd370 msgr2=0x7f780c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 -- 172.21.15.44:0/1993217290 shutdown_connections 2024-04-20T11:22:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.059+0000 7f780b200640 1 -- 172.21.15.44:0/1993217290 wait complete. 2024-04-20T11:22:43.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.060+0000 7f780b200640 1 Processor -- start 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 -- start start 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780c075fa0 0x7f780c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 0x7f780c1a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1141b0 con 0x7f780c075fa0 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f780c114030 con 0x7f780c10e3d0 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f780c114330 con 0x7f780c0749c0 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f7809e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f7809e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48586/0 (socket says 172.21.15.44:48586) 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f7809e00640 1 -- 172.21.15.44:0/175082298 learned_addr learned my addr 172.21.15.44:0/175082298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f780a800640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 0x7f780c1a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.061+0000 7f7809400640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780c075fa0 0x7f780c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7809e00640 1 -- 172.21.15.44:0/175082298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 msgr2=0x7f780c1a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7809e00640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 0x7f780c1a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7809e00640 1 -- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780c075fa0 msgr2=0x7f780c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7809e00640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780c075fa0 0x7f780c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7809e00640 1 -- 172.21.15.44:0/175082298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c1a63e0 con 0x7f780c0749c0 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f780a800640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 0x7f780c1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7809e00640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c19e7c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9f0 tx=0x7f77fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00cd80 con 0x7f780c0749c0 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f77fc010930 con 0x7f780c0749c0 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.062+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc018b00 con 0x7f780c0749c0 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.063+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c1a66d0 con 0x7f780c0749c0 2024-04-20T11:22:43.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.063+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c1a6ba0 con 0x7f780c0749c0 2024-04-20T11:22:43.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.064+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c1045a0 con 0x7f780c0749c0 2024-04-20T11:22:43.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.066+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f77fc018ca0 con 0x7f780c0749c0 2024-04-20T11:22:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.068+0000 7f7802a00640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f77f00a3ec0 0x7f77f00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.068+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f77fc0d0dd0 con 0x7f780c0749c0 2024-04-20T11:22:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.069+0000 7f7809400640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f77f00a3ec0 0x7f77f00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.069+0000 7f7809400640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f77f00a3ec0 0x7f77f00a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77f8000c00 tx=0x7f77f8005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.071+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f77fc09cbe0 con 0x7f780c0749c0 2024-04-20T11:22:43.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.284+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f780c02d860 con 0x7f780c0749c0 2024-04-20T11:22:43.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.285+0000 7f7802a00640 1 -- 172.21.15.44:0/175082298 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22887 (secure 0 0 0) 0x7f77fc0a0ee0 con 0x7f780c0749c0 2024-04-20T11:22:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:43.287 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:22:36.090257+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:43.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.288+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f77f00a3ec0 msgr2=0x7f77f00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.288+0000 7f780b200640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f77f00a3ec0 0x7f77f00a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77f8000c00 tx=0x7f77f8005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.288+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 msgr2=0x7f780c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.288+0000 7f780b200640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c19e7c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9f0 tx=0x7f77fc00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 shutdown_connections 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f780c10e3d0 0x7f780c1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f77f00a3ec0 0x7f77f00a6380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780c075fa0 0x7f780c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 --2- 172.21.15.44:0/175082298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f780c0749c0 0x7f780c19e7c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 >> 172.21.15.44:0/175082298 conn(0x7f780c0fd370 msgr2=0x7f780c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 shutdown_connections 2024-04-20T11:22:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.289+0000 7f780b200640 1 -- 172.21.15.44:0/175082298 wait complete. 2024-04-20T11:22:43.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:22:43.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 -- 172.21.15.44:0/3078069287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1053b0 msgr2=0x7f7dec105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 --2- 172.21.15.44:0/3078069287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1053b0 0x7f7dec105830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009900 tx=0x7f7dd402bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 -- 172.21.15.44:0/3078069287 shutdown_connections 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 --2- 172.21.15.44:0/3078069287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec105d70 0x7f7dec10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 --2- 172.21.15.44:0/3078069287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1053b0 0x7f7dec105830 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 --2- 172.21.15.44:0/3078069287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1041c0 0x7f7dec1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 -- 172.21.15.44:0/3078069287 >> 172.21.15.44:0/3078069287 conn(0x7f7dec0ff750 msgr2=0x7f7dec101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 -- 172.21.15.44:0/3078069287 shutdown_connections 2024-04-20T11:22:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.507+0000 7f7df1a00640 1 -- 172.21.15.44:0/3078069287 wait complete. 2024-04-20T11:22:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.508+0000 7f7df1a00640 1 Processor -- start 2024-04-20T11:22:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.508+0000 7f7df1a00640 1 -- start start 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7df1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1041c0 0x7f7dec19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7df1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 0x7f7dec19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7df1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec105d70 0x7f7dec1a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7df1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec10ff20 con 0x7f7dec1041c0 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7df1a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec10fda0 con 0x7f7dec105d70 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7df1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec1100a0 con 0x7f7dec1053b0 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 0x7f7dec19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1041c0 0x7f7dec19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 0x7f7dec19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48612/0 (socket says 172.21.15.44:48612) 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 -- 172.21.15.44:0/2213223841 learned_addr learned my addr 172.21.15.44:0/2213223841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7deb400640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec105d70 0x7f7dec1a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 -- 172.21.15.44:0/2213223841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec105d70 msgr2=0x7f7dec1a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec105d70 0x7f7dec1a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 -- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1041c0 msgr2=0x7f7dec19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1041c0 0x7f7dec19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7de3e00640 1 -- 172.21.15.44:0/2213223841 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec1a2190 con 0x7f7dec1053b0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.509+0000 7f7deaa00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1041c0 0x7f7dec19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.510+0000 7f7de3e00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 0x7f7dec19aab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4005ab0 tx=0x7f7dd4004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.510+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4043020 con 0x7f7dec1053b0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.510+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec1a2420 con 0x7f7dec1053b0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.510+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7dd4004250 con 0x7f7dec1053b0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.510+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4003c00 con 0x7f7dec1053b0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.510+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a29c0 con 0x7f7dec1053b0 2024-04-20T11:22:43.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.511+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec10b480 con 0x7f7dec1053b0 2024-04-20T11:22:43.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.513+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f7dd4046020 con 0x7f7dec1053b0 2024-04-20T11:22:43.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.514+0000 7f7de3400640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7dd00a3e40 0x7f7dd00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.515+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7dd40f29c0 con 0x7f7dec1053b0 2024-04-20T11:22:43.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.515+0000 7f7deaa00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7dd00a3e40 0x7f7dd00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.515+0000 7f7deaa00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7dd00a3e40 0x7f7dd00a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7dec105210 tx=0x7f7ddc005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:43.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.518+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7dd404d020 con 0x7f7dec1053b0 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='client.25432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi044 to 177.5M 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: Unable to set osd_memory_target on smithi044 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T11:22:43.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:22:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:22:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:22:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:22:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:22:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/175082298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:43.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='client.25432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi044 to 177.5M 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: Unable to set osd_memory_target on smithi044 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:22:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:22:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:22:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:22:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/175082298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='client.25432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi044 to 177.5M 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: Unable to set osd_memory_target on smithi044 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-20T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-20T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-20T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/175082298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.742+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7dec02d780 con 0x7f7dec1053b0 2024-04-20T11:22:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.743+0000 7f7de3400640 1 -- 172.21.15.44:0/2213223841 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f7dd40bef30 con 0x7f7dec1053b0 2024-04-20T11:22:43.743 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:22:43.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7dd00a3e40 msgr2=0x7f7dd00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7dd00a3e40 0x7f7dd00a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7dec105210 tx=0x7f7ddc005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 msgr2=0x7f7dec19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 0x7f7dec19aab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4005ab0 tx=0x7f7dd4004a50 comp rx=0 tx=0).stop 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 shutdown_connections 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dec105d70 0x7f7dec1a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7dec1053b0 0x7f7dec19aab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7dd00a3e40 0x7f7dd00a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 --2- 172.21.15.44:0/2213223841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dec1041c0 0x7f7dec19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 >> 172.21.15.44:0/2213223841 conn(0x7f7dec0ff750 msgr2=0x7f7dec108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 shutdown_connections 2024-04-20T11:22:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.746+0000 7f7df1a00640 1 -- 172.21.15.44:0/2213223841 wait complete. 2024-04-20T11:22:43.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 -- 172.21.15.44:0/3120292220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf981039a0 msgr2=0x7fcf9810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 --2- 172.21.15.44:0/3120292220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf981039a0 0x7fcf9810a2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009ca0 tx=0x7fcf9002c0f0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 -- 172.21.15.44:0/3120292220 shutdown_connections 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 --2- 172.21.15.44:0/3120292220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf981039a0 0x7fcf9810a2c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 --2- 172.21.15.44:0/3120292220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 0x7fcf98103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 --2- 172.21.15.44:0/3120292220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf98101e90 0x7fcf981022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 -- 172.21.15.44:0/3120292220 >> 172.21.15.44:0/3120292220 conn(0x7fcf980fd400 msgr2=0x7fcf980ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 -- 172.21.15.44:0/3120292220 shutdown_connections 2024-04-20T11:22:43.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.957+0000 7fcf97e00640 1 -- 172.21.15.44:0/3120292220 wait complete. 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.958+0000 7fcf97e00640 1 Processor -- start 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.958+0000 7fcf97e00640 1 -- start start 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 0x7fcf981982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 0x7fcf981987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf981039a0 0x7fcf9819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9810dc60 con 0x7fcf98102fe0 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9810dae0 con 0x7fcf98101e90 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9810dde0 con 0x7fcf981039a0 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 0x7fcf981982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 0x7fcf981982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36880/0 (socket says 172.21.15.44:36880) 2024-04-20T11:22:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 -- 172.21.15.44:0/2605615388 learned_addr learned my addr 172.21.15.44:0/2605615388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96000640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 0x7fcf981987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf97400640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf981039a0 0x7fcf9819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 -- 172.21.15.44:0/2605615388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf981039a0 msgr2=0x7fcf9819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf981039a0 0x7fcf9819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 -- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 msgr2=0x7fcf981987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 0x7fcf981987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96a00640 1 -- 172.21.15.44:0/2605615388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9819fe80 con 0x7fcf98101e90 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.959+0000 7fcf96000640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 0x7fcf981987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.960+0000 7fcf96a00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 0x7fcf981982b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800e9f0 tx=0x7fcf8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.960+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800cd80 con 0x7fcf98101e90 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.960+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf88010930 con 0x7fcf98101e90 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.960+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88018b00 con 0x7fcf98101e90 2024-04-20T11:22:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.960+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981a0170 con 0x7fcf98101e90 2024-04-20T11:22:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.960+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a0710 con 0x7fcf98101e90 2024-04-20T11:22:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.961+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf981090d0 con 0x7fcf98101e90 2024-04-20T11:22:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.963+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fcf88018ca0 con 0x7fcf98101e90 2024-04-20T11:22:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.965+0000 7fcf8f400640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcf7c0a3ec0 0x7fcf7c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.965+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcf880d0e80 con 0x7fcf98101e90 2024-04-20T11:22:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.965+0000 7fcf96000640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcf7c0a3ec0 0x7fcf7c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.966+0000 7fcf96000640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcf7c0a3ec0 0x7fcf7c0a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf98199770 tx=0x7fcf84005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:43.969+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcf8809d2e0 con 0x7fcf98101e90 2024-04-20T11:22:44.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.181+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf98062b60 con 0x7fcf98101e90 2024-04-20T11:22:44.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.182+0000 7fcf8f400640 1 -- 172.21.15.44:0/2605615388 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fcf8809cbe0 con 0x7fcf98101e90 2024-04-20T11:22:44.182 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcf7c0a3ec0 msgr2=0x7fcf7c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcf7c0a3ec0 0x7fcf7c0a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf98199770 tx=0x7fcf84005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 msgr2=0x7fcf981982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 0x7fcf981982b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800e9f0 tx=0x7fcf8800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 shutdown_connections 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf981039a0 0x7fcf9819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcf7c0a3ec0 0x7fcf7c0a6380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf98102fe0 0x7fcf981987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 --2- 172.21.15.44:0/2605615388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98101e90 0x7fcf981982b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 >> 172.21.15.44:0/2605615388 conn(0x7fcf980fd400 msgr2=0x7fcf981062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:44.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 shutdown_connections 2024-04-20T11:22:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.184+0000 7fcf97e00640 1 -- 172.21.15.44:0/2605615388 wait complete. 2024-04-20T11:22:44.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 -- 172.21.15.44:0/3918594463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104d20 msgr2=0x7faff8108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 --2- 172.21.15.44:0/3918594463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104d20 0x7faff8108d60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009900 tx=0x7fafe002bc50 comp rx=0 tx=0).stop 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 -- 172.21.15.44:0/3918594463 shutdown_connections 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 --2- 172.21.15.44:0/3918594463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 0x7faff8111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 --2- 172.21.15.44:0/3918594463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104d20 0x7faff8108d60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 --2- 172.21.15.44:0/3918594463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff8104370 0x7faff8104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 -- 172.21.15.44:0/3918594463 >> 172.21.15.44:0/3918594463 conn(0x7faff80ffc90 msgr2=0x7faff81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 -- 172.21.15.44:0/3918594463 shutdown_connections 2024-04-20T11:22:44.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.386+0000 7faffdc00640 1 -- 172.21.15.44:0/3918594463 wait complete. 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.387+0000 7faffdc00640 1 Processor -- start 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.387+0000 7faffdc00640 1 -- start start 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.387+0000 7faffdc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104370 0x7faff81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.387+0000 7faffdc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 0x7faff81a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.387+0000 7faffdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff819a930 0x7faff819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.387+0000 7faffdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faff8114610 con 0x7faff819a930 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7faffdc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faff8114490 con 0x7faff8104370 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7faffdc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faff8114790 con 0x7faff8109440 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7faff7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff819a930 0x7faff819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7faff6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104370 0x7faff81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7faff6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104370 0x7faff81a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36910/0 (socket says 172.21.15.44:36910) 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7faff6a00640 1 -- 172.21.15.44:0/143574230 learned_addr learned my addr 172.21.15.44:0/143574230 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7fafee000640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 0x7faff81a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7fafee000640 1 -- 172.21.15.44:0/143574230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104370 msgr2=0x7faff81a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7fafee000640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104370 0x7faff81a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7fafee000640 1 -- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff819a930 msgr2=0x7faff819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7fafee000640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff819a930 0x7faff819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.388+0000 7fafee000640 1 -- 172.21.15.44:0/143574230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff819b450 con 0x7faff8109440 2024-04-20T11:22:44.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.389+0000 7fafee000640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 0x7faff81a0cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fafe0036040 tx=0x7fafe002dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:44.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.389+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0043020 con 0x7faff8109440 2024-04-20T11:22:44.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.389+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fafe002de00 con 0x7faff8109440 2024-04-20T11:22:44.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.389+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0003a10 con 0x7faff8109440 2024-04-20T11:22:44.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.389+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff819b6e0 con 0x7faff8109440 2024-04-20T11:22:44.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.389+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a7980 con 0x7faff8109440 2024-04-20T11:22:44.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.391+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff80691d0 con 0x7faff8109440 2024-04-20T11:22:44.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.394+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fafe0046020 con 0x7faff8109440 2024-04-20T11:22:44.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.395+0000 7fafefe00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fafdc0a3ec0 0x7fafdc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.395+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fafe00f5120 con 0x7faff8109440 2024-04-20T11:22:44.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.396+0000 7faff6a00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fafdc0a3ec0 0x7fafdc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.396+0000 7faff6a00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fafdc0a3ec0 0x7fafdc0a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faff819ba70 tx=0x7fafe8009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:44.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.400+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fafe004d020 con 0x7faff8109440 2024-04-20T11:22:44.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.610+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faff810c730 con 0x7faff8109440 2024-04-20T11:22:44.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.611+0000 7fafefe00640 1 -- 172.21.15.44:0/143574230 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fafe00bee80 con 0x7faff8109440 2024-04-20T11:22:44.612 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fafdc0a3ec0 msgr2=0x7fafdc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fafdc0a3ec0 0x7fafdc0a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faff819ba70 tx=0x7fafe8009500 comp rx=0 tx=0).stop 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 msgr2=0x7faff81a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 0x7faff81a0cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fafe0036040 tx=0x7fafe002dc10 comp rx=0 tx=0).stop 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 shutdown_connections 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fafdc0a3ec0 0x7fafdc0a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faff819a930 0x7faff819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faff8109440 0x7faff81a0cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 --2- 172.21.15.44:0/143574230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff8104370 0x7faff81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 >> 172.21.15.44:0/143574230 conn(0x7faff80ffc90 msgr2=0x7faff8101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:44.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 shutdown_connections 2024-04-20T11:22:44.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.614+0000 7faffdc00640 1 -- 172.21.15.44:0/143574230 wait complete. 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.632 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.633 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2213223841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:44.633 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2605615388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:44.634 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2213223841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:44.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2605615388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:44.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:44.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T11:22:44.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2213223841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2605615388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:22:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 -- 172.21.15.44:0/2726356787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074c30 msgr2=0x7f063c10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 --2- 172.21.15.44:0/2726356787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074c30 0x7f063c10e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0628009900 tx=0x7f062802bcc0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 -- 172.21.15.44:0/2726356787 shutdown_connections 2024-04-20T11:22:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 --2- 172.21.15.44:0/2726356787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c10e600 0x7f063c1109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 --2- 172.21.15.44:0/2726356787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074c30 0x7f063c10e0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 --2- 172.21.15.44:0/2726356787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f063c074310 0x7f063c0746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 -- 172.21.15.44:0/2726356787 >> 172.21.15.44:0/2726356787 conn(0x7f063c0fd890 msgr2=0x7f063c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 -- 172.21.15.44:0/2726356787 shutdown_connections 2024-04-20T11:22:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.821+0000 7f063be00640 1 -- 172.21.15.44:0/2726356787 wait complete. 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 Processor -- start 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 -- start start 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 0x7f063c101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c074c30 0x7f063c102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f063c10e600 0x7f063c1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f063c114420 con 0x7f063c074c30 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1142a0 con 0x7f063c074310 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1145a0 con 0x7f063c10e600 2024-04-20T11:22:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 0x7f063c101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.823+0000 7f063aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 0x7f063c101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36932/0 (socket says 172.21.15.44:36932) 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 -- 172.21.15.44:0/4184457644 learned_addr learned my addr 172.21.15.44:0/4184457644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063a000640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c074c30 0x7f063c102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063b400640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f063c10e600 0x7f063c1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 -- 172.21.15.44:0/4184457644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f063c10e600 msgr2=0x7f063c1027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f063c10e600 0x7f063c1027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 -- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c074c30 msgr2=0x7f063c102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c074c30 0x7f063c102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 -- 172.21.15.44:0/4184457644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c1ab000 con 0x7f063c074310 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063a000640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c074c30 0x7f063c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063aa00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 0x7f063c101d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f062c00ea20 tx=0x7f062c00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c00cd80 con 0x7f063c074310 2024-04-20T11:22:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f062c010930 con 0x7f063c074310 2024-04-20T11:22:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c018a30 con 0x7f063c074310 2024-04-20T11:22:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.824+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c1ab2f0 con 0x7f063c074310 2024-04-20T11:22:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.825+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1ab810 con 0x7f063c074310 2024-04-20T11:22:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.826+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c0759c0 con 0x7f063c074310 2024-04-20T11:22:44.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.828+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f062c010ae0 con 0x7f063c074310 2024-04-20T11:22:44.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.829+0000 7f0633400640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06200a3e40 0x7f06200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.829+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f062c0d0ef0 con 0x7f063c074310 2024-04-20T11:22:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.829+0000 7f063a000640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06200a3e40 0x7f06200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.830+0000 7f063a000640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06200a3e40 0x7f06200a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f063c103520 tx=0x7f062802c7c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:44.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:44.833+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f062c09d350 con 0x7f063c074310 2024-04-20T11:22:45.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.098+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f063c02d5d0 con 0x7f063c074310 2024-04-20T11:22:45.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.099+0000 7f0633400640 1 -- 172.21.15.44:0/4184457644 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f062c09cc50 con 0x7f063c074310 2024-04-20T11:22:45.100 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:45.100 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:22:35.546711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:22:45.100 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-04-20T11:22:45.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06200a3e40 msgr2=0x7f06200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06200a3e40 0x7f06200a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f063c103520 tx=0x7f062802c7c0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 msgr2=0x7f063c101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 0x7f063c101d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f062c00ea20 tx=0x7f062c00eee0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 shutdown_connections 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f063c10e600 0x7f063c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06200a3e40 0x7f06200a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f063c074c30 0x7f063c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 --2- 172.21.15.44:0/4184457644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f063c074310 0x7f063c101d20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 >> 172.21.15.44:0/4184457644 conn(0x7f063c0fd890 msgr2=0x7f063c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 shutdown_connections 2024-04-20T11:22:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.103+0000 7f063be00640 1 -- 172.21.15.44:0/4184457644 wait complete. 2024-04-20T11:22:45.124 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 -- 172.21.15.44:0/1579545732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 msgr2=0x7f2c1c10bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 --2- 172.21.15.44:0/1579545732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 0x7f2c1c10bff0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802bd60 comp rx=0 tx=0).stop 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 -- 172.21.15.44:0/1579545732 shutdown_connections 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 --2- 172.21.15.44:0/1579545732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c1c10c530 0x7f2c1c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 --2- 172.21.15.44:0/1579545732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 0x7f2c1c10bff0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 --2- 172.21.15.44:0/1579545732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 -- 172.21.15.44:0/1579545732 >> 172.21.15.44:0/1579545732 conn(0x7f2c1c077f60 msgr2=0x7f2c1c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 -- 172.21.15.44:0/1579545732 shutdown_connections 2024-04-20T11:22:45.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.306+0000 7f2c20c00640 1 -- 172.21.15.44:0/1579545732 wait complete. 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.307+0000 7f2c20c00640 1 Processor -- start 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.307+0000 7f2c20c00640 1 -- start start 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c20c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c20c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 0x7f2c1c101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c20c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c1c10c530 0x7f2c1c1018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c20c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c112450 con 0x7f2c1c103b10 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c20c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1122d0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c20c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1125d0 con 0x7f2c1c10c530 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 0x7f2c1c101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36940/0 (socket says 172.21.15.44:36940) 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 -- 172.21.15.44:0/2563228730 learned_addr learned my addr 172.21.15.44:0/2563228730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c1a800640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c1c10c530 0x7f2c1c1018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 -- 172.21.15.44:0/2563228730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c1c10c530 msgr2=0x7f2c1c1018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c1c10c530 0x7f2c1c1018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 -- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 msgr2=0x7f2c1c101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 0x7f2c1c101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.308+0000 7f2c19e00640 1 -- 172.21.15.44:0/2563228730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c1a6df0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.309+0000 7f2c19e00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c100e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00e9f0 tx=0x7f2c0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.309+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00cd80 con 0x7f2c1c1031f0 2024-04-20T11:22:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.309+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c0c010930 con 0x7f2c1c1031f0 2024-04-20T11:22:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.309+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c1a70e0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.309+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c018b00 con 0x7f2c1c1031f0 2024-04-20T11:22:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.309+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1a7600 con 0x7f2c1c1031f0 2024-04-20T11:22:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.311+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c068d80 con 0x7f2c1c1031f0 2024-04-20T11:22:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.312+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f2c0c018ca0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.314+0000 7f2c12a00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c000a3e70 0x7f2c000a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.314+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2c0c0d0d90 con 0x7f2c1c1031f0 2024-04-20T11:22:45.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.314+0000 7f2c19400640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c000a3e70 0x7f2c000a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.315+0000 7f2c19400640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c000a3e70 0x7f2c000a6330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006010 tx=0x7f2c08005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.318+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2c0c09cba0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.534+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c1c1076b0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.537 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:22:45.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.536+0000 7f2c12a00640 1 -- 172.21.15.44:0/2563228730 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22887 (secure 0 0 0) 0x7f2c0c0a0ea0 con 0x7f2c1c1031f0 2024-04-20T11:22:45.538 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:22:36.090257+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c000a3e70 msgr2=0x7f2c000a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c000a3e70 0x7f2c000a6330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006010 tx=0x7f2c08005e40 comp rx=0 tx=0).stop 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 msgr2=0x7f2c1c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c100e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00e9f0 tx=0x7f2c0c00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 shutdown_connections 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c1c10c530 0x7f2c1c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c000a3e70 0x7f2c000a6330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c103b10 0x7f2c1c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 --2- 172.21.15.44:0/2563228730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c1c1031f0 0x7f2c1c100e40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 >> 172.21.15.44:0/2563228730 conn(0x7f2c1c077f60 msgr2=0x7f2c1c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 shutdown_connections 2024-04-20T11:22:45.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.539+0000 7f2c20c00640 1 -- 172.21.15.44:0/2563228730 wait complete. 2024-04-20T11:22:45.560 DEBUG:tasks.fwd_scrub:Yielding 2024-04-20T11:22:45.560 INFO:teuthology.run_tasks:Running task exec... 2024-04-20T11:22:45.569 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T11:22:45.569 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:22:45.570 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-20T11:22:45.570 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-20T11:22:45.570 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:22:45.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:22:45.570 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:22:45.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/143574230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:45 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4184457644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-04-20T11:22:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/143574230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:45 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4184457644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 -- 172.21.15.44:0/969287482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4112d80 msgr2=0x7fe7b4115170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 --2- 172.21.15.44:0/969287482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4112d80 0x7fe7b4115170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009ca0 tx=0x7fe7ac02c190 comp rx=0 tx=0).stop 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 -- 172.21.15.44:0/969287482 shutdown_connections 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 --2- 172.21.15.44:0/969287482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4112d80 0x7fe7b4115170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 --2- 172.21.15.44:0/969287482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4074c20 0x7fe7b4112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 --2- 172.21.15.44:0/969287482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074300 0x7fe7b40746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.770+0000 7fe7baa00640 1 -- 172.21.15.44:0/969287482 >> 172.21.15.44:0/969287482 conn(0x7fe7b40fd8e0 msgr2=0x7fe7b40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.771+0000 7fe7baa00640 1 -- 172.21.15.44:0/969287482 shutdown_connections 2024-04-20T11:22:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.771+0000 7fe7baa00640 1 -- 172.21.15.44:0/969287482 wait complete. 2024-04-20T11:22:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 Processor -- start 2024-04-20T11:22:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 -- start start 2024-04-20T11:22:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4074300 0x7fe7b41a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 0x7fe7b41a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 -- 172.21.15.44:0/4008711776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 msgr2=0x7f96bc102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4112d80 0x7fe7b419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41188a0 con 0x7fe7b4074300 2024-04-20T11:22:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4118720 con 0x7fe7b4074c20 2024-04-20T11:22:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7baa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4118a20 con 0x7fe7b4112d80 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 --2- 172.21.15.44:0/4008711776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc102580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009900 tx=0x7f96b002bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 -- 172.21.15.44:0/4008711776 shutdown_connections 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 --2- 172.21.15.44:0/4008711776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 0x7f96bc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 --2- 172.21.15.44:0/4008711776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96bc102b50 0x7f96bc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 --2- 172.21.15.44:0/4008711776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc102580 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 -- 172.21.15.44:0/4008711776 >> 172.21.15.44:0/4008711776 conn(0x7f96bc0fda00 msgr2=0x7f96bc0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7f96c4600640 1 -- 172.21.15.44:0/4008711776 shutdown_connections 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7b8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 0x7fe7b41a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7b8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 0x7fe7b41a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:36982/0 (socket says 172.21.15.44:36982) 2024-04-20T11:22:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7b8c00640 1 -- 172.21.15.44:0/3441536974 learned_addr learned my addr 172.21.15.44:0/3441536974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7ba000640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4112d80 0x7fe7b419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.772+0000 7fe7b9600640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4074300 0x7fe7b41a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7f96c4600640 1 -- 172.21.15.44:0/4008711776 wait complete. 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7ba000640 1 -- 172.21.15.44:0/3441536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 msgr2=0x7fe7b41a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7ba000640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 0x7fe7b41a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7ba000640 1 -- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4074300 msgr2=0x7fe7b41a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7ba000640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4074300 0x7fe7b41a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7ba000640 1 -- 172.21.15.44:0/3441536974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b419f260 con 0x7fe7b4112d80 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7b8c00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 0x7fe7b41a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.773+0000 7fe7ba000640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4112d80 0x7fe7b419eae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac037040 tx=0x7fe7ac034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac039040 con 0x7fe7b4112d80 2024-04-20T11:22:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b419f4f0 con 0x7fe7b4112d80 2024-04-20T11:22:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe7ac044070 con 0x7fe7b4112d80 2024-04-20T11:22:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0042a0 con 0x7fe7b4112d80 2024-04-20T11:22:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7f96c4600640 1 Processor -- start 2024-04-20T11:22:45.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b41ab7f0 con 0x7fe7b4112d80 2024-04-20T11:22:45.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7f96c4600640 1 -- start start 2024-04-20T11:22:45.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7f96c4600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.774+0000 7f96c4600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 0x7f96bc0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c4600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96bc074ce0 0x7f96bc1a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c4600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1123f0 con 0x7f96bc107270 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c4600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc112270 con 0x7f96bc1021a0 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c4600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc112570 con 0x7f96bc074ce0 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 0x7f96bc0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 0x7f96bc0747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50322/0 (socket says 172.21.15.44:50322) 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1200640 1 -- 172.21.15.44:0/3905659 learned_addr learned my addr 172.21.15.44:0/3905659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b4050540 con 0x7fe7b4112d80 2024-04-20T11:22:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1c00640 1 -- 172.21.15.44:0/3905659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96bc074ce0 msgr2=0x7f96bc1a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96bc074ce0 0x7f96bc1a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1c00640 1 -- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 msgr2=0x7f96bc0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 0x7f96bc0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.775+0000 7f96c1c00640 1 -- 172.21.15.44:0/3905659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc1a71e0 con 0x7f96bc1021a0 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.776+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc074260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009230 tx=0x7f96b002dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.776+0000 7f96ba000640 1 -- 172.21.15.44:0/3905659 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0043020 con 0x7f96bc1021a0 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.776+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc1a7470 con 0x7f96bc1021a0 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.776+0000 7f96ba000640 1 -- 172.21.15.44:0/3905659 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96b002de50 con 0x7f96bc1021a0 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.776+0000 7f96ba000640 1 -- 172.21.15.44:0/3905659 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0003a10 con 0x7f96bc1021a0 2024-04-20T11:22:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.776+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc1a7a10 con 0x7f96bc1021a0 2024-04-20T11:22:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.777+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fe7ac004440 con 0x7fe7b4112d80 2024-04-20T11:22:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.778+0000 7fe7b2000640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7980a3e30 0x7fe7980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.779+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe7ac0f2c20 con 0x7fe7b4112d80 2024-04-20T11:22:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.779+0000 7fe7b9600640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7980a3e30 0x7fe7980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.780+0000 7f96ba000640 1 -- 172.21.15.44:0/3905659 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f96b0046020 con 0x7f96bc1021a0 2024-04-20T11:22:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.780+0000 7fe7b9600640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7980a3e30 0x7fe7980a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009bd0 tx=0x7fe7a4005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.780+0000 7f96c7381740 1 Processor -- start 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.780+0000 7f96c7381740 1 -- start start 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.780+0000 7f96c7381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 0x55a382cca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.780+0000 7f96c7381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cca830 0x55a382cd2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c7381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c7381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a382c2ed20 con 0x55a382cc9f10 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c7381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a382bd53e0 con 0x55a382cd3410 2024-04-20T11:22:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c7381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a382c2ace0 con 0x55a382cca830 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 0x55a382cca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 0x55a382cca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50328/0 (socket says 172.21.15.44:50328) 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 -- 172.21.15.44:0/3019954285 learned_addr learned my addr 172.21.15.44:0/3019954285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cca830 0x55a382cd2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c1200640 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 -- 172.21.15.44:0/3019954285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cca830 msgr2=0x55a382cd2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cca830 0x55a382cd2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 -- 172.21.15.44:0/3019954285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 msgr2=0x55a382cd5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c2600640 1 -- 172.21.15.44:0/3019954285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a382bf4980 con 0x55a382cc9f10 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cca830 0x55a382cd2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.781+0000 7f96ba000640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f96a40a3e40 0x7f96a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.782+0000 7f96ba000640 1 -- 172.21.15.44:0/3905659 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f96b00f2870 con 0x7f96bc1021a0 2024-04-20T11:22:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.782+0000 7f96c2600640 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 0x55a382cca2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e20 tx=0x7f96b400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315cc9a54002b046 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.782+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f96a40a3e40 0x7f96a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.782+0000 7f96aea00640 1 -- 172.21.15.44:0/3019954285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4004d20 con 0x55a382cc9f10 2024-04-20T11:22:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.782+0000 7f96aea00640 1 -- 172.21.15.44:0/3019954285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96b400ee70 con 0x55a382cc9f10 2024-04-20T11:22:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.782+0000 7f96aea00640 1 -- 172.21.15.44:0/3019954285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4010990 con 0x55a382cc9f10 2024-04-20T11:22:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe7ac0bf190 con 0x7fe7b4112d80 2024-04-20T11:22:45.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 -- 172.21.15.44:0/3019954285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 msgr2=0x55a382cca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 0x55a382cca2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e20 tx=0x7f96b400e720 comp rx=0 tx=0).stop 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c1c00640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f96a40a3e40 0x7f96a40a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96b00f0020 tx=0x7f96b0009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 -- 172.21.15.44:0/3019954285 shutdown_connections 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cca830 0x55a382cd2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 --2- 172.21.15.44:0/3019954285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cc9f10 0x55a382cca2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 -- 172.21.15.44:0/3019954285 >> 172.21.15.44:0/3019954285 conn(0x55a382cc4640 msgr2=0x55a382cc4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 -- 172.21.15.44:0/3019954285 shutdown_connections 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.783+0000 7f96c7381740 1 -- 172.21.15.44:0/3019954285 wait complete. 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 -- start start 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 Processor -- start 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a382cd6200 con 0x55a382cca830 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a382be87a0 con 0x55a382cd3410 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a382cd6380 con 0x55a382cc9f10 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.784+0000 7f96c7381740 10 client.0 _ll_register_callbacks cb 0x55a382beab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48722/0 (socket says 172.21.15.44:48722) 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 -- 172.21.15.44:0/1224933256 learned_addr learned my addr 172.21.15.44:0/1224933256 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 -- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 msgr2=0x55a382cd0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 -- 172.21.15.44:0/1224933256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 msgr2=0x55a382cd09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.785+0000 7f96c2600640 1 -- 172.21.15.44:0/1224933256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a382a097e0 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96c2600640 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd04b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009fd0 tx=0x7f96b4010570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606d379ac307cd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96a3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96a3400640 1 -- 172.21.15.44:0/1224933256 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b401c030 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96a3400640 1 -- 172.21.15.44:0/1224933256 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f96b4010d70 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b400ee70 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96a3400640 1 -- 172.21.15.44:0/1224933256 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96b4030050 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96c7381740 10 client.25489 fetch_fsmap learned FSMap version 27 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.786+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a382a69f70 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96a3400640 1 -- 172.21.15.44:0/1224933256 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96b400ee70 con 0x55a382cc9f10 2024-04-20T11:22:45.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96a3400640 1 -- 172.21.15.44:0/1224933256 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f96b4004660 con 0x55a382cc9f10 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c7381740 10 client.25489 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c7381740 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c7381740 20 client.25489 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c7381740 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382ce36c0 0x55a382ce5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c7381740 4 client.25489 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a382c3c740 con 0x55a382ce36c0 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.787+0000 7f96c1200640 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382ce36c0 0x55a382ce5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.788+0000 7f96c1200640 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382ce36c0 0x55a382ce5ab0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0794f8536c5615d server_cookie=2d1926c2e81baeed in_seq=0 out_seq=0 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.788+0000 7f96a3400640 10 client.25489 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96a3400640 1 -- 172.21.15.44:0/1224933256 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a382c3c740 con 0x55a382ce36c0 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96a3400640 10 client.25489 handle_command_reply: tid=0 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 client.25489 shutdown 2024-04-20T11:22:45.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 msgr2=0x55a382cd04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd04b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009fd0 tx=0x7f96b4010570 comp rx=0 tx=0).stop 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 shutdown_connections 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382ce36c0 0x55a382ce5ab0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 --2- 172.21.15.44:0/1224933256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd04b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 >> 172.21.15.44:0/1224933256 conn(0x55a382cc4640 msgr2=0x55a382cc6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96a3400640 0 client.25489 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 shutdown_connections 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 1 -- 172.21.15.44:0/1224933256 wait complete. 2024-04-20T11:22:45.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.789+0000 7f96c7381740 20 client.25489 trim_cache size 0 max 16384 2024-04-20T11:22:45.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 Processor -- start 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 -- start start 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cdf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1afe40 con 0x55a382cca830 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b1980 con 0x55a382cd3410 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c7381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b0510 con 0x55a382cc9f10 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.809+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cdf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cdf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50336/0 (socket says 172.21.15.44:50336) 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 -- 172.21.15.44:0/645265484 learned_addr learned my addr 172.21.15.44:0/645265484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48728/0 (socket says 172.21.15.44:48728) 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c2600640 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 -- 172.21.15.44:0/645265484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 msgr2=0x55a382cd2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 -- 172.21.15.44:0/645265484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 msgr2=0x55a382cd0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c1200640 1 -- 172.21.15.44:0/645265484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a382cd5b70 con 0x55a382cca830 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.810+0000 7f96c2600640 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.811+0000 7f96c1200640 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cdf0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96bc075ca0 tx=0x7f96a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144ae6c031bf22c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.811+0000 7f96ae000640 1 -- 172.21.15.44:0/645265484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8004110 con 0x55a382cca830 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.811+0000 7f96ae000640 1 -- 172.21.15.44:0/645265484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a8005520 con 0x55a382cca830 2024-04-20T11:22:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.811+0000 7f96ae000640 1 -- 172.21.15.44:0/645265484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80035f0 con 0x55a382cca830 2024-04-20T11:22:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 -- 172.21.15.44:0/645265484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 msgr2=0x55a382cdf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cdf0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96bc075ca0 tx=0x7f96a8005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 -- 172.21.15.44:0/645265484 shutdown_connections 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382cd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cdf0c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 --2- 172.21.15.44:0/645265484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 -- 172.21.15.44:0/645265484 >> 172.21.15.44:0/645265484 conn(0x55a382cc4640 msgr2=0x55a382cd0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 -- 172.21.15.44:0/645265484 shutdown_connections 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 -- 172.21.15.44:0/645265484 wait complete. 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.812+0000 7f96c7381740 1 -- start start 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 Processor -- start 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382d48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b12b0 con 0x55a382cca830 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b0be0 con 0x55a382cd3410 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1ade10 con 0x55a382cc9f10 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c7381740 10 client.0 _ll_register_callbacks cb 0x7f96b4009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50340/0 (socket says 172.21.15.44:50340) 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c1200640 1 -- 172.21.15.44:0/822702781 learned_addr learned my addr 172.21.15.44:0/822702781 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.813+0000 7f96c2600640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382d48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 -- 172.21.15.44:0/822702781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 msgr2=0x55a382cd76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 -- 172.21.15.44:0/822702781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 msgr2=0x55a382cd7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 -- 172.21.15.44:0/822702781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a382901170 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c2600640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382d48b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96b4013040 tx=0x7f96b4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3316c00368d7ced server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96a3400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96a3400640 1 -- 172.21.15.44:0/822702781 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b401c020 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a8005520 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.814+0000 7f96a3400640 1 -- 172.21.15.44:0/822702781 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f96b4018cc0 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96a3400640 1 -- 172.21.15.44:0/822702781 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96b4022040 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96c7381740 10 client.25493 fetch_fsmap learned FSMap version 27 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a382d4a160 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96a3400640 1 -- 172.21.15.44:0/822702781 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f96a8005520 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96a3400640 1 -- 172.21.15.44:0/822702781 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f96b403f410 con 0x55a382cd3410 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96c7381740 10 client.25493 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96c7381740 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:22:45.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96c7381740 20 client.25493 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.815+0000 7f96c7381740 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382d4a7c0 0x55a382d4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.816+0000 7f96c7381740 4 client.25493 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:22:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.816+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a382c365b0 con 0x55a382d4a7c0 2024-04-20T11:22:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.816+0000 7f96c1200640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382d4a7c0 0x55a382d4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.816+0000 7f96c1200640 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382d4a7c0 0x55a382d4cbe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef02c1477fb73c78 server_cookie=788d200f9034f6e1 in_seq=0 out_seq=0 2024-04-20T11:22:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.816+0000 7f96a3400640 10 client.25493 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96a3400640 1 -- 172.21.15.44:0/822702781 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a382c365b0 con 0x55a382d4a7c0 2024-04-20T11:22:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96a3400640 10 client.25493 handle_command_reply: tid=0 2024-04-20T11:22:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 client.25493 shutdown 2024-04-20T11:22:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 msgr2=0x55a382d48b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382d48b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96b4013040 tx=0x7f96b4000f40 comp rx=0 tx=0).stop 2024-04-20T11:22:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 shutdown_connections 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a382d4a7c0 0x55a382d4cbe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a382cd3410 0x55a382d48b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a382cca830 0x55a382cd7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96a3400640 0 client.25493 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 --2- 172.21.15.44:0/822702781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a382cc9f10 0x55a382cd76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 >> 172.21.15.44:0/822702781 conn(0x55a382cc4640 msgr2=0x55a382cc3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 shutdown_connections 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 1 -- 172.21.15.44:0/822702781 wait complete. 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.818+0000 7f96c7381740 20 client.25493 trim_cache size 0 max 16384 2024-04-20T11:22:45.822 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:45.823 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:22:45.823 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "7b974030-bb2b-4992-bbc1-76c00458df4a", 2024-04-20T11:22:45.823 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:22:45.823 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f96a40a3e40 msgr2=0x7f96a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f96a40a3e40 0x7f96a40a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96b00f0020 tx=0x7f96b0009260 comp rx=0 tx=0).stop 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 msgr2=0x7f96bc074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc074260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009230 tx=0x7f96b002dc60 comp rx=0 tx=0).stop 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 shutdown_connections 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96bc074ce0 0x7f96bc1a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f96a40a3e40 0x7f96a40a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96bc107270 0x7f96bc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 --2- 172.21.15.44:0/3905659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96bc1021a0 0x7f96bc074260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.819+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 >> 172.21.15.44:0/3905659 conn(0x7f96bc0fda00 msgr2=0x7f96bc0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.820+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 shutdown_connections 2024-04-20T11:22:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:45.820+0000 7f96c4600640 1 -- 172.21.15.44:0/3905659 wait complete. 2024-04-20T11:22:45.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:22:45.839 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: Updating smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/143574230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:22:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:45 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4184457644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:22:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.020+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe7b41052c0 con 0x7fe7980a3e30 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 -- 172.21.15.44:0/2390395463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 msgr2=0x7f3440100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 --2- 172.21.15.44:0/2390395463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440100560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f342c009930 tx=0x7f342c02bd10 comp rx=0 tx=0).stop 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 -- 172.21.15.44:0/2390395463 shutdown_connections 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 --2- 172.21.15.44:0/2390395463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 0x7f344010eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 --2- 172.21.15.44:0/2390395463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440100560 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 --2- 172.21.15.44:0/2390395463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34401070e0 0x7f34401074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 -- 172.21.15.44:0/2390395463 >> 172.21.15.44:0/2390395463 conn(0x7f3440076fc0 msgr2=0x7f34400fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 -- 172.21.15.44:0/2390395463 shutdown_connections 2024-04-20T11:22:46.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.036+0000 7f3448200640 1 -- 172.21.15.44:0/2390395463 wait complete. 2024-04-20T11:22:46.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 Processor -- start 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 -- start start 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 0x7f34401020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34401070e0 0x7f34401025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3440112490 con 0x7f34401070e0 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3440112310 con 0x7f3440100100 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.038+0000 7f3448200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3440112610 con 0x7f3440100aa0 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3444e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 0x7f34401020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37068/0 (socket says 172.21.15.44:37068) 2024-04-20T11:22:46.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 -- 172.21.15.44:0/939754757 learned_addr learned my addr 172.21.15.44:0/939754757 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3446200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34401070e0 0x7f34401025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 -- 172.21.15.44:0/939754757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 msgr2=0x7f34401020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 0x7f34401020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 -- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34401070e0 msgr2=0x7f34401025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34401070e0 0x7f34401025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3445800640 1 -- 172.21.15.44:0/939754757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34401a6e50 con 0x7f3440100100 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.039+0000 7f3444e00640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 0x7f34401020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.040+0000 7f3445800640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440101b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3434009d50 tx=0x7f343400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.040+0000 7f343e000640 1 -- 172.21.15.44:0/939754757 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434003b70 con 0x7f3440100100 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.040+0000 7f343e000640 1 -- 172.21.15.44:0/939754757 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3434003d10 con 0x7f3440100100 2024-04-20T11:22:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.040+0000 7f343e000640 1 -- 172.21.15.44:0/939754757 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434010a90 con 0x7f3440100100 2024-04-20T11:22:46.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.041+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401a7140 con 0x7f3440100100 2024-04-20T11:22:46.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.041+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401a7610 con 0x7f3440100100 2024-04-20T11:22:46.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.045+0000 7f344b003740 1 Processor -- start 2024-04-20T11:22:46.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.045+0000 7f344b003740 1 -- start start 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f343e000640 1 -- 172.21.15.44:0/939754757 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f3434004000 con 0x7f3440100100 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f344b003740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52797f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f344b003740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527988160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f344b003740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e52798aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f344b003740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e5278e5150 con 0x55e52797f1a0 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f344b003740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e52788b370 con 0x55e5279886a0 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f344b003740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e5278e1110 con 0x55e52797fac0 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f3445800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527988160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f3444e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e52798aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f3446200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52797f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f3445800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527988160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48758/0 (socket says 172.21.15.44:48758) 2024-04-20T11:22:46.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.046+0000 7f3445800640 1 -- 172.21.15.44:0/1422394973 learned_addr learned my addr 172.21.15.44:0/1422394973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3444e00640 1 -- 172.21.15.44:0/1422394973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 msgr2=0x55e527988160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3444e00640 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527988160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3444e00640 1 -- 172.21.15.44:0/1422394973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 msgr2=0x55e52797f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3444e00640 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52797f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3444e00640 1 -- 172.21.15.44:0/1422394973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5276c17d0 con 0x55e5279886a0 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3445800640 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527988160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3444e00640 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e52798aa90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f342c02c250 tx=0x7f342c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ae72ca272d98aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3432a00640 1 -- 172.21.15.44:0/1422394973 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c045020 con 0x55e5279886a0 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.047+0000 7f3432a00640 1 -- 172.21.15.44:0/1422394973 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f342c02c670 con 0x55e5279886a0 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.048+0000 7f343e000640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34280a3ec0 0x7f34280a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.048+0000 7f3432a00640 1 -- 172.21.15.44:0/1422394973 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c004920 con 0x55e5279886a0 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.048+0000 7f343e000640 1 -- 172.21.15.44:0/939754757 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f34340d10a0 con 0x7f3440100100 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.048+0000 7f344b003740 1 -- 172.21.15.44:0/1422394973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 msgr2=0x55e52798aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.048+0000 7f344b003740 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e52798aa90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f342c02c250 tx=0x7f342c0095f0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f3446200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34280a3ec0 0x7f34280a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 -- 172.21.15.44:0/1422394973 shutdown_connections 2024-04-20T11:22:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e52798aa90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527988160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 --2- 172.21.15.44:0/1422394973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52797f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 -- 172.21.15.44:0/1422394973 >> 172.21.15.44:0/1422394973 conn(0x55e5279798d0 msgr2=0x55e527979cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 -- 172.21.15.44:0/1422394973 shutdown_connections 2024-04-20T11:22:46.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f344b003740 1 -- 172.21.15.44:0/1422394973 wait complete. 2024-04-20T11:22:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.049+0000 7f3446200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34280a3ec0 0x7f34280a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3438009ca0 tx=0x7f3438005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 -- start start 2024-04-20T11:22:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 Processor -- start 2024-04-20T11:22:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e5279856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527985c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527986150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e52798b430 con 0x55e52797f1a0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e52789e740 con 0x55e5279886a0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.050+0000 7f344b003740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e52798b5b0 con 0x55e52797fac0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f344b003740 10 client.0 _ll_register_callbacks cb 0x55e527899b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527985c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527985c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48766/0 (socket says 172.21.15.44:48766) 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 -- 172.21.15.44:0/2032581799 learned_addr learned my addr 172.21.15.44:0/2032581799 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3445800640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e5279856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3446200640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527986150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 -- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 msgr2=0x55e527986150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527986150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 -- 172.21.15.44:0/2032581799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 msgr2=0x55e5279856d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e5279856d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.051+0000 7f3444e00640 1 -- 172.21.15.44:0/2032581799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5275f73f0 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f3446200640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527986150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f3444e00640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527985c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f342c000c00 tx=0x7f342c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c554974253ddd9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f3427400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f3427400640 1 -- 172.21.15.44:0/2032581799 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c04e070 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f3427400640 1 -- 172.21.15.44:0/2032581799 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f342c003800 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c02c670 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f3427400640 1 -- 172.21.15.44:0/2032581799 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f342c045070 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f344b003740 10 client.25507 fetch_fsmap learned FSMap version 27 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.052+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5275b70f0 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f3427400640 1 -- 172.21.15.44:0/2032581799 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f342c02c670 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f3427400640 1 -- 172.21.15.44:0/2032581799 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f342c051030 con 0x55e52797fac0 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f344b003740 10 client.25507 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f344b003740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f344b003740 20 client.25507 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f344b003740 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e527998850 0x55e52799ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f344b003740 4 client.25507 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:22:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.053+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5278f2b70 con 0x55e527998850 2024-04-20T11:22:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.054+0000 7f3445800640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e527998850 0x55e52799ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.054+0000 7f3445800640 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e527998850 0x55e52799ac40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4d52b5dfb0cf22 server_cookie=8542f30447b79f64 in_seq=0 out_seq=0 2024-04-20T11:22:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.054+0000 7f3427400640 10 client.25507 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.055+0000 7f3427400640 1 -- 172.21.15.44:0/2032581799 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5278f2b70 con 0x55e527998850 2024-04-20T11:22:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.055+0000 7f3427400640 10 client.25507 handle_command_reply: tid=0 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.055+0000 7f344b003740 1 client.25507 shutdown 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.055+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 msgr2=0x55e527985c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527985c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f342c000c00 tx=0x7f342c004610 comp rx=0 tx=0).stop 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 shutdown_connections 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e527998850 0x55e52799ac40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527986150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e527985c10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 --2- 172.21.15.44:0/2032581799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e5279856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 >> 172.21.15.44:0/2032581799 conn(0x55e5279798d0 msgr2=0x55e52797b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f3427400640 0 client.25507 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 shutdown_connections 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 1 -- 172.21.15.44:0/2032581799 wait complete. 2024-04-20T11:22:46.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.056+0000 7f344b003740 20 client.25507 trim_cache size 0 max 16384 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 Processor -- start 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 -- start start 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e527987e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e5279942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527984610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34401afdd0 con 0x55e52797f1a0 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b1910 con 0x55e5279886a0 2024-04-20T11:22:46.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.078+0000 7f344b003740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b04a0 con 0x55e52797fac0 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e5279942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3445800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e527987e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e5279942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48782/0 (socket says 172.21.15.44:48782) 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3446200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527984610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 -- 172.21.15.44:0/1786530334 learned_addr learned my addr 172.21.15.44:0/1786530334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 -- 172.21.15.44:0/1786530334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 msgr2=0x55e527984610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527984610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 -- 172.21.15.44:0/1786530334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 msgr2=0x55e527987e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e527987e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 -- 172.21.15.44:0/1786530334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52797a250 con 0x55e52797fac0 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.079+0000 7f3444e00640 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e5279942f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f342c02bd50 tx=0x7f342c009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc7b7c7eeca3dda server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.080+0000 7f3432000640 1 -- 172.21.15.44:0/1786530334 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c04e070 con 0x55e52797fac0 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.080+0000 7f3432000640 1 -- 172.21.15.44:0/1786530334 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f342c003d00 con 0x55e52797fac0 2024-04-20T11:22:46.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.080+0000 7f3432000640 1 -- 172.21.15.44:0/1786530334 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c041a40 con 0x55e52797fac0 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 -- 172.21.15.44:0/1786530334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 msgr2=0x55e5279942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e5279942f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f342c02bd50 tx=0x7f342c009670 comp rx=0 tx=0).stop 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 -- 172.21.15.44:0/1786530334 shutdown_connections 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5279886a0 0x55e527984610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e52797fac0 0x55e5279942f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 --2- 172.21.15.44:0/1786530334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e527987e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 -- 172.21.15.44:0/1786530334 >> 172.21.15.44:0/1786530334 conn(0x55e5279798d0 msgr2=0x55e5279788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 -- 172.21.15.44:0/1786530334 shutdown_connections 2024-04-20T11:22:46.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.081+0000 7f344b003740 1 -- 172.21.15.44:0/1786530334 wait complete. 2024-04-20T11:22:46.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.082+0000 7f344b003740 1 -- start start 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.082+0000 7f344b003740 1 Processor -- start 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52798c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e52797fac0 0x55e52798c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5279886a0 0x55e52798cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b1240 con 0x55e52797f1a0 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b0b70 con 0x55e52797fac0 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34401addc0 con 0x55e5279886a0 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f344b003740 10 client.0 _ll_register_callbacks cb 0x7f34400773d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3445800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52798c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3446200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5279886a0 0x55e52798cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3444e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e52797fac0 0x55e52798c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3445800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52798c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50410/0 (socket says 172.21.15.44:50410) 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3445800640 1 -- 172.21.15.44:0/2838139701 learned_addr learned my addr 172.21.15.44:0/2838139701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3444e00640 1 -- 172.21.15.44:0/2838139701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5279886a0 msgr2=0x55e52798cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3444e00640 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5279886a0 0x55e52798cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3444e00640 1 -- 172.21.15.44:0/2838139701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 msgr2=0x55e52798c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3444e00640 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52798c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.083+0000 7f3444e00640 1 -- 172.21.15.44:0/2838139701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5279fdb90 con 0x55e52797fac0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f3444e00640 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e52797fac0 0x55e52798c860 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55e527973ee0 tx=0x7f342c02c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ddc0be75bd8873 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f3427400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f3427400640 1 -- 172.21.15.44:0/2838139701 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c060020 con 0x55e52797fac0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c003d00 con 0x55e52797fac0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f3427400640 1 -- 172.21.15.44:0/2838139701 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f342c0416c0 con 0x55e52797fac0 2024-04-20T11:22:46.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f3427400640 1 -- 172.21.15.44:0/2838139701 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f342c04e070 con 0x55e52797fac0 2024-04-20T11:22:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f344b003740 10 client.25511 fetch_fsmap learned FSMap version 27 2024-04-20T11:22:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.084+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5279fee80 con 0x55e52797fac0 2024-04-20T11:22:46.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f3427400640 1 -- 172.21.15.44:0/2838139701 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f342c003d00 con 0x55e52797fac0 2024-04-20T11:22:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f3427400640 1 -- 172.21.15.44:0/2838139701 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f342c05a020 con 0x55e52797fac0 2024-04-20T11:22:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f344b003740 10 client.25511 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:22:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f344b003740 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:22:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f344b003740 20 client.25511 populate_metadata read hostname 'smithi044' 2024-04-20T11:22:46.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f344b003740 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e5279ff3b0 0x55e527a017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f344b003740 4 client.25511 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.085+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5278ec9e0 con 0x55e5279ff3b0 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.086+0000 7f3445800640 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e5279ff3b0 0x55e527a017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.086+0000 7f3445800640 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e5279ff3b0 0x55e527a017a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5144c3fcae2c3672 server_cookie=6016fb4a093d2c33 in_seq=0 out_seq=0 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.086+0000 7f3427400640 10 client.25511 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.086+0000 7f3427400640 1 -- 172.21.15.44:0/2838139701 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e5278ec9e0 con 0x55e5279ff3b0 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.086+0000 7f3427400640 10 client.25511 handle_command_reply: tid=0 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.086+0000 7f344b003740 1 client.25511 shutdown 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e52797fac0 msgr2=0x55e52798c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e52797fac0 0x55e52798c860 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55e527973ee0 tx=0x7f342c02c520 comp rx=0 tx=0).stop 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 shutdown_connections 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e5279ff3b0 0x55e527a017a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5279886a0 0x55e52798cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e52797fac0 0x55e52798c860 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 --2- 172.21.15.44:0/2838139701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e52797f1a0 0x55e52798c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 >> 172.21.15.44:0/2838139701 conn(0x55e5279798d0 msgr2=0x55e527985690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f3427400640 0 client.25511 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 shutdown_connections 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 1 -- 172.21.15.44:0/2838139701 wait complete. 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.087+0000 7f344b003740 20 client.25511 trim_cache size 0 max 16384 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:22:46.090 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout: "7b974030-bb2b-4992-bbc1-76c00458df4a": { 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "7b974030-bb2b-4992-bbc1-76c00458df4a", 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:22:46.091 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34280a3ec0 msgr2=0x7f34280a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34280a3ec0 0x7f34280a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3438009ca0 tx=0x7f3438005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 msgr2=0x7f3440101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440101b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3434009d50 tx=0x7f343400c6a0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 shutdown_connections 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34280a3ec0 0x7f34280a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34401070e0 0x7f34401025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3440100aa0 0x7f34401020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 --2- 172.21.15.44:0/939754757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3440100100 0x7f3440101b70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.088+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 >> 172.21.15.44:0/939754757 conn(0x7f3440076fc0 msgr2=0x7f34400fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.089+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 shutdown_connections 2024-04-20T11:22:46.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.089+0000 7f3448200640 1 -- 172.21.15.44:0/939754757 wait complete. 2024-04-20T11:22:46.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.105+0000 7fe7b2000640 1 -- 172.21.15.44:0/3441536974 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe7a401b3f0 con 0x7fe7980a3e30 2024-04-20T11:22:46.106 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path / 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7980a3e30 msgr2=0x7fe7980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7980a3e30 0x7fe7980a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009bd0 tx=0x7fe7a4005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4112d80 msgr2=0x7fe7b419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4112d80 0x7fe7b419eae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac037040 tx=0x7fe7ac034670 comp rx=0 tx=0).stop 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 shutdown_connections 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b4112d80 0x7fe7b419eae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7b4074c20 0x7fe7b41a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7980a3e30 0x7fe7980a62f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 --2- 172.21.15.44:0/3441536974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b4074300 0x7fe7b41a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 >> 172.21.15.44:0/3441536974 conn(0x7fe7b40fd8e0 msgr2=0x7fe7b40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.110 INFO:tasks.cephfs.filesystem:scrub status for tag:7b974030-bb2b-4992-bbc1-76c00458df4a - {'path': '/', 'tag': '7b974030-bb2b-4992-bbc1-76c00458df4a', 'options': 'recursive,force'} 2024-04-20T11:22:46.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 shutdown_connections 2024-04-20T11:22:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.108+0000 7fe7baa00640 1 -- 172.21.15.44:0/3441536974 wait complete. 2024-04-20T11:22:46.127 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-20T11:22:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.377+0000 7fac2ec00640 1 -- 172.21.15.44:0/2385747767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281072b0 msgr2=0x7fac2810ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.377+0000 7fac2ec00640 1 --2- 172.21.15.44:0/2385747767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281072b0 0x7fac2810ee80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac20009ca0 tx=0x7fac2002c0f0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 -- 172.21.15.44:0/2385747767 shutdown_connections 2024-04-20T11:22:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 --2- 172.21.15.44:0/2385747767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281072b0 0x7fac2810ee80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 --2- 172.21.15.44:0/2385747767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac28102b90 0x7fac28106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 --2- 172.21.15.44:0/2385747767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281021e0 0x7fac281025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 -- 172.21.15.44:0/2385747767 >> 172.21.15.44:0/2385747767 conn(0x7fac280793f0 msgr2=0x7fac28079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 -- 172.21.15.44:0/2385747767 shutdown_connections 2024-04-20T11:22:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.378+0000 7fac2ec00640 1 -- 172.21.15.44:0/2385747767 wait complete. 2024-04-20T11:22:46.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 Processor -- start 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 -- start start 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281021e0 0x7fac281a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 0x7fac281a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2819a7e0 0x7fac2819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac281124a0 con 0x7fac281072b0 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac28112320 con 0x7fac281021e0 2024-04-20T11:22:46.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.379+0000 7fac2ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac28112620 con 0x7fac2819a7e0 2024-04-20T11:22:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 0x7fac281a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2819a7e0 0x7fac2819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281021e0 0x7fac281a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 0x7fac281a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50424/0 (socket says 172.21.15.44:50424) 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 -- 172.21.15.44:0/1393513779 learned_addr learned my addr 172.21.15.44:0/1393513779 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 -- 172.21.15.44:0/1393513779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2819a7e0 msgr2=0x7fac2819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2819a7e0 0x7fac2819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 -- 172.21.15.44:0/1393513779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281021e0 msgr2=0x7fac281a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281021e0 0x7fac281a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.380+0000 7fac27400640 1 -- 172.21.15.44:0/1393513779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2819b370 con 0x7fac281072b0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.381+0000 7fac27400640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 0x7fac281a0bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac1000e9f0 tx=0x7fac1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.381+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1000ccc0 con 0x7fac281072b0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.381+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2819b660 con 0x7fac281072b0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.381+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac281a7840 con 0x7fac281072b0 2024-04-20T11:22:46.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.381+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac10021940 con 0x7fac281072b0 2024-04-20T11:22:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.381+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10010940 con 0x7fac281072b0 2024-04-20T11:22:46.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.382+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac040052c0 con 0x7fac281072b0 2024-04-20T11:22:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.384+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fac10021440 con 0x7fac281072b0 2024-04-20T11:22:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.386+0000 7fac24c00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fac0c0a3ee0 0x7fac0c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.386+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fac100d0d60 con 0x7fac281072b0 2024-04-20T11:22:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.386+0000 7fac27e00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fac0c0a3ee0 0x7fac0c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:46.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.387+0000 7fac27e00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fac0c0a3ee0 0x7fac0c0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac18009bd0 tx=0x7fac18005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.391+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fac2819b660 con 0x7fac281072b0 2024-04-20T11:22:46.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.630+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fac04000c90 con 0x7fac0c0a3ee0 2024-04-20T11:22:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:46 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2563228730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:46 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:22:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:46 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:22:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:46 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:22:46.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:46 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:22:46.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.789+0000 7fac24c00640 1 -- 172.21.15.44:0/1393513779 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fac04000c90 con 0x7fac0c0a3ee0 2024-04-20T11:22:46.790 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path / 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fac0c0a3ee0 msgr2=0x7fac0c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fac0c0a3ee0 0x7fac0c0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac18009bd0 tx=0x7fac18005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 msgr2=0x7fac281a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 0x7fac281a0bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac1000e9f0 tx=0x7fac1000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 shutdown_connections 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2819a7e0 0x7fac2819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fac0c0a3ee0 0x7fac0c0a63a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac281072b0 0x7fac281a0bb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 --2- 172.21.15.44:0/1393513779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac281021e0 0x7fac281a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:46.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.792+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 >> 172.21.15.44:0/1393513779 conn(0x7fac280793f0 msgr2=0x7fac28100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:46.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.793+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 shutdown_connections 2024-04-20T11:22:46.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:46.793+0000 7fac2ec00640 1 -- 172.21.15.44:0/1393513779 wait complete. 2024-04-20T11:22:46.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:46 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2563228730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:46.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:46 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:22:46.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:46 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:22:46.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:46 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:22:46.813 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:46 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:22:46.816 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T11:22:46.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:46 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2563228730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:22:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:46 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:22:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:46 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:22:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:46 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:22:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:46 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:22:47.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.021+0000 7fbf6b000640 1 -- 172.21.15.44:0/1182988416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf6410e430 msgr2=0x7fbf6410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.021+0000 7fbf6b000640 1 --2- 172.21.15.44:0/1182988416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf6410e430 0x7fbf6410e810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009930 tx=0x7fbf5802bc20 comp rx=0 tx=0).stop 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 -- 172.21.15.44:0/1182988416 shutdown_connections 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 --2- 172.21.15.44:0/1182988416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf64108e20 0x7fbf64115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 --2- 172.21.15.44:0/1182988416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf641084a0 0x7fbf641088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 --2- 172.21.15.44:0/1182988416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf6410e430 0x7fbf6410e810 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 -- 172.21.15.44:0/1182988416 >> 172.21.15.44:0/1182988416 conn(0x7fbf64077d40 msgr2=0x7fbf64106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 -- 172.21.15.44:0/1182988416 shutdown_connections 2024-04-20T11:22:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.022+0000 7fbf6b000640 1 -- 172.21.15.44:0/1182988416 wait complete. 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 Processor -- start 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 -- start start 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf641084a0 0x7fbf6419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf64108e20 0x7fbf6419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 0x7fbf641a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641188c0 con 0x7fbf641084a0 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64118740 con 0x7fbf64108e20 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.023+0000 7fbf6b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64118a40 con 0x7fbf6410e430 2024-04-20T11:22:47.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf69c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf641084a0 0x7fbf6419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 0x7fbf641a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 0x7fbf641a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48816/0 (socket says 172.21.15.44:48816) 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 -- 172.21.15.44:0/2914014879 learned_addr learned my addr 172.21.15.44:0/2914014879 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf69200640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf64108e20 0x7fbf6419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 -- 172.21.15.44:0/2914014879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf64108e20 msgr2=0x7fbf6419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf64108e20 0x7fbf6419f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 -- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf641084a0 msgr2=0x7fbf6419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf641084a0 0x7fbf6419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.024+0000 7fbf6a600640 1 -- 172.21.15.44:0/2914014879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf641a3d90 con 0x7fbf6410e430 2024-04-20T11:22:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.025+0000 7fbf6a600640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 0x7fbf641a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00eb10 tx=0x7fbf5c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:47.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.025+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c004b20 con 0x7fbf6410e430 2024-04-20T11:22:47.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.025+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf5c004cc0 con 0x7fbf6410e430 2024-04-20T11:22:47.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.025+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf641a4080 con 0x7fbf6410e430 2024-04-20T11:22:47.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.025+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0107f0 con 0x7fbf6410e430 2024-04-20T11:22:47.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.025+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641ab910 con 0x7fbf6410e430 2024-04-20T11:22:47.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.026+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf64050540 con 0x7fbf6410e430 2024-04-20T11:22:47.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.029+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fbf5c010990 con 0x7fbf6410e430 2024-04-20T11:22:47.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.030+0000 7fbf62000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf4c0a3e30 0x7fbf4c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.030+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbf5c0d1130 con 0x7fbf6410e430 2024-04-20T11:22:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.030+0000 7fbf69c00640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf4c0a3e30 0x7fbf4c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.031+0000 7fbf69c00640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf4c0a3e30 0x7fbf4c0a62f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005ab0 tx=0x7fbf58005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.032+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf5c053ff0 con 0x7fbf6410e430 2024-04-20T11:22:47.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.269+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fbf640623a0 con 0x7fbf4c0a3e30 2024-04-20T11:22:47.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.287+0000 7fbf62000640 1 -- 172.21.15.44:0/2914014879 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fbf640623a0 con 0x7fbf4c0a3e30 2024-04-20T11:22:47.290 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T11:22:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-20T11:22:47.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf4c0a3e30 msgr2=0x7fbf4c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf4c0a3e30 0x7fbf4c0a62f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005ab0 tx=0x7fbf58005fd0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 msgr2=0x7fbf641a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 0x7fbf641a3660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00eb10 tx=0x7fbf5c00c2d0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 shutdown_connections 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6410e430 0x7fbf641a3660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf64108e20 0x7fbf6419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf4c0a3e30 0x7fbf4c0a62f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 --2- 172.21.15.44:0/2914014879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf641084a0 0x7fbf6419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.289+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 >> 172.21.15.44:0/2914014879 conn(0x7fbf64077d40 msgr2=0x7fbf64078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.290+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 shutdown_connections 2024-04-20T11:22:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.290+0000 7fbf6b000640 1 -- 172.21.15.44:0/2914014879 wait complete. 2024-04-20T11:22:47.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T11:22:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 -- 172.21.15.44:0/1755913264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c5810e430 msgr2=0x7f9c5810e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/1755913264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c5810e430 0x7f9c5810e810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009930 tx=0x7f9c4802bc30 comp rx=0 tx=0).stop 2024-04-20T11:22:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 -- 172.21.15.44:0/1755913264 shutdown_connections 2024-04-20T11:22:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/1755913264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c58108e20 0x7f9c58115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/1755913264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c581084a0 0x7f9c581088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/1755913264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c5810e430 0x7f9c5810e810 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 -- 172.21.15.44:0/1755913264 >> 172.21.15.44:0/1755913264 conn(0x7f9c58077d40 msgr2=0x7f9c58106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 -- 172.21.15.44:0/1755913264 shutdown_connections 2024-04-20T11:22:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.534+0000 7f9c5ca00640 1 -- 172.21.15.44:0/1755913264 wait complete. 2024-04-20T11:22:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 Processor -- start 2024-04-20T11:22:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 -- start start 2024-04-20T11:22:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 0x7f9c5819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c58108e20 0x7f9c5819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c5810e430 0x7f9c581a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58118870 con 0x7f9c581084a0 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.535+0000 7f9c5ca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581186f0 con 0x7f9c5810e430 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c5ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581189f0 con 0x7f9c58108e20 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 0x7f9c5819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 0x7f9c5819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50466/0 (socket says 172.21.15.44:50466) 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 -- 172.21.15.44:0/3845723473 learned_addr learned my addr 172.21.15.44:0/3845723473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:22:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57e00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c5810e430 0x7f9c581a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c56a00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c58108e20 0x7f9c5819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 -- 172.21.15.44:0/3845723473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c58108e20 msgr2=0x7f9c5819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c58108e20 0x7f9c5819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 -- 172.21.15.44:0/3845723473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c5810e430 msgr2=0x7f9c581a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c5810e430 0x7f9c581a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.536+0000 7f9c57400640 1 -- 172.21.15.44:0/3845723473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c581a3d20 con 0x7f9c581084a0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.537+0000 7f9c57400640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 0x7f9c5819ed30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009230 tx=0x7f9c4802dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.537+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48043020 con 0x7f9c581084a0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.537+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c48004250 con 0x7f9c581084a0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.537+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48003970 con 0x7f9c581084a0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.537+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c581a3fb0 con 0x7f9c581084a0 2024-04-20T11:22:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.537+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581ab8a0 con 0x7f9c581084a0 2024-04-20T11:22:47.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.538+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c58050540 con 0x7f9c581084a0 2024-04-20T11:22:47.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.540+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f9c48046020 con 0x7f9c581084a0 2024-04-20T11:22:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.542+0000 7f9c4fe00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9c3c0a3e30 0x7f9c3c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:22:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.542+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9c480f2900 con 0x7f9c581084a0 2024-04-20T11:22:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.542+0000 7f9c56a00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9c3c0a3e30 0x7f9c3c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:22:47.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.543+0000 7f9c56a00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9c3c0a3e30 0x7f9c3c0a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c5810dbb0 tx=0x7f9c44005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:22:47.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.546+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9c4804d020 con 0x7f9c581084a0 2024-04-20T11:22:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:47 smithi044 ceph-mon[16855]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:47 smithi044 ceph-mon[16855]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.0 KiB/s wr, 20 op/s 2024-04-20T11:22:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:47 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:22:47.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:47 smithi165 ceph-mon[21975]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:47.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:47 smithi165 ceph-mon[21975]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.0 KiB/s wr, 20 op/s 2024-04-20T11:22:47.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:47 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:22:47.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.778+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9c580623a0 con 0x7f9c3c0a3e30 2024-04-20T11:22:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.793+0000 7f9c4fe00640 1 -- 172.21.15.44:0/3845723473 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9c580623a0 con 0x7f9c3c0a3e30 2024-04-20T11:22:47.794 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1m 6m3h 2024-04-20T11:22:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.795+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9c3c0a3e30 msgr2=0x7f9c3c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.795+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9c3c0a3e30 0x7f9c3c0a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c5810dbb0 tx=0x7f9c44005f90 comp rx=0 tx=0).stop 2024-04-20T11:22:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.795+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 msgr2=0x7f9c5819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:22:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.795+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 0x7f9c5819ed30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009230 tx=0x7f9c4802dc90 comp rx=0 tx=0).stop 2024-04-20T11:22:47.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 shutdown_connections 2024-04-20T11:22:47.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c5810e430 0x7f9c581a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c58108e20 0x7f9c5819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9c3c0a3e30 0x7f9c3c0a62f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 --2- 172.21.15.44:0/3845723473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c581084a0 0x7f9c5819ed30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:22:47.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 >> 172.21.15.44:0/3845723473 conn(0x7f9c58077d40 msgr2=0x7f9c58078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:22:47.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 shutdown_connections 2024-04-20T11:22:47.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:22:47.796+0000 7f9c5ca00640 1 -- 172.21.15.44:0/3845723473 wait complete. 2024-04-20T11:22:47.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-20T11:22:47.910 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-20T11:22:47.952 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T11:22:47.975 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:22:47.975 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T11:22:47.975 INFO:tasks.workunit:timeout=3h 2024-04-20T11:22:47.976 INFO:tasks.workunit:cleanup=True 2024-04-20T11:22:47.976 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:47.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:47 smithi078 ceph-mon[21464]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:47.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:47 smithi078 ceph-mon[21464]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.0 KiB/s wr, 20 op/s 2024-04-20T11:22:47.977 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:47 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 11:20:10.298801697 +0000 2024-04-20T11:22:48.038 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 11:20:10.298801697 +0000 2024-04-20T11:22:48.039 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 11:22:34.463886883 +0000 2024-04-20T11:22:48.039 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-20 11:20:10.298801697 +0000 2024-04-20T11:22:48.039 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T11:22:48.039 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T11:22:48.151 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T11:22:48.219 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T11:22:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:48 smithi165 ceph-mon[21975]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:48 smithi165 ceph-mon[21975]: from='client.25525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:48.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:48 smithi078 ceph-mon[21464]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:48.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:48 smithi078 ceph-mon[21464]: from='client.25525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:48 smithi044 ceph-mon[16855]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:48 smithi044 ceph-mon[16855]: from='client.25525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:49 smithi078 ceph-mon[21464]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:49.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:49 smithi078 ceph-mon[21464]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 53 op/s 2024-04-20T11:22:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:49 smithi165 ceph-mon[21975]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:49 smithi165 ceph-mon[21975]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 53 op/s 2024-04-20T11:22:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:49 smithi044 ceph-mon[16855]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:22:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:49 smithi044 ceph-mon[16855]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 53 op/s 2024-04-20T11:22:51.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:51 smithi078 ceph-mon[21464]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-20T11:22:51.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:51 smithi044 ceph-mon[16855]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-20T11:22:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:51 smithi165 ceph-mon[21975]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-20T11:22:53.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:53 smithi078 ceph-mon[21464]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-20T11:22:53.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:53 smithi044 ceph-mon[16855]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-20T11:22:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:53 smithi165 ceph-mon[21975]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-20T11:22:55.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:55 smithi078 ceph-mon[21464]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-20T11:22:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:55 smithi044 ceph-mon[16855]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-20T11:22:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:55 smithi165 ceph-mon[21975]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-20T11:22:57.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:57 smithi078 ceph-mon[21464]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-20T11:22:57.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:57 smithi044 ceph-mon[16855]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-20T11:22:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:57 smithi165 ceph-mon[21975]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-20T11:22:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:22:59 smithi078 ceph-mon[21464]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-20T11:22:59.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:22:59 smithi044 ceph-mon[16855]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-20T11:22:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:22:59 smithi165 ceph-mon[21975]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-20T11:23:01.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:23:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:23:00.816+0000 7f6177a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:23:01.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:23:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:23:00.816+0000 7f6177a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:23:01.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:23:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:23:00.816+0000 7f6177a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:23:01.195 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:23:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:23:00.816+0000 7f6177a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:23:01.196 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:23:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:23:00.816+0000 7f6177a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T11:23:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:01 smithi078 ceph-mon[21464]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-20T11:23:01.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:01 smithi044 ceph-mon[16855]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-20T11:23:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:01 smithi165 ceph-mon[21975]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-20T11:23:03.782 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:03 smithi165 ceph-mon[21975]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-20T11:23:03.782 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:03 smithi165 ceph-mon[21975]: mgrmap e26: x(active, since 26s), standbys: z, y 2024-04-20T11:23:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:03 smithi078 ceph-mon[21464]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-20T11:23:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:03 smithi078 ceph-mon[21464]: mgrmap e26: x(active, since 26s), standbys: z, y 2024-04-20T11:23:03.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:03 smithi044 ceph-mon[16855]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-20T11:23:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:03 smithi044 ceph-mon[16855]: mgrmap e26: x(active, since 26s), standbys: z, y 2024-04-20T11:23:05.780 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:05 smithi044 ceph-mon[16855]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-20T11:23:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:05 smithi078 ceph-mon[21464]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-20T11:23:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:05 smithi165 ceph-mon[21975]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-20T11:23:07.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:07 smithi044 ceph-mon[16855]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:23:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:07 smithi078 ceph-mon[21464]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:23:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:07 smithi165 ceph-mon[21975]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:23:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:09 smithi078 ceph-mon[21464]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-20T11:23:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:09 smithi044 ceph-mon[16855]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-20T11:23:09.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:09 smithi165 ceph-mon[21975]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-20T11:23:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:11 smithi078 ceph-mon[21464]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T11:23:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:11 smithi044 ceph-mon[16855]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T11:23:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:11 smithi165 ceph-mon[21975]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T11:23:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:13 smithi078 ceph-mon[21464]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T11:23:13.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:13 smithi044 ceph-mon[16855]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T11:23:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:13 smithi165 ceph-mon[21975]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T11:23:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:15 smithi078 ceph-mon[21464]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-20T11:23:15.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:15 smithi044 ceph-mon[16855]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-20T11:23:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:15 smithi165 ceph-mon[21975]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-20T11:23:16.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:23:16.112 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:23:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 -- 172.21.15.44:0/1109899501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c076db0 msgr2=0x7f0d8c112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 --2- 172.21.15.44:0/1109899501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c076db0 0x7f0d8c112760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009900 tx=0x7f0d7c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:23:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 -- 172.21.15.44:0/1109899501 shutdown_connections 2024-04-20T11:23:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 --2- 172.21.15.44:0/1109899501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 --2- 172.21.15.44:0/1109899501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c076db0 0x7f0d8c112760 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 --2- 172.21.15.44:0/1109899501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c076490 0x7f0d8c076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 -- 172.21.15.44:0/1109899501 >> 172.21.15.44:0/1109899501 conn(0x7f0d8c0ffa70 msgr2=0x7f0d8c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 -- 172.21.15.44:0/1109899501 shutdown_connections 2024-04-20T11:23:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.342+0000 7f0d91a00640 1 -- 172.21.15.44:0/1109899501 wait complete. 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 Processor -- start 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 -- start start 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c076490 0x7f0d8c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c076db0 0x7f0d8c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c1a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c118940 con 0x7f0d8c112ca0 2024-04-20T11:23:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1187c0 con 0x7f0d8c076db0 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.343+0000 7f0d91a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c118ac0 con 0x7f0d8c076490 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c1a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c1a3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47994/0 (socket says 172.21.15.44:47994) 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c076490 0x7f0d8c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 -- 172.21.15.44:0/3700397305 learned_addr learned my addr 172.21.15.44:0/3700397305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d8b400640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c076db0 0x7f0d8c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 -- 172.21.15.44:0/3700397305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c076490 msgr2=0x7f0d8c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c076490 0x7f0d8c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 -- 172.21.15.44:0/3700397305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c076db0 msgr2=0x7f0d8c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c076db0 0x7f0d8c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d91000640 1 -- 172.21.15.44:0/3700397305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c1a3e60 con 0x7f0d8c112ca0 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.344+0000 7f0d8be00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c076490 0x7f0d8c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.345+0000 7f0d91000640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c1a3730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400eb40 tx=0x7f0d8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.345+0000 7f0d88c00640 1 -- 172.21.15.44:0/3700397305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84004b20 con 0x7f0d8c112ca0 2024-04-20T11:23:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.345+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c1a4150 con 0x7f0d8c112ca0 2024-04-20T11:23:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.345+0000 7f0d88c00640 1 -- 172.21.15.44:0/3700397305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d84004cc0 con 0x7f0d8c112ca0 2024-04-20T11:23:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.345+0000 7f0d88c00640 1 -- 172.21.15.44:0/3700397305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840108f0 con 0x7f0d8c112ca0 2024-04-20T11:23:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.345+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c1ab9e0 con 0x7f0d8c112ca0 2024-04-20T11:23:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 Processor -- start 2024-04-20T11:23:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d88c00640 1 -- 172.21.15.44:0/3700397305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f0d84005020 con 0x7f0d8c112ca0 2024-04-20T11:23:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 -- start start 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2729760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729ca0 0x55a4b2732340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 0x55a4b2734c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a4b268f150 con 0x55a4b2729380 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a4b2635370 con 0x55a4b2729ca0 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.349+0000 7f0d95de7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a4b268b110 con 0x55a4b2732880 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2729760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 0x55a4b2734c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 0x55a4b2734c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51230/0 (socket says 172.21.15.44:51230) 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 -- 172.21.15.44:0/3761839593 learned_addr learned my addr 172.21.15.44:0/3761839593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8be00640 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729ca0 0x55a4b2732340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 -- 172.21.15.44:0/3761839593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729ca0 msgr2=0x55a4b2732340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729ca0 0x55a4b2732340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 -- 172.21.15.44:0/3761839593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 msgr2=0x55a4b2729760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2729760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 -- 172.21.15.44:0/3761839593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b246b7d0 con 0x55a4b2732880 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8be00640 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729ca0 0x55a4b2732340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:16.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.350+0000 7f0d8b400640 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 0x55a4b2734c70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009900 tx=0x7f0d7c005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16667c1ceb24585 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.351+0000 7f0d81600640 1 -- 172.21.15.44:0/3761839593 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c04a070 con 0x55a4b2732880 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.351+0000 7f0d81600640 1 -- 172.21.15.44:0/3761839593 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d7c02cbe0 con 0x55a4b2732880 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.351+0000 7f0d81600640 1 -- 172.21.15.44:0/3761839593 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0344e0 con 0x55a4b2732880 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.351+0000 7f0d88c00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d700a3f50 0x7f0d700a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.351+0000 7f0d88c00640 1 -- 172.21.15.44:0/3700397305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f0d840d10b0 con 0x7f0d8c112ca0 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d8be00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d700a3f50 0x7f0d700a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 -- 172.21.15.44:0/3761839593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 msgr2=0x55a4b2734c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 0x55a4b2734c70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009900 tx=0x7f0d7c005e40 comp rx=0 tx=0).stop 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 -- 172.21.15.44:0/3761839593 shutdown_connections 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2732880 0x55a4b2734c70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729ca0 0x55a4b2732340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 --2- 172.21.15.44:0/3761839593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2729760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 -- 172.21.15.44:0/3761839593 >> 172.21.15.44:0/3761839593 conn(0x55a4b2723ab0 msgr2=0x55a4b2723e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 -- 172.21.15.44:0/3761839593 shutdown_connections 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d95de7740 1 -- 172.21.15.44:0/3761839593 wait complete. 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.352+0000 7f0d8be00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d700a3f50 0x7f0d700a6410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009fd0 tx=0x7f0d74005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.353+0000 7f0d95de7740 1 -- start start 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.353+0000 7f0d95de7740 1 Processor -- start 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.353+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729380 0x55a4b272f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.353+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 0x55a4b272fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2730340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d95de7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a4b2735610 con 0x55a4b2729ca0 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d95de7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a4b2648740 con 0x55a4b2732880 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d95de7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a4b2735790 con 0x55a4b2729380 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d95de7740 10 client.0 _ll_register_callbacks cb 0x55a4b2643b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 0x55a4b272fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 0x55a4b272fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48014/0 (socket says 172.21.15.44:48014) 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 -- 172.21.15.44:0/900354181 learned_addr learned my addr 172.21.15.44:0/900354181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8be00640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729380 0x55a4b272f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d91000640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2730340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 -- 172.21.15.44:0/900354181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729380 msgr2=0x55a4b272f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729380 0x55a4b272f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 -- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 msgr2=0x55a4b2730340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2730340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.354+0000 7f0d8b400640 1 -- 172.21.15.44:0/900354181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b23a13f0 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d8b400640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 0x55a4b272fe00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c02c210 tx=0x7f0d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7336bd7050c51272 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d7a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d7a000640 1 -- 172.21.15.44:0/900354181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c04a070 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d7a000640 1 -- 172.21.15.44:0/900354181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f0d7c004450 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d7c02cbe0 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d7a000640 1 -- 172.21.15.44:0/900354181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d7c005040 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d95de7740 10 client.15429 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.355+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4b23610f0 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d7a000640 1 -- 172.21.15.44:0/900354181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d7c02cbe0 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d7a000640 1 -- 172.21.15.44:0/900354181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f0d7c056050 con 0x55a4b2729ca0 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d95de7740 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d95de7740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d95de7740 20 client.15429 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d95de7740 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b2742a80 0x55a4b2744e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d95de7740 4 client.15429 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:16.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.356+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4b269cb70 con 0x55a4b2742a80 2024-04-20T11:23:16.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.357+0000 7f0d8be00640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b2742a80 0x55a4b2744e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.357+0000 7f0d8be00640 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b2742a80 0x55a4b2744e70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35108b4c4e112358 server_cookie=a8cc3290c8e9499a in_seq=0 out_seq=0 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.357+0000 7f0d7a000640 10 client.15429 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d7a000640 1 -- 172.21.15.44:0/900354181 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4b269cb70 con 0x55a4b2742a80 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d7a000640 10 client.15429 handle_command_reply: tid=0 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 client.15429 shutdown 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 msgr2=0x55a4b272fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 0x55a4b272fe00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c02c210 tx=0x7f0d7c0057f0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 shutdown_connections 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b2742a80 0x55a4b2744e70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2730340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729ca0 0x55a4b272fe00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 --2- 172.21.15.44:0/900354181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729380 0x55a4b272f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d7a000640 0 client.15429 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.358+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 >> 172.21.15.44:0/900354181 conn(0x55a4b2723ab0 msgr2=0x55a4b2725800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.359+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 shutdown_connections 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.359+0000 7f0d95de7740 1 -- 172.21.15.44:0/900354181 wait complete. 2024-04-20T11:23:16.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.359+0000 7f0d95de7740 20 client.15429 trim_cache size 0 max 16384 2024-04-20T11:23:16.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.378+0000 7f0d95de7740 1 Processor -- start 2024-04-20T11:23:16.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.378+0000 7f0d95de7740 1 -- start start 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.378+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 0x55a4b2731ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b27255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2732880 0x55a4b2747e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d95de7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1b4030 con 0x55a4b2732880 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d95de7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1b5b70 con 0x55a4b2729380 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d95de7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1b4700 con 0x55a4b2729ca0 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 0x55a4b2731ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 0x55a4b2731ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43644/0 (socket says 172.21.15.44:43644) 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 -- 172.21.15.44:0/1053380106 learned_addr learned my addr 172.21.15.44:0/1053380106 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d91000640 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2732880 0x55a4b2747e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8b400640 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b27255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 -- 172.21.15.44:0/1053380106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 msgr2=0x55a4b27255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b27255f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 -- 172.21.15.44:0/1053380106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2732880 msgr2=0x55a4b2747e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2732880 0x55a4b2747e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.379+0000 7f0d8be00640 1 -- 172.21.15.44:0/1053380106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b272de30 con 0x55a4b2729380 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.380+0000 7f0d91000640 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2732880 0x55a4b2747e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.380+0000 7f0d8be00640 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 0x55a4b2731ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55a4b27357d0 tx=0x7f0d74017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94fc497a2a890fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.380+0000 7f0d80c00640 1 -- 172.21.15.44:0/1053380106 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74003ac0 con 0x55a4b2729380 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.380+0000 7f0d80c00640 1 -- 172.21.15.44:0/1053380106 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d7402c920 con 0x55a4b2729380 2024-04-20T11:23:16.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.380+0000 7f0d80c00640 1 -- 172.21.15.44:0/1053380106 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740054e0 con 0x55a4b2729380 2024-04-20T11:23:16.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 -- 172.21.15.44:0/1053380106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 msgr2=0x55a4b2731ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 0x55a4b2731ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55a4b27357d0 tx=0x7f0d74017000 comp rx=0 tx=0).stop 2024-04-20T11:23:16.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 -- 172.21.15.44:0/1053380106 shutdown_connections 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2732880 0x55a4b2747e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b27255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1053380106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2729380 0x55a4b2731ff0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 -- 172.21.15.44:0/1053380106 >> 172.21.15.44:0/1053380106 conn(0x55a4b2723ab0 msgr2=0x55a4b2724ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 -- 172.21.15.44:0/1053380106 shutdown_connections 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 -- 172.21.15.44:0/1053380106 wait complete. 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.381+0000 7f0d95de7740 1 -- start start 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 Processor -- start 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2748420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b2745800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2745f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1b54a0 con 0x55a4b2729380 2024-04-20T11:23:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1b4dd0 con 0x55a4b2732880 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1b2060 con 0x55a4b2729ca0 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d95de7740 10 client.0 _ll_register_callbacks cb 0x7f0d7c048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2748420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2748420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48032/0 (socket says 172.21.15.44:48032) 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b2745800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.382+0000 7f0d8be00640 1 -- 172.21.15.44:0/1213541003 learned_addr learned my addr 172.21.15.44:0/1213541003 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d91000640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2745f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8be00640 1 -- 172.21.15.44:0/1213541003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 msgr2=0x55a4b2745800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8be00640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b2745800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8be00640 1 -- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 msgr2=0x55a4b2745f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8be00640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2745f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8be00640 1 -- 172.21.15.44:0/1213541003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b27a7c60 con 0x55a4b2729380 2024-04-20T11:23:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d91000640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2745f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8b400640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b2745800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d8be00640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2748420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d740040a0 tx=0x7f0d74012360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4dec5f4e7cf9db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.383+0000 7f0d7a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d7a000640 1 -- 172.21.15.44:0/1213541003 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7403a020 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d7a000640 1 -- 172.21.15.44:0/1213541003 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f0d74032040 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a4b26969e0 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d7a000640 1 -- 172.21.15.44:0/1213541003 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d74045050 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 10 client.15435 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4b27a8e30 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d7a000640 1 -- 172.21.15.44:0/1213541003 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55a4b26969e0 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d7a000640 1 -- 172.21.15.44:0/1213541003 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f0d7400f890 con 0x55a4b2729380 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 10 client.15435 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 20 client.15435 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.384+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b27a9570 0x55a4b27ab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.385+0000 7f0d95de7740 4 client.15435 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.385+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4b2612450 con 0x55a4b27a9570 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.385+0000 7f0d8b400640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b27a9570 0x55a4b27ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.385+0000 7f0d8b400640 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b27a9570 0x55a4b27ab990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e789aea463163b server_cookie=5ec583aef77c0565 in_seq=0 out_seq=0 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.385+0000 7f0d7a000640 10 client.15435 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d7a000640 1 -- 172.21.15.44:0/1213541003 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a4b2612450 con 0x55a4b27a9570 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d7a000640 10 client.15435 handle_command_reply: tid=0 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 client.15435 shutdown 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 msgr2=0x55a4b2748420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2748420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d740040a0 tx=0x7f0d74012360 comp rx=0 tx=0).stop 2024-04-20T11:23:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 shutdown_connections 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a4b27a9570 0x55a4b27ab990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a4b2732880 0x55a4b2745f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4b2729ca0 0x55a4b2745800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 --2- 172.21.15.44:0/1213541003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a4b2729380 0x55a4b2748420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d7a000640 0 client.15435 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 >> 172.21.15.44:0/1213541003 conn(0x55a4b2723ab0 msgr2=0x55a4b2722b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 shutdown_connections 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 1 -- 172.21.15.44:0/1213541003 wait complete. 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.386+0000 7f0d95de7740 20 client.15435 trim_cache size 0 max 16384 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:23:16.391 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d700a3f50 msgr2=0x7f0d700a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d700a3f50 0x7f0d700a6410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009fd0 tx=0x7f0d74005f90 comp rx=0 tx=0).stop 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 msgr2=0x7f0d8c1a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c1a3730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400eb40 tx=0x7f0d8400c2d0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 shutdown_connections 2024-04-20T11:23:16.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d700a3f50 0x7f0d700a6410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d8c112ca0 0x7f0d8c1a3730 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c076db0 0x7f0d8c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 --2- 172.21.15.44:0/3700397305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d8c076490 0x7f0d8c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 >> 172.21.15.44:0/3700397305 conn(0x7f0d8c0ffa70 msgr2=0x7f0d8c1142f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 shutdown_connections 2024-04-20T11:23:16.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.387+0000 7f0d91a00640 1 -- 172.21.15.44:0/3700397305 wait complete. 2024-04-20T11:23:16.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:23:16.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 -- 172.21.15.44:0/3712187459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a8104b70 msgr2=0x7f62a8108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 --2- 172.21.15.44:0/3712187459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a8104b70 0x7f62a8108bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6294009900 tx=0x7f629402bc30 comp rx=0 tx=0).stop 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 -- 172.21.15.44:0/3712187459 shutdown_connections 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 --2- 172.21.15.44:0/3712187459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8109290 0x7f62a8110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 --2- 172.21.15.44:0/3712187459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a8104b70 0x7f62a8108bb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 --2- 172.21.15.44:0/3712187459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a81041c0 0x7f62a81045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 -- 172.21.15.44:0/3712187459 >> 172.21.15.44:0/3712187459 conn(0x7f62a80ffa00 msgr2=0x7f62a8101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 -- 172.21.15.44:0/3712187459 shutdown_connections 2024-04-20T11:23:16.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.617+0000 7f62aea00640 1 -- 172.21.15.44:0/3712187459 wait complete. 2024-04-20T11:23:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.618+0000 7f62aea00640 1 Processor -- start 2024-04-20T11:23:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.618+0000 7f62aea00640 1 -- start start 2024-04-20T11:23:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62aea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 0x7f62a81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62aea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8109290 0x7f62a81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62aea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a819a7f0 0x7f62a819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62aea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81143f0 con 0x7f62a81041c0 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62aea00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8114270 con 0x7f62a8109290 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62aea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8114570 con 0x7f62a819a7f0 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62ad600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 0x7f62a81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62ad600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 0x7f62a81a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48046/0 (socket says 172.21.15.44:48046) 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62ad600640 1 -- 172.21.15.44:0/1491208682 learned_addr learned my addr 172.21.15.44:0/1491208682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62acc00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8109290 0x7f62a81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.619+0000 7f62ae000640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a819a7f0 0x7f62a819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62ad600640 1 -- 172.21.15.44:0/1491208682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a819a7f0 msgr2=0x7f62a819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62ad600640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a819a7f0 0x7f62a819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62ad600640 1 -- 172.21.15.44:0/1491208682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8109290 msgr2=0x7f62a81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62ad600640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8109290 0x7f62a81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62ad600640 1 -- 172.21.15.44:0/1491208682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a819b380 con 0x7f62a81041c0 2024-04-20T11:23:16.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62ad600640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 0x7f62a81a0680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f629c00e9f0 tx=0x7f629c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c00cd80 con 0x7f62a81041c0 2024-04-20T11:23:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f629c010930 con 0x7f62a81041c0 2024-04-20T11:23:16.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a819b670 con 0x7f62a81041c0 2024-04-20T11:23:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c018b00 con 0x7f62a81041c0 2024-04-20T11:23:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.620+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a7870 con 0x7f62a81041c0 2024-04-20T11:23:16.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.621+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a8105690 con 0x7f62a81041c0 2024-04-20T11:23:16.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.624+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f629c018ca0 con 0x7f62a81041c0 2024-04-20T11:23:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.625+0000 7f62a6000640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62900a3f80 0x7f62900a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.625+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f629c0d0e50 con 0x7f62a81041c0 2024-04-20T11:23:16.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.625+0000 7f62acc00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62900a3f80 0x7f62900a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:16.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.626+0000 7f62acc00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62900a3f80 0x7f62900a6440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f629402c240 tx=0x7f6294005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:16.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.630+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f62a819b670 con 0x7f62a81041c0 2024-04-20T11:23:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.879+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a810c550 con 0x7f62a81041c0 2024-04-20T11:23:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.881+0000 7f62a6000640 1 -- 172.21.15.44:0/1491208682 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f629c09d320 con 0x7f62a81041c0 2024-04-20T11:23:16.881 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:23:16.881 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:22:35.546711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:23:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-04-20T11:23:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.883+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62900a3f80 msgr2=0x7f62900a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.883+0000 7f62aea00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62900a3f80 0x7f62900a6440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f629402c240 tx=0x7f6294005fd0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 msgr2=0x7f62a81a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 0x7f62a81a0680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f629c00e9f0 tx=0x7f629c00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 shutdown_connections 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a819a7f0 0x7f62a819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8109290 0x7f62a81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62900a3f80 0x7f62900a6440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 --2- 172.21.15.44:0/1491208682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a81041c0 0x7f62a81a0680 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 >> 172.21.15.44:0/1491208682 conn(0x7f62a80ffa00 msgr2=0x7f62a8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 shutdown_connections 2024-04-20T11:23:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:16.884+0000 7f62aea00640 1 -- 172.21.15.44:0/1491208682 wait complete. 2024-04-20T11:23:16.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:23:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.113+0000 7f109f600640 1 -- 172.21.15.44:0/4012371518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098108590 msgr2=0x7f1098108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.113+0000 7f109f600640 1 --2- 172.21.15.44:0/4012371518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098108590 0x7f1098108a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1084009930 tx=0x7f108402bc20 comp rx=0 tx=0).stop 2024-04-20T11:23:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.113+0000 7f109f600640 1 -- 172.21.15.44:0/4012371518 shutdown_connections 2024-04-20T11:23:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.114+0000 7f109f600640 1 --2- 172.21.15.44:0/4012371518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108f50 0x7f10981153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.114+0000 7f109f600640 1 --2- 172.21.15.44:0/4012371518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098108590 0x7f1098108a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.114+0000 7f109f600640 1 --2- 172.21.15.44:0/4012371518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f109810e600 0x7f109810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.114+0000 7f109f600640 1 -- 172.21.15.44:0/4012371518 >> 172.21.15.44:0/4012371518 conn(0x7f10980764a0 msgr2=0x7f10980788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.114+0000 7f109f600640 1 -- 172.21.15.44:0/4012371518 shutdown_connections 2024-04-20T11:23:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.114+0000 7f109f600640 1 -- 172.21.15.44:0/4012371518 wait complete. 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 Processor -- start 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 -- start start 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 0x7f109819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1098108f50 0x7f109819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109810e600 0x7f10981a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1098118a20 con 0x7f1098108f50 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10981188a0 con 0x7f109810e600 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.115+0000 7f109f600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1098118ba0 con 0x7f1098108590 2024-04-20T11:23:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 0x7f109819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 0x7f109819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51292/0 (socket says 172.21.15.44:51292) 2024-04-20T11:23:17.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 -- 172.21.15.44:0/3981421606 learned_addr learned my addr 172.21.15.44:0/3981421606 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109d600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109810e600 0x7f10981a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f1097e00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1098108f50 0x7f109819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 -- 172.21.15.44:0/3981421606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109810e600 msgr2=0x7f10981a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109810e600 0x7f10981a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 -- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1098108f50 msgr2=0x7f109819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1098108f50 0x7f109819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 -- 172.21.15.44:0/3981421606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981a3ef0 con 0x7f1098108590 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f1097e00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1098108f50 0x7f109819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.116+0000 7f109cc00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 0x7f109819ef00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e9f0 tx=0x7f108c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.117+0000 7f1095600640 1 -- 172.21.15.44:0/3981421606 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c00cd80 con 0x7f1098108590 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.117+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981a41e0 con 0x7f1098108590 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.117+0000 7f1095600640 1 -- 172.21.15.44:0/3981421606 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f108c010930 con 0x7f1098108590 2024-04-20T11:23:17.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.117+0000 7f1095600640 1 -- 172.21.15.44:0/3981421606 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c018b00 con 0x7f1098108590 2024-04-20T11:23:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.117+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981aba70 con 0x7f1098108590 2024-04-20T11:23:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.120+0000 7f10a224e740 1 Processor -- start 2024-04-20T11:23:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.120+0000 7f10a224e740 1 -- start start 2024-04-20T11:23:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.120+0000 7f1095600640 1 -- 172.21.15.44:0/3981421606 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f108c018ca0 con 0x7f1098108590 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.120+0000 7f10a224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 0x55de304ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.120+0000 7f10a224e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de304f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f10a224e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304f64b0 0x55de304f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f10a224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55de30452050 con 0x55de304ecfb0 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f10a224e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de303f8490 con 0x55de304f64b0 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f10a224e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55de3044e010 con 0x55de304ed8d0 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 0x55de304ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de304f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 0x55de304ed390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48072/0 (socket says 172.21.15.44:48072) 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 -- 172.21.15.44:0/829918243 learned_addr learned my addr 172.21.15.44:0/829918243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f1097e00640 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304f64b0 0x55de304f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 -- 172.21.15.44:0/829918243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 msgr2=0x55de304f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de304f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 -- 172.21.15.44:0/829918243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304f64b0 msgr2=0x55de304f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304f64b0 0x55de304f88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109d600640 1 -- 172.21.15.44:0/829918243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de30400070 con 0x55de304ecfb0 2024-04-20T11:23:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.121+0000 7f109cc00640 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de304f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f109d600640 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 0x55de304ed390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1090009fd0 tx=0x7f10900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b1c91ad34d4cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f1089c00640 1 -- 172.21.15.44:0/829918243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090004d20 con 0x55de304ecfb0 2024-04-20T11:23:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f1095600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f10800a3ed0 0x7f10800a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f1089c00640 1 -- 172.21.15.44:0/829918243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1090004050 con 0x55de304ecfb0 2024-04-20T11:23:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f1089c00640 1 -- 172.21.15.44:0/829918243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090005280 con 0x55de304ecfb0 2024-04-20T11:23:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f1095600640 1 -- 172.21.15.44:0/3981421606 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f108c0d0e30 con 0x7f1098108590 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.122+0000 7f109cc00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f10800a3ed0 0x7f10800a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 -- 172.21.15.44:0/829918243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 msgr2=0x55de304ed390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 0x55de304ed390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1090009fd0 tx=0x7f10900118d0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 -- 172.21.15.44:0/829918243 shutdown_connections 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304f64b0 0x55de304f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de304f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 --2- 172.21.15.44:0/829918243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304ecfb0 0x55de304ed390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 -- 172.21.15.44:0/829918243 >> 172.21.15.44:0/829918243 conn(0x55de304e76e0 msgr2=0x55de304e7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 -- 172.21.15.44:0/829918243 shutdown_connections 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f10a224e740 1 -- 172.21.15.44:0/829918243 wait complete. 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.123+0000 7f109cc00640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f10800a3ed0 0x7f10800a6390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f108c0ce410 tx=0x7f108c0d4900 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 -- start start 2024-04-20T11:23:17.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 Processor -- start 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304f3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55de304f92a0 con 0x55de304f64b0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de3040b860 con 0x55de304ed8d0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55de304f9420 con 0x55de304ecfb0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.124+0000 7f10a224e740 10 client.0 _ll_register_callbacks cb 0x55de303d9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f109d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f1097e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f109d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51312/0 (socket says 172.21.15.44:51312) 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f109d600640 1 -- 172.21.15.44:0/3954586120 learned_addr learned my addr 172.21.15.44:0/3954586120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f109d600640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f1097e00640 1 -- 172.21.15.44:0/3954586120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 msgr2=0x55de304f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f1097e00640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f1097e00640 1 -- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 msgr2=0x55de304f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f1097e00640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f1097e00640 1 -- 172.21.15.44:0/3954586120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de3040f950 con 0x55de304ed8d0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.125+0000 7f109d600640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f1097e00640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304f3ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1084005ab0 tx=0x7f108402cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207765a67b887bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f107e000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f107e000640 1 -- 172.21.15.44:0/3954586120 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084043070 con 0x55de304ed8d0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1090004050 con 0x55de304ed8d0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f107e000640 1 -- 172.21.15.44:0/3954586120 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1084004250 con 0x55de304ed8d0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f107e000640 1 -- 172.21.15.44:0/3954586120 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f108404f050 con 0x55de304ed8d0 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f10a224e740 10 client.25571 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.126+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de303dd820 con 0x55de304ed8d0 2024-04-20T11:23:17.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f107e000640 1 -- 172.21.15.44:0/3954586120 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1090004050 con 0x55de304ed8d0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f107e000640 1 -- 172.21.15.44:0/3954586120 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f1084004a60 con 0x55de304ed8d0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f10a224e740 10 client.25571 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f10a224e740 10 client.25571 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f10a224e740 20 client.25571 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f10a224e740 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de305064d0 0x55de305088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f10a224e740 4 client.25571 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.127+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de3045fa70 con 0x55de305064d0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.128+0000 7f109d600640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de305064d0 0x55de305088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.128+0000 7f109d600640 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de305064d0 0x55de305088e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5eafa18d3e9de05 server_cookie=8db52bc82bc85cb6 in_seq=0 out_seq=0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.128+0000 7f107e000640 10 client.25571 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f107e000640 1 -- 172.21.15.44:0/3954586120 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de3045fa70 con 0x55de305064d0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f107e000640 10 client.25571 handle_command_reply: tid=0 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 client.25571 shutdown 2024-04-20T11:23:17.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 msgr2=0x55de304f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304f3ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1084005ab0 tx=0x7f108402cbf0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 shutdown_connections 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de305064d0 0x55de305088e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304f3ab0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 --2- 172.21.15.44:0/3954586120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f107e000640 0 client.25571 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 >> 172.21.15.44:0/3954586120 conn(0x55de304e76e0 msgr2=0x55de304e9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.129+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 shutdown_connections 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.130+0000 7f10a224e740 1 -- 172.21.15.44:0/3954586120 wait complete. 2024-04-20T11:23:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.130+0000 7f10a224e740 20 client.25571 trim_cache size 0 max 16384 2024-04-20T11:23:17.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 Processor -- start 2024-04-20T11:23:17.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 -- start start 2024-04-20T11:23:17.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ecfb0 0x55de304f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de30502160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de3050e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b4240 con 0x55de304f64b0 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b5d80 con 0x55de304ecfb0 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.145+0000 7f10a224e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b4910 con 0x55de304ed8d0 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de30502160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de30502160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51322/0 (socket says 172.21.15.44:51322) 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f109d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de3050e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 -- 172.21.15.44:0/35490391 learned_addr learned my addr 172.21.15.44:0/35490391 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f109d600640 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ecfb0 0x55de304f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 -- 172.21.15.44:0/35490391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ecfb0 msgr2=0x55de304f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ecfb0 0x55de304f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 -- 172.21.15.44:0/35490391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 msgr2=0x55de3050e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de3050e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 -- 172.21.15.44:0/35490391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de301643f0 con 0x55de304ed8d0 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.146+0000 7f1097e00640 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de30502160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1084003610 tx=0x7f1084005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2e45661ce4e94f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.147+0000 7f1089200640 1 -- 172.21.15.44:0/35490391 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084043070 con 0x55de304ed8d0 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.147+0000 7f1089200640 1 -- 172.21.15.44:0/35490391 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1084003690 con 0x55de304ed8d0 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.147+0000 7f1089200640 1 -- 172.21.15.44:0/35490391 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084004c30 con 0x55de304ed8d0 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.147+0000 7f10a224e740 1 -- 172.21.15.44:0/35490391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 msgr2=0x55de30502160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.147+0000 7f10a224e740 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de30502160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1084003610 tx=0x7f1084005fd0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 -- 172.21.15.44:0/35490391 shutdown_connections 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de3050e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ed8d0 0x55de30502160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 --2- 172.21.15.44:0/35490391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ecfb0 0x55de304f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 -- 172.21.15.44:0/35490391 >> 172.21.15.44:0/35490391 conn(0x55de304e76e0 msgr2=0x55de304ee8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 -- 172.21.15.44:0/35490391 shutdown_connections 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 -- 172.21.15.44:0/35490391 wait complete. 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 -- start start 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.148+0000 7f10a224e740 1 Processor -- start 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304fe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304febb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b56b0 con 0x55de304f64b0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b4fe0 con 0x55de304ed8d0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10981b2210 con 0x55de304ecfb0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f10a224e740 10 client.0 _ll_register_callbacks cb 0x7f1084005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f109d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f109d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f1097e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f109d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304fe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51338/0 (socket says 172.21.15.44:51338) 2024-04-20T11:23:17.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.149+0000 7f109d600640 1 -- 172.21.15.44:0/3225978368 learned_addr learned my addr 172.21.15.44:0/3225978368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f1097e00640 1 -- 172.21.15.44:0/3225978368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 msgr2=0x55de304fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f1097e00640 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f1097e00640 1 -- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 msgr2=0x55de304efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f1097e00640 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f1097e00640 1 -- 172.21.15.44:0/3225978368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de3022e7d0 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f1097e00640 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304febb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55de304e3040 tx=0x7f108402bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961b533ddf812f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f107e000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f107e000640 1 -- 172.21.15.44:0/3225978368 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084043040 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1084003690 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.150+0000 7f107e000640 1 -- 172.21.15.44:0/3225978368 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1084003960 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f107e000640 1 -- 172.21.15.44:0/3225978368 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1084003ba0 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 10 client.25577 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de3056c300 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f107e000640 1 -- 172.21.15.44:0/3225978368 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1084003690 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f107e000640 1 -- 172.21.15.44:0/3225978368 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f1084063020 con 0x55de304ed8d0 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 10 client.25577 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 10 client.25577 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:23:17.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 20 client.25577 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de3056c970 0x55de3056ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 4 client.25577 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.151+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de304598e0 con 0x55de3056c970 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.152+0000 7f109d600640 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de3056c970 0x55de3056ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.152+0000 7f109d600640 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de3056c970 0x55de3056ed90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=707d40d81c650211 server_cookie=a18a92b41b2336ec in_seq=0 out_seq=0 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.152+0000 7f107e000640 10 client.25577 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f107e000640 1 -- 172.21.15.44:0/3225978368 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de304598e0 con 0x55de3056c970 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f107e000640 10 client.25577 handle_command_reply: tid=0 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 client.25577 shutdown 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 msgr2=0x55de304febb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304febb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55de304e3040 tx=0x7f108402bc20 comp rx=0 tx=0).stop 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 shutdown_connections 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de304f64b0 0x55de304efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55de3056c970 0x55de3056ed90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de304ed8d0 0x55de304febb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 --2- 172.21.15.44:0/3225978368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de304ecfb0 0x55de304fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f107e000640 0 client.25577 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 >> 172.21.15.44:0/3225978368 conn(0x55de304e76e0 msgr2=0x55de304e8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 shutdown_connections 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 1 -- 172.21.15.44:0/3225978368 wait complete. 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.153+0000 7f10a224e740 20 client.25577 trim_cache size 0 max 16384 2024-04-20T11:23:17.156 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f10800a3ed0 msgr2=0x7f10800a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f10800a3ed0 0x7f10800a6390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f108c0ce410 tx=0x7f108c0d4900 comp rx=0 tx=0).stop 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 msgr2=0x7f109819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 0x7f109819ef00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e9f0 tx=0x7f108c00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 shutdown_connections 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109810e600 0x7f10981a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f10800a3ed0 0x7f10800a6390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1098108f50 0x7f109819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 --2- 172.21.15.44:0/3981421606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1098108590 0x7f109819ef00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 >> 172.21.15.44:0/3981421606 conn(0x7f10980764a0 msgr2=0x7f1098077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 shutdown_connections 2024-04-20T11:23:17.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.154+0000 7f109f600640 1 -- 172.21.15.44:0/3981421606 wait complete. 2024-04-20T11:23:17.172 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:23:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.372+0000 7f53a6200640 1 -- 172.21.15.44:0/1014881318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0109320 msgr2=0x7f53a0110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.372+0000 7f53a6200640 1 --2- 172.21.15.44:0/1014881318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0109320 0x7f53a0110ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5398009ca0 tx=0x7f539802c100 comp rx=0 tx=0).stop 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 -- 172.21.15.44:0/1014881318 shutdown_connections 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 --2- 172.21.15.44:0/1014881318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0109320 0x7f53a0110ef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 --2- 172.21.15.44:0/1014881318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53a0104c00 0x7f53a0108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 --2- 172.21.15.44:0/1014881318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0104250 0x7f53a0104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 -- 172.21.15.44:0/1014881318 >> 172.21.15.44:0/1014881318 conn(0x7f53a00ffa70 msgr2=0x7f53a0101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 -- 172.21.15.44:0/1014881318 shutdown_connections 2024-04-20T11:23:17.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.373+0000 7f53a6200640 1 -- 172.21.15.44:0/1014881318 wait complete. 2024-04-20T11:23:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 Processor -- start 2024-04-20T11:23:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 -- start start 2024-04-20T11:23:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 0x7f53a01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0109320 0x7f53a01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53a019a770 0x7f53a019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01144f0 con 0x7f53a0104250 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0114370 con 0x7f53a0109320 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a6200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0114670 con 0x7f53a019a770 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.374+0000 7f53a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 0x7f53a01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 0x7f53a01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48118/0 (socket says 172.21.15.44:48118) 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 -- 172.21.15.44:0/3122958317 learned_addr learned my addr 172.21.15.44:0/3122958317 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f539fe00640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0109320 0x7f53a01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a5800640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53a019a770 0x7f53a019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 -- 172.21.15.44:0/3122958317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53a019a770 msgr2=0x7f53a019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53a019a770 0x7f53a019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 -- 172.21.15.44:0/3122958317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0109320 msgr2=0x7f53a01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0109320 0x7f53a01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 -- 172.21.15.44:0/3122958317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a019b370 con 0x7f53a0104250 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.375+0000 7f53a4e00640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 0x7f53a01a06b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f539400ea60 tx=0x7f539400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.376+0000 7f539d600640 1 -- 172.21.15.44:0/3122958317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400cde0 con 0x7f53a0104250 2024-04-20T11:23:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.376+0000 7f539d600640 1 -- 172.21.15.44:0/3122958317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53940035f0 con 0x7f53a0104250 2024-04-20T11:23:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.376+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a01a7450 con 0x7f53a0104250 2024-04-20T11:23:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.376+0000 7f539d600640 1 -- 172.21.15.44:0/3122958317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394010720 con 0x7f53a0104250 2024-04-20T11:23:17.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.376+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a01a7850 con 0x7f53a0104250 2024-04-20T11:23:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.379+0000 7f53aa122740 1 Processor -- start 2024-04-20T11:23:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.379+0000 7f539d600640 1 -- 172.21.15.44:0/3122958317 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f53940037a0 con 0x7f53a0104250 2024-04-20T11:23:17.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.379+0000 7f53aa122740 1 -- start start 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53aa122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc8e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53aa122740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8ea90 0x562e5cc97130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53aa122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 0x562e5cc99a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53aa122740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e5cbf3050 con 0x562e5cc8ea90 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53aa122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e5cb99490 con 0x562e5cc97670 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53aa122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562e5cbef010 con 0x562e5cc8e170 2024-04-20T11:23:17.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8ea90 0x562e5cc97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f539fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 0x562e5cc99a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f53a5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc8e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f539fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 0x562e5cc99a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43766/0 (socket says 172.21.15.44:43766) 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.380+0000 7f539fe00640 1 -- 172.21.15.44:0/3148899336 learned_addr learned my addr 172.21.15.44:0/3148899336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f539fe00640 1 -- 172.21.15.44:0/3148899336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 msgr2=0x562e5cc8e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f539fe00640 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc8e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f539fe00640 1 -- 172.21.15.44:0/3148899336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8ea90 msgr2=0x562e5cc97130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f539fe00640 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8ea90 0x562e5cc97130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f539fe00640 1 -- 172.21.15.44:0/3148899336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5cba1070 con 0x562e5cc97670 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f539fe00640 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 0x562e5cc99a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f538c009c20 tx=0x7f538c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def0fefdc4292498 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f5392000640 1 -- 172.21.15.44:0/3148899336 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c004050 con 0x562e5cc97670 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f5392000640 1 -- 172.21.15.44:0/3148899336 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f538c005520 con 0x562e5cc97670 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.381+0000 7f5392000640 1 -- 172.21.15.44:0/3148899336 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0035f0 con 0x562e5cc97670 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.382+0000 7f539d600640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f53880a3ed0 0x7f53880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.382+0000 7f539d600640 1 -- 172.21.15.44:0/3122958317 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f53940d0db0 con 0x7f53a0104250 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.382+0000 7f53a5800640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f53880a3ed0 0x7f53880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.382+0000 7f53aa122740 1 -- 172.21.15.44:0/3148899336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 msgr2=0x562e5cc99a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.382+0000 7f53aa122740 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 0x562e5cc99a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f538c009c20 tx=0x7f538c011810 comp rx=0 tx=0).stop 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 -- 172.21.15.44:0/3148899336 shutdown_connections 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc97670 0x562e5cc99a60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8ea90 0x562e5cc97130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 --2- 172.21.15.44:0/3148899336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc8e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 -- 172.21.15.44:0/3148899336 >> 172.21.15.44:0/3148899336 conn(0x562e5cc888a0 msgr2=0x562e5cc88c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 -- 172.21.15.44:0/3148899336 shutdown_connections 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53a5800640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f53880a3ed0 0x7f53880a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5398005e30 tx=0x7f5398005d80 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 -- 172.21.15.44:0/3148899336 wait complete. 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 -- start start 2024-04-20T11:23:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.383+0000 7f53aa122740 1 Processor -- start 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e5cc9a460 con 0x562e5cc97670 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e5cbac860 con 0x562e5cc8ea90 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562e5cc9a5e0 con 0x562e5cc8e170 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53aa122740 10 client.0 _ll_register_callbacks cb 0x562e5cb7afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53a5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53a5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48140/0 (socket says 172.21.15.44:48140) 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53a5800640 1 -- 172.21.15.44:0/2252214142 learned_addr learned my addr 172.21.15.44:0/2252214142 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f53a4e00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.384+0000 7f539fe00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a4e00640 1 -- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 msgr2=0x562e5cc94b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a4e00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc94b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a4e00640 1 -- 172.21.15.44:0/2252214142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 msgr2=0x562e5cc950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a4e00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a4e00640 1 -- 172.21.15.44:0/2252214142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5cbb0950 con 0x562e5cc8e170 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a5800640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53a4e00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc94630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53940ce330 tx=0x7f5394003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24000cadee811ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f5386a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f5386a00640 1 -- 172.21.15.44:0/2252214142 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539409c950 con 0x562e5cc8e170 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f5386a00640 1 -- 172.21.15.44:0/2252214142 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f539409bfd0 con 0x562e5cc8e170 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.385+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538c005520 con 0x562e5cc8e170 2024-04-20T11:23:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f5386a00640 1 -- 172.21.15.44:0/2252214142 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f539409c4e0 con 0x562e5cc8e170 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f53aa122740 10 client.25591 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e5cb7e820 con 0x562e5cc8e170 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f5386a00640 1 -- 172.21.15.44:0/2252214142 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f538c005520 con 0x562e5cc8e170 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f5386a00640 1 -- 172.21.15.44:0/2252214142 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f5394004000 con 0x562e5cc8e170 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f53aa122740 10 client.25591 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f53aa122740 10 client.25591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f53aa122740 20 client.25591 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.386+0000 7f53aa122740 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cca75a0 0x562e5cca99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.387+0000 7f53aa122740 4 client.25591 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.387+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e5cc00a70 con 0x562e5cca75a0 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.387+0000 7f539fe00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cca75a0 0x562e5cca99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.387+0000 7f5386a00640 10 client.25591 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.387+0000 7f539fe00640 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cca75a0 0x562e5cca99f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a31bf7db75edac73 server_cookie=f766bad2154f469c in_seq=0 out_seq=0 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.388+0000 7f5386a00640 1 -- 172.21.15.44:0/2252214142 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e5cc00a70 con 0x562e5cca75a0 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.388+0000 7f5386a00640 10 client.25591 handle_command_reply: tid=0 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.388+0000 7f53aa122740 1 client.25591 shutdown 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 msgr2=0x562e5cc94630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc94630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53940ce330 tx=0x7f5394003f30 comp rx=0 tx=0).stop 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 shutdown_connections 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cca75a0 0x562e5cca99f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 --2- 172.21.15.44:0/2252214142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8e170 0x562e5cc94630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 >> 172.21.15.44:0/2252214142 conn(0x562e5cc888a0 msgr2=0x562e5cc8a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f5386a00640 0 client.25591 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 shutdown_connections 2024-04-20T11:23:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 1 -- 172.21.15.44:0/2252214142 wait complete. 2024-04-20T11:23:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.389+0000 7f53aa122740 20 client.25591 trim_cache size 0 max 16384 2024-04-20T11:23:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 Processor -- start 2024-04-20T11:23:17.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 -- start start 2024-04-20T11:23:17.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8e170 0x562e5cca3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8ea90 0x562e5cc87960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01afba0 con 0x562e5cc97670 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01b16e0 con 0x562e5cc8e170 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53aa122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01b0270 con 0x562e5cc8ea90 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53a5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53a5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc8a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48144/0 (socket says 172.21.15.44:48144) 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53a5800640 1 -- 172.21.15.44:0/4095057344 learned_addr learned my addr 172.21.15.44:0/4095057344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.404+0000 7f53a4e00640 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8e170 0x562e5cca3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f539fe00640 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8ea90 0x562e5cc87960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f53a5800640 1 -- 172.21.15.44:0/4095057344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8ea90 msgr2=0x562e5cc87960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f53a5800640 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8ea90 0x562e5cc87960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f53a5800640 1 -- 172.21.15.44:0/4095057344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8e170 msgr2=0x562e5cca3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f53a5800640 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8e170 0x562e5cca3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f53a5800640 1 -- 172.21.15.44:0/4095057344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5c9053f0 con 0x562e5cc97670 2024-04-20T11:23:17.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f53a5800640 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc8a350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f539802c640 tx=0x7f5398041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0861e52484c0fac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.405+0000 7f5391600640 1 -- 172.21.15.44:0/4095057344 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398039040 con 0x562e5cc97670 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f5391600640 1 -- 172.21.15.44:0/4095057344 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5398032950 con 0x562e5cc97670 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f5391600640 1 -- 172.21.15.44:0/4095057344 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539804e8f0 con 0x562e5cc97670 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 -- 172.21.15.44:0/4095057344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 msgr2=0x562e5cc8a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc8a350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f539802c640 tx=0x7f5398041000 comp rx=0 tx=0).stop 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 -- 172.21.15.44:0/4095057344 shutdown_connections 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc97670 0x562e5cc8a350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc8ea90 0x562e5cc87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 --2- 172.21.15.44:0/4095057344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8e170 0x562e5cca3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 -- 172.21.15.44:0/4095057344 >> 172.21.15.44:0/4095057344 conn(0x562e5cc888a0 msgr2=0x562e5cc89c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 -- 172.21.15.44:0/4095057344 shutdown_connections 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.406+0000 7f53aa122740 1 -- 172.21.15.44:0/4095057344 wait complete. 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.407+0000 7f53aa122740 1 -- start start 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.407+0000 7f53aa122740 1 Processor -- start 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 0x562e5cc9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc9fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc97670 0x562e5cc9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01b1010 con 0x562e5cc8e170 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01b0940 con 0x562e5cc8ea90 2024-04-20T11:23:17.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01adb90 con 0x562e5cc97670 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53aa122740 10 client.0 _ll_register_callbacks cb 0x7f5394003db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 0x562e5cc9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 0x562e5cc9f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48150/0 (socket says 172.21.15.44:48150) 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 -- 172.21.15.44:0/3287839918 learned_addr learned my addr 172.21.15.44:0/3287839918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a5800640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc97670 0x562e5cc9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f539fe00640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc9fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 -- 172.21.15.44:0/3287839918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc97670 msgr2=0x562e5cc9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc97670 0x562e5cc9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 -- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 msgr2=0x562e5cc9fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc9fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.408+0000 7f53a4e00640 1 -- 172.21.15.44:0/3287839918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5c9cf7d0 con 0x562e5cc8e170 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f53a4e00640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 0x562e5cc9f830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f539409ae30 tx=0x7f53940d4370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6f4912a8904acd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f5386a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f5386a00640 1 -- 172.21.15.44:0/3287839918 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539409c6c0 con 0x562e5cc8e170 2024-04-20T11:23:17.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f5386a00640 1 -- 172.21.15.44:0/3287839918 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f539409c1d0 con 0x562e5cc8e170 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5398032950 con 0x562e5cc8e170 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f5386a00640 1 -- 172.21.15.44:0/3287839918 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f539409d280 con 0x562e5cc8e170 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f53aa122740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e5cd16c10 con 0x562e5cc8e170 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f5386a00640 1 -- 172.21.15.44:0/3287839918 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5398032950 con 0x562e5cc8e170 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.409+0000 7f5386a00640 1 -- 172.21.15.44:0/3287839918 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f539409bbd0 con 0x562e5cc8e170 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f53aa122740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f53aa122740 10 client.15474 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f53aa122740 20 client.15474 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f53aa122740 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cd171a0 0x562e5cd19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f53aa122740 4 client.15474 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:17.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e5cbfa8e0 con 0x562e5cd171a0 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f539fe00640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cd171a0 0x562e5cd19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f539fe00640 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cd171a0 0x562e5cd19590 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=876c0df8706489c9 server_cookie=16c919357e0802fd in_seq=0 out_seq=0 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.410+0000 7f5386a00640 10 client.15474 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f5386a00640 1 -- 172.21.15.44:0/3287839918 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e5cbfa8e0 con 0x562e5cd171a0 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f5386a00640 10 client.15474 handle_command_reply: tid=0 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 client.15474 shutdown 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 msgr2=0x562e5cc9f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 0x562e5cc9f830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f539409ae30 tx=0x7f53940d4370 comp rx=0 tx=0).stop 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 shutdown_connections 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e5cc97670 0x562e5cc9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e5cc8ea90 0x562e5cc9fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562e5cd171a0 0x562e5cd19590 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 --2- 172.21.15.44:0/3287839918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e5cc8e170 0x562e5cc9f830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f5386a00640 0 client.15474 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 >> 172.21.15.44:0/3287839918 conn(0x562e5cc888a0 msgr2=0x562e5cc94e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 shutdown_connections 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 1 -- 172.21.15.44:0/3287839918 wait complete. 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.411+0000 7f53aa122740 20 client.15474 trim_cache size 0 max 16384 2024-04-20T11:23:17.415 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f53880a3ed0 msgr2=0x7f53880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f53880a3ed0 0x7f53880a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5398005e30 tx=0x7f5398005d80 comp rx=0 tx=0).stop 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 msgr2=0x7f53a01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 0x7f53a01a06b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f539400ea60 tx=0x7f539400ef20 comp rx=0 tx=0).stop 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 shutdown_connections 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53a019a770 0x7f53a019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a0109320 0x7f53a01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f53880a3ed0 0x7f53880a6390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 --2- 172.21.15.44:0/3122958317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53a0104250 0x7f53a01a06b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 >> 172.21.15.44:0/3122958317 conn(0x7f53a00ffa70 msgr2=0x7f53a01005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 shutdown_connections 2024-04-20T11:23:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.412+0000 7f53a6200640 1 -- 172.21.15.44:0/3122958317 wait complete. 2024-04-20T11:23:17.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:23:17.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 -- 172.21.15.44:0/3709503398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964102070 msgr2=0x7f3964102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 --2- 172.21.15.44:0/3709503398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964102070 0x7f3964102450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402bc30 comp rx=0 tx=0).stop 2024-04-20T11:23:17.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 -- 172.21.15.44:0/3709503398 shutdown_connections 2024-04-20T11:23:17.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 --2- 172.21.15.44:0/3709503398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3964107140 0x7f396410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 --2- 172.21.15.44:0/3709503398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102a20 0x7f3964106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 --2- 172.21.15.44:0/3709503398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3964102070 0x7f3964102450 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.631+0000 7f3963e00640 1 -- 172.21.15.44:0/3709503398 >> 172.21.15.44:0/3709503398 conn(0x7f39640fd930 msgr2=0x7f39640ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.632+0000 7f3963e00640 1 -- 172.21.15.44:0/3709503398 shutdown_connections 2024-04-20T11:23:17.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.632+0000 7f3963e00640 1 -- 172.21.15.44:0/3709503398 wait complete. 2024-04-20T11:23:17.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 Processor -- start 2024-04-20T11:23:17.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 -- start start 2024-04-20T11:23:17.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 0x7f39641a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3964107140 0x7f39641a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f396419a7c0 0x7f396419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39641122e0 con 0x7f396419a7c0 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112160 con 0x7f3964102070 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.633+0000 7f3963e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112460 con 0x7f3964107140 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3963400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f396419a7c0 0x7f396419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 0x7f39641a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3964107140 0x7f39641a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 0x7f39641a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43822/0 (socket says 172.21.15.44:43822) 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 -- 172.21.15.44:0/1922600767 learned_addr learned my addr 172.21.15.44:0/1922600767 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 -- 172.21.15.44:0/1922600767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3964107140 msgr2=0x7f39641a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3964107140 0x7f39641a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 -- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f396419a7c0 msgr2=0x7f396419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f396419a7c0 0x7f396419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 -- 172.21.15.44:0/1922600767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396419b350 con 0x7f3964102070 2024-04-20T11:23:17.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.634+0000 7f3962a00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 0x7f39641a0650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3954005ab0 tx=0x7f395402dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.635+0000 7f395b400640 1 -- 172.21.15.44:0/1922600767 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954043020 con 0x7f3964102070 2024-04-20T11:23:17.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.635+0000 7f395b400640 1 -- 172.21.15.44:0/1922600767 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3954004250 con 0x7f3964102070 2024-04-20T11:23:17.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.635+0000 7f395b400640 1 -- 172.21.15.44:0/1922600767 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954003a10 con 0x7f3964102070 2024-04-20T11:23:17.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.635+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396419b5e0 con 0x7f3964102070 2024-04-20T11:23:17.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.635+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641a7810 con 0x7f3964102070 2024-04-20T11:23:17.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.638+0000 7f396c963740 1 Processor -- start 2024-04-20T11:23:17.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.638+0000 7f395b400640 1 -- 172.21.15.44:0/1922600767 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f3954046020 con 0x7f3964102070 2024-04-20T11:23:17.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.638+0000 7f396c963740 1 -- start start 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f396c963740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbf170 0x56425edbf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f396c963740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edc8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f396c963740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edc8670 0x56425edcaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f396c963740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56425ed24050 con 0x56425edbfa90 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f396c963740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56425ecca490 con 0x56425edc8670 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f396c963740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56425ed20010 con 0x56425edbf170 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f3962a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edc8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f3962a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edc8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48174/0 (socket says 172.21.15.44:48174) 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f3962a00640 1 -- 172.21.15.44:0/4227803845 learned_addr learned my addr 172.21.15.44:0/4227803845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f3962000640 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edc8670 0x56425edcaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.639+0000 7f3963400640 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbf170 0x56425edbf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f3962000640 1 -- 172.21.15.44:0/4227803845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbf170 msgr2=0x56425edbf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f3962000640 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbf170 0x56425edbf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f3962000640 1 -- 172.21.15.44:0/4227803845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 msgr2=0x56425edc8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f3962000640 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edc8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f3962000640 1 -- 172.21.15.44:0/4227803845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425ecd2070 con 0x56425edc8670 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f3962000640 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edc8670 0x56425edcaa60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f396419bb10 tx=0x7f3950011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25888f57bad3df97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.640+0000 7f395b400640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39480a3f50 0x7f39480a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f394fe00640 1 -- 172.21.15.44:0/4227803845 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950009d70 con 0x56425edc8670 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f394fe00640 1 -- 172.21.15.44:0/4227803845 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3950019910 con 0x56425edc8670 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f395b400640 1 -- 172.21.15.44:0/1922600767 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f39540f28a0 con 0x7f3964102070 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f394fe00640 1 -- 172.21.15.44:0/4227803845 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500189f0 con 0x56425edc8670 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f3963400640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39480a3f50 0x7f39480a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 -- 172.21.15.44:0/4227803845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edc8670 msgr2=0x56425edcaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f3963400640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39480a3f50 0x7f39480a6410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f395c005fd0 tx=0x7f395c009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edc8670 0x56425edcaa60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f396419bb10 tx=0x7f3950011810 comp rx=0 tx=0).stop 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 -- 172.21.15.44:0/4227803845 shutdown_connections 2024-04-20T11:23:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edc8670 0x56425edcaa60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edc8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 --2- 172.21.15.44:0/4227803845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbf170 0x56425edbf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.641+0000 7f396c963740 1 -- 172.21.15.44:0/4227803845 >> 172.21.15.44:0/4227803845 conn(0x56425edb98a0 msgr2=0x56425edb9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.642+0000 7f396c963740 1 -- 172.21.15.44:0/4227803845 shutdown_connections 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.642+0000 7f396c963740 1 -- 172.21.15.44:0/4227803845 wait complete. 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.642+0000 7f396c963740 1 -- start start 2024-04-20T11:23:17.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.642+0000 7f396c963740 1 Processor -- start 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edc56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbfa90 0x56425edc5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 0x56425edc6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56425edcb460 con 0x56425edc8670 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56425ecdd860 con 0x56425edbf170 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56425edcb5e0 con 0x56425edbfa90 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f396c963740 10 client.0 _ll_register_callbacks cb 0x56425ecabfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3963400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 0x56425edc6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edc56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edc56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43832/0 (socket says 172.21.15.44:43832) 2024-04-20T11:23:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3963400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 0x56425edc6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48190/0 (socket says 172.21.15.44:48190) 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962a00640 1 -- 172.21.15.44:0/4271004554 learned_addr learned my addr 172.21.15.44:0/4271004554 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962000640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbfa90 0x56425edc5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962a00640 1 -- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbfa90 msgr2=0x56425edc5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962a00640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbfa90 0x56425edc5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.643+0000 7f3962a00640 1 -- 172.21.15.44:0/4271004554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 msgr2=0x56425edc6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f3962a00640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 0x56425edc6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f3962a00640 1 -- 172.21.15.44:0/4271004554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425ece1950 con 0x56425edbf170 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f3963400640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 0x56425edc6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f3962a00640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edc56d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3954004650 tx=0x7f39540ba420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd98a801ec49886 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f394cc00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f394cc00640 1 -- 172.21.15.44:0/4271004554 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39540be0e0 con 0x56425edbf170 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3950019910 con 0x56425edbf170 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f394cc00640 1 -- 172.21.15.44:0/4271004554 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f39540c6f60 con 0x56425edbf170 2024-04-20T11:23:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f394cc00640 1 -- 172.21.15.44:0/4271004554 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f39540bee80 con 0x56425edbf170 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f396c963740 10 client.25607 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.644+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56425ecaf820 con 0x56425edbf170 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f394cc00640 1 -- 172.21.15.44:0/4271004554 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3950019910 con 0x56425edbf170 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f394cc00640 1 -- 172.21.15.44:0/4271004554 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f39540ba040 con 0x56425edbf170 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f396c963740 10 client.25607 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f396c963740 10 client.25607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f396c963740 20 client.25607 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f396c963740 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425edd85e0 0x56425edda9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f396c963740 4 client.25607 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425ed31a70 con 0x56425edd85e0 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.645+0000 7f3962000640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425edd85e0 0x56425edda9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.646+0000 7f3962000640 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425edd85e0 0x56425edda9f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=192dbfdc3067a6b8 server_cookie=70b087736858fdb9 in_seq=0 out_seq=0 2024-04-20T11:23:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.646+0000 7f394cc00640 10 client.25607 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f394cc00640 1 -- 172.21.15.44:0/4271004554 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56425ed31a70 con 0x56425edd85e0 2024-04-20T11:23:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f394cc00640 10 client.25607 handle_command_reply: tid=0 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 client.25607 shutdown 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 msgr2=0x56425edc56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edc56d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3954004650 tx=0x7f39540ba420 comp rx=0 tx=0).stop 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 shutdown_connections 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425edd85e0 0x56425edda9f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edc8670 0x56425edc6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edbfa90 0x56425edc5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 --2- 172.21.15.44:0/4271004554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edc56d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f394cc00640 0 client.25607 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.647+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 >> 172.21.15.44:0/4271004554 conn(0x56425edb98a0 msgr2=0x56425edbb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.648+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 shutdown_connections 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.648+0000 7f396c963740 1 -- 172.21.15.44:0/4271004554 wait complete. 2024-04-20T11:23:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.648+0000 7f396c963740 20 client.25607 trim_cache size 0 max 16384 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 Processor -- start 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 -- start start 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbf170 0x56425edc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 0x56425edd4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425ede0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39641afbf0 con 0x56425edbf170 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b1730 con 0x56425edbfa90 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.663+0000 7f396c963740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b02c0 con 0x56425edc8670 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 0x56425edd4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 0x56425edd4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43836/0 (socket says 172.21.15.44:43836) 2024-04-20T11:23:17.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 -- 172.21.15.44:0/749330714 learned_addr learned my addr 172.21.15.44:0/749330714 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3963400640 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425ede0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962a00640 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbf170 0x56425edc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 -- 172.21.15.44:0/749330714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 msgr2=0x56425ede0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425ede0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 -- 172.21.15.44:0/749330714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbf170 msgr2=0x56425edc7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbf170 0x56425edc7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 -- 172.21.15.44:0/749330714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425edbf8c0 con 0x56425edbfa90 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962a00640 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbf170 0x56425edc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f3962000640 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 0x56425edd4320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x56425edbaa30 tx=0x7f39500185d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ebecfcba7905d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.664+0000 7f394f400640 1 -- 172.21.15.44:0/749330714 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950005c50 con 0x56425edbfa90 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f394f400640 1 -- 172.21.15.44:0/749330714 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39500055c0 con 0x56425edbfa90 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f394f400640 1 -- 172.21.15.44:0/749330714 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500035f0 con 0x56425edbfa90 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 -- 172.21.15.44:0/749330714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 msgr2=0x56425edd4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 0x56425edd4320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x56425edbaa30 tx=0x7f39500185d0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 -- 172.21.15.44:0/749330714 shutdown_connections 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425ede0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbfa90 0x56425edd4320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 --2- 172.21.15.44:0/749330714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbf170 0x56425edc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 -- 172.21.15.44:0/749330714 >> 172.21.15.44:0/749330714 conn(0x56425edb98a0 msgr2=0x56425edbb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 -- 172.21.15.44:0/749330714 shutdown_connections 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.665+0000 7f396c963740 1 -- 172.21.15.44:0/749330714 wait complete. 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 -- start start 2024-04-20T11:23:17.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 Processor -- start 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edd0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edd0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425edcd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b1060 con 0x56425edbfa90 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b0990 con 0x56425edbf170 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39641adbc0 con 0x56425edc8670 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.666+0000 7f396c963740 10 client.0 _ll_register_callbacks cb 0x7f39540f8220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edd0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edd0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48214/0 (socket says 172.21.15.44:48214) 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 -- 172.21.15.44:0/1775966774 learned_addr learned my addr 172.21.15.44:0/1775966774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962a00640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edd0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3963400640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425edcd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 -- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 msgr2=0x56425edcd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425edcd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 -- 172.21.15.44:0/1775966774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 msgr2=0x56425edd0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edd0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 -- 172.21.15.44:0/1775966774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425ea363f0 con 0x56425edbfa90 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962a00640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edd0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f3962000640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edd0d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3950005bf0 tx=0x7f39500098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a047f311266a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.667+0000 7f394cc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f394cc00640 1 -- 172.21.15.44:0/1775966774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500052e0 con 0x56425edbfa90 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39500055c0 con 0x56425edbfa90 2024-04-20T11:23:17.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f394cc00640 1 -- 172.21.15.44:0/1775966774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3950018640 con 0x56425edbfa90 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f394cc00640 1 -- 172.21.15.44:0/1775966774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f395003e050 con 0x56425edbfa90 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f396c963740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56425eb007d0 con 0x56425edbfa90 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f394cc00640 1 -- 172.21.15.44:0/1775966774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39500055c0 con 0x56425edbfa90 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f394cc00640 1 -- 172.21.15.44:0/1775966774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f395002c070 con 0x56425edbfa90 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f396c963740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f396c963740 10 client.15492 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.668+0000 7f396c963740 20 client.15492 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.669+0000 7f396c963740 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425ee3f650 0x56425ee41a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.669+0000 7f396c963740 4 client.15492 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.669+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56425ed2b8e0 con 0x56425ee3f650 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.669+0000 7f3962a00640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425ee3f650 0x56425ee41a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.669+0000 7f3962a00640 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425ee3f650 0x56425ee41a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32c2e2e36670b7a1 server_cookie=b26352cc6417173 in_seq=0 out_seq=0 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.669+0000 7f394cc00640 10 client.15492 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:17.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f394cc00640 1 -- 172.21.15.44:0/1775966774 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56425ed2b8e0 con 0x56425ee3f650 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f394cc00640 10 client.15492 handle_command_reply: tid=0 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 client.15492 shutdown 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 msgr2=0x56425edd0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edd0d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3950005bf0 tx=0x7f39500098f0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 shutdown_connections 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x56425ee3f650 0x56425ee41a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56425edc8670 0x56425edcd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56425edbfa90 0x56425edd0d70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 --2- 172.21.15.44:0/1775966774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56425edbf170 0x56425edd0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f394cc00640 0 client.15492 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 >> 172.21.15.44:0/1775966774 conn(0x56425edb98a0 msgr2=0x56425edc5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 shutdown_connections 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 1 -- 172.21.15.44:0/1775966774 wait complete. 2024-04-20T11:23:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.670+0000 7f396c963740 20 client.15492 trim_cache size 0 max 16384 2024-04-20T11:23:17.674 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39480a3f50 msgr2=0x7f39480a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39480a3f50 0x7f39480a6410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f395c005fd0 tx=0x7f395c009500 comp rx=0 tx=0).stop 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 msgr2=0x7f39641a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 0x7f39641a0650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3954005ab0 tx=0x7f395402dc90 comp rx=0 tx=0).stop 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 shutdown_connections 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39480a3f50 0x7f39480a6410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f396419a7c0 0x7f396419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3964107140 0x7f39641a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 --2- 172.21.15.44:0/1922600767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3964102070 0x7f39641a0650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 >> 172.21.15.44:0/1922600767 conn(0x7f39640fd930 msgr2=0x7f396410ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 shutdown_connections 2024-04-20T11:23:17.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.671+0000 7f3963e00640 1 -- 172.21.15.44:0/1922600767 wait complete. 2024-04-20T11:23:17.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:17 smithi044 ceph-mon[16855]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T11:23:17.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1491208682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:23:17.689 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 -- 172.21.15.44:0/4149836121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c104c30 msgr2=0x7ff02c108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 --2- 172.21.15.44:0/4149836121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c104c30 0x7ff02c108c70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff018009860 tx=0x7ff01802bb70 comp rx=0 tx=0).stop 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 -- 172.21.15.44:0/4149836121 shutdown_connections 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 --2- 172.21.15.44:0/4149836121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c109350 0x7ff02c110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 --2- 172.21.15.44:0/4149836121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c104c30 0x7ff02c108c70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 --2- 172.21.15.44:0/4149836121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c104280 0x7ff02c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 -- 172.21.15.44:0/4149836121 >> 172.21.15.44:0/4149836121 conn(0x7ff02c0ffaa0 msgr2=0x7ff02c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 -- 172.21.15.44:0/4149836121 shutdown_connections 2024-04-20T11:23:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.903+0000 7ff02be00640 1 -- 172.21.15.44:0/4149836121 wait complete. 2024-04-20T11:23:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.905+0000 7ff02be00640 1 Processor -- start 2024-04-20T11:23:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.905+0000 7ff02be00640 1 -- start start 2024-04-20T11:23:17.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.905+0000 7ff02be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 0x7ff02c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.905+0000 7ff02be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c109350 0x7ff02c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.905+0000 7ff02be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c19a8a0 0x7ff02c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c114520 con 0x7ff02c109350 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1143a0 con 0x7ff02c104280 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1146a0 con 0x7ff02c19a8a0 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c109350 0x7ff02c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 0x7ff02c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c19a8a0 0x7ff02c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c109350 0x7ff02c1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48224/0 (socket says 172.21.15.44:48224) 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02a000640 1 -- 172.21.15.44:0/4290620644 learned_addr learned my addr 172.21.15.44:0/4290620644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 0x7ff02c1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43862/0 (socket says 172.21.15.44:43862) 2024-04-20T11:23:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02b400640 1 -- 172.21.15.44:0/4290620644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 msgr2=0x7ff02c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02b400640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 0x7ff02c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02b400640 1 -- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c109350 msgr2=0x7ff02c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02b400640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c109350 0x7ff02c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02b400640 1 -- 172.21.15.44:0/4290620644 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c19b3c0 con 0x7ff02c19a8a0 2024-04-20T11:23:17.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.906+0000 7ff02aa00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 0x7ff02c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.907+0000 7ff02b400640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c19a8a0 0x7ff02c19ad50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff024009fd0 tx=0x7ff02400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.907+0000 7ff023400640 1 -- 172.21.15.44:0/4290620644 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400cd60 con 0x7ff02c19a8a0 2024-04-20T11:23:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.907+0000 7ff023400640 1 -- 172.21.15.44:0/4290620644 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff024004b20 con 0x7ff02c19a8a0 2024-04-20T11:23:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.907+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c19b6b0 con 0x7ff02c19a8a0 2024-04-20T11:23:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.907+0000 7ff023400640 1 -- 172.21.15.44:0/4290620644 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024010760 con 0x7ff02c19a8a0 2024-04-20T11:23:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.907+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a78b0 con 0x7ff02c19a8a0 2024-04-20T11:23:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.910+0000 7ff0343dd740 1 Processor -- start 2024-04-20T11:23:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.910+0000 7ff023400640 1 -- 172.21.15.44:0/4290620644 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7ff024004cd0 con 0x7ff02c19a8a0 2024-04-20T11:23:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 -- start start 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d730b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730ba90 0x55a8d7314130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d7314670 0x55a8d7316a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a8d7270050 con 0x55a8d7314670 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a8d7216490 con 0x55a8d730b170 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff0343dd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a8d726c010 con 0x55a8d730ba90 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d7314670 0x55a8d7316a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.911+0000 7ff02aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730ba90 0x55a8d7314130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d7314670 0x55a8d7316a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48234/0 (socket says 172.21.15.44:48234) 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02a000640 1 -- 172.21.15.44:0/3703044976 learned_addr learned my addr 172.21.15.44:0/3703044976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02a000640 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d730b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02aa00640 1 -- 172.21.15.44:0/3703044976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 msgr2=0x55a8d730b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02aa00640 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d730b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02aa00640 1 -- 172.21.15.44:0/3703044976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d7314670 msgr2=0x55a8d7316a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02aa00640 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d7314670 0x55a8d7316a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02aa00640 1 -- 172.21.15.44:0/3703044976 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d721e070 con 0x55a8d730ba90 2024-04-20T11:23:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02a000640 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d730b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.912+0000 7ff02aa00640 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730ba90 0x55a8d7314130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc84d877992f1b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff023400640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0100a3ed0 0x7ff0100a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff017400640 1 -- 172.21.15.44:0/3703044976 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c01e070 con 0x55a8d730ba90 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff017400640 1 -- 172.21.15.44:0/3703044976 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff01c0035f0 con 0x55a8d730ba90 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff017400640 1 -- 172.21.15.44:0/3703044976 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c00f7a0 con 0x55a8d730ba90 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff023400640 1 -- 172.21.15.44:0/4290620644 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff0240d1040 con 0x7ff02c19a8a0 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff02a000640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0100a3ed0 0x7ff0100a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff0343dd740 1 -- 172.21.15.44:0/3703044976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730ba90 msgr2=0x55a8d7314130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.913+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730ba90 0x55a8d7314130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c0118d0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 -- 172.21.15.44:0/3703044976 shutdown_connections 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d7314670 0x55a8d7316a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730ba90 0x55a8d7314130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3703044976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d730b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 -- 172.21.15.44:0/3703044976 >> 172.21.15.44:0/3703044976 conn(0x55a8d73058a0 msgr2=0x55a8d7305c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff02a000640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0100a3ed0 0x7ff0100a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a8d7306a30 tx=0x7ff01802bb70 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 -- 172.21.15.44:0/3703044976 shutdown_connections 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 -- 172.21.15.44:0/3703044976 wait complete. 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 -- start start 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.914+0000 7ff0343dd740 1 Processor -- start 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 0x55a8d7311730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730ba90 0x55a8d7311c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a8d7317460 con 0x55a8d730b170 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a8d7229860 con 0x55a8d730ba90 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a8d73175e0 con 0x55a8d7314670 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff0343dd740 10 client.0 _ll_register_callbacks cb 0x55a8d71f7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff02aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 0x55a8d7311730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff02aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 0x55a8d7311730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48248/0 (socket says 172.21.15.44:48248) 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff02aa00640 1 -- 172.21.15.44:0/4040534598 learned_addr learned my addr 172.21.15.44:0/4040534598 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff02a000640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730ba90 0x55a8d7311c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.915+0000 7ff02b400640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff02aa00640 1 -- 172.21.15.44:0/4040534598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 msgr2=0x55a8d73121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff02aa00640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff02aa00640 1 -- 172.21.15.44:0/4040534598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730ba90 msgr2=0x55a8d7311c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff02aa00640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730ba90 0x55a8d7311c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff02aa00640 1 -- 172.21.15.44:0/4040534598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d722d950 con 0x55a8d730b170 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff02aa00640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 0x55a8d7311730 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff01c003e40 tx=0x7ff01c004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca0e3171aba1e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff00be00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff00be00640 1 -- 172.21.15.44:0/4040534598 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c01e040 con 0x55a8d730b170 2024-04-20T11:23:17.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.916+0000 7ff00be00640 1 -- 172.21.15.44:0/4040534598 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff01c004340 con 0x55a8d730b170 2024-04-20T11:23:17.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.917+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff01c0035f0 con 0x55a8d730b170 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.917+0000 7ff00be00640 1 -- 172.21.15.44:0/4040534598 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff01c032050 con 0x55a8d730b170 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.917+0000 7ff0343dd740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.917+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8d71fb820 con 0x55a8d730b170 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.917+0000 7ff00be00640 1 -- 172.21.15.44:0/4040534598 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff01c0035f0 con 0x55a8d730b170 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.917+0000 7ff00be00640 1 -- 172.21.15.44:0/4040534598 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7ff01c021030 con 0x55a8d730b170 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff0343dd740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff0343dd740 10 client.15504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff0343dd740 20 client.15504 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff0343dd740 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d7324690 0x55a8d7326a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff0343dd740 4 client.15504 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d727da70 con 0x55a8d7324690 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.918+0000 7ff02a000640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d7324690 0x55a8d7326a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.919+0000 7ff02a000640 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d7324690 0x55a8d7326a80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60a8aba34097b182 server_cookie=43f02eb30b546507 in_seq=0 out_seq=0 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.919+0000 7ff00be00640 10 client.15504 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.919+0000 7ff00be00640 1 -- 172.21.15.44:0/4040534598 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8d727da70 con 0x55a8d7324690 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.919+0000 7ff00be00640 10 client.15504 handle_command_reply: tid=0 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.919+0000 7ff0343dd740 1 client.15504 shutdown 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 msgr2=0x55a8d7311730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 0x55a8d7311730 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff01c003e40 tx=0x7ff01c004150 comp rx=0 tx=0).stop 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 shutdown_connections 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d7324690 0x55a8d7326a80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730ba90 0x55a8d7311c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 --2- 172.21.15.44:0/4040534598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730b170 0x55a8d7311730 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff00be00640 0 client.15504 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:17.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 >> 172.21.15.44:0/4040534598 conn(0x55a8d73058a0 msgr2=0x55a8d73075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 shutdown_connections 2024-04-20T11:23:17.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 1 -- 172.21.15.44:0/4040534598 wait complete. 2024-04-20T11:23:17.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.920+0000 7ff0343dd740 20 client.15504 trim_cache size 0 max 16384 2024-04-20T11:23:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.935+0000 7ff0343dd740 1 Processor -- start 2024-04-20T11:23:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.935+0000 7ff0343dd740 1 -- start start 2024-04-20T11:23:17.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.935+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d7313e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.935+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d7320320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.935+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.935+0000 7ff0343dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1afbb0 con 0x55a8d730ba90 2024-04-20T11:23:17.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff0343dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b16f0 con 0x55a8d730b170 2024-04-20T11:23:17.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff0343dd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b0280 con 0x55a8d7314670 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d7320320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d7320320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48252/0 (socket says 172.21.15.44:48252) 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 -- 172.21.15.44:0/3162667475 learned_addr learned my addr 172.21.15.44:0/3162667475 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02aa00640 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d7313e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02b400640 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 -- 172.21.15.44:0/3162667475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 msgr2=0x55a8d73073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 -- 172.21.15.44:0/3162667475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 msgr2=0x55a8d7313e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d7313e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.936+0000 7ff02a000640 1 -- 172.21.15.44:0/3162667475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d6f823f0 con 0x55a8d730ba90 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.937+0000 7ff02a000640 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d7320320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff018037040 tx=0x7ff01803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafc1aefceac0a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.937+0000 7ff016a00640 1 -- 172.21.15.44:0/3162667475 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018039040 con 0x55a8d730ba90 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.937+0000 7ff016a00640 1 -- 172.21.15.44:0/3162667475 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff01802c510 con 0x55a8d730ba90 2024-04-20T11:23:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.937+0000 7ff016a00640 1 -- 172.21.15.44:0/3162667475 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018032a30 con 0x55a8d730ba90 2024-04-20T11:23:17.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:17 smithi078 ceph-mon[21464]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T11:23:17.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1491208682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:23:17.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.937+0000 7ff0343dd740 1 -- 172.21.15.44:0/3162667475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 msgr2=0x55a8d7320320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.937+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d7320320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff018037040 tx=0x7ff01803b000 comp rx=0 tx=0).stop 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 -- 172.21.15.44:0/3162667475 shutdown_connections 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d7314670 0x55a8d73073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d7320320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 --2- 172.21.15.44:0/3162667475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d730b170 0x55a8d7313e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 -- 172.21.15.44:0/3162667475 >> 172.21.15.44:0/3162667475 conn(0x55a8d73058a0 msgr2=0x55a8d7306cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 -- 172.21.15.44:0/3162667475 shutdown_connections 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 -- 172.21.15.44:0/3162667475 wait complete. 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 -- start start 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.938+0000 7ff0343dd740 1 Processor -- start 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730b170 0x55a8d730e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d730eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 0x55a8d730efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b1020 con 0x55a8d730ba90 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b0950 con 0x55a8d7314670 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1adbe0 con 0x55a8d730b170 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff0343dd740 10 client.0 _ll_register_callbacks cb 0x7ff01c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff02b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 0x55a8d730efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff02a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d730eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff02b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 0x55a8d730efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43894/0 (socket says 172.21.15.44:43894) 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff02b400640 1 -- 172.21.15.44:0/2396299522 learned_addr learned my addr 172.21.15.44:0/2396299522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff02aa00640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730b170 0x55a8d730e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.939+0000 7ff02b400640 1 -- 172.21.15.44:0/2396299522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730b170 msgr2=0x55a8d730e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff02b400640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730b170 0x55a8d730e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff02b400640 1 -- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 msgr2=0x55a8d730eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff02b400640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d730eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff02b400640 1 -- 172.21.15.44:0/2396299522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d704c7d0 con 0x55a8d7314670 2024-04-20T11:23:17.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff02a000640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d730eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff02b400640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 0x55a8d730efe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff0240bbfd0 tx=0x7ff024005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf5d37bca490c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff00be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff00be00640 1 -- 172.21.15.44:0/2396299522 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a11c0 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff00be00640 1 -- 172.21.15.44:0/2396299522 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff0240b6290 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff01802c510 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff00be00640 1 -- 172.21.15.44:0/2396299522 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0240a5a80 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff0343dd740 10 client.25631 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.940+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a8d738ad30 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff00be00640 1 -- 172.21.15.44:0/2396299522 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff01802c510 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff00be00640 1 -- 172.21.15.44:0/2396299522 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7ff02409bcd0 con 0x55a8d7314670 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff0343dd740 10 client.25631 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff0343dd740 10 client.25631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff0343dd740 20 client.25631 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:17.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff0343dd740 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d738b4a0 0x55a8d738d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff0343dd740 4 client.25631 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8d72778e0 con 0x55a8d738b4a0 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.941+0000 7ff02aa00640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d738b4a0 0x55a8d738d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff02aa00640 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d738b4a0 0x55a8d738d8c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84ba6f6f44ec56c6 server_cookie=2cb1d3946aacd792 in_seq=0 out_seq=0 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff00be00640 10 client.25631 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff00be00640 1 -- 172.21.15.44:0/2396299522 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8d72778e0 con 0x55a8d738b4a0 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff00be00640 10 client.25631 handle_command_reply: tid=0 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 client.25631 shutdown 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 msgr2=0x55a8d730efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 0x55a8d730efe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff0240bbfd0 tx=0x7ff024005c00 comp rx=0 tx=0).stop 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 shutdown_connections 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55a8d738b4a0 0x55a8d738d8c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a8d7314670 0x55a8d730efe0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8d730ba90 0x55a8d730eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 --2- 172.21.15.44:0/2396299522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8d730b170 0x55a8d730e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff00be00640 0 client.25631 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 >> 172.21.15.44:0/2396299522 conn(0x55a8d73058a0 msgr2=0x55a8d7304960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.942+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 shutdown_connections 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff0343dd740 1 -- 172.21.15.44:0/2396299522 wait complete. 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff0343dd740 20 client.25631 trim_cache size 0 max 16384 2024-04-20T11:23:17.947 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:17.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0100a3ed0 msgr2=0x7ff0100a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0100a3ed0 0x7ff0100a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a8d7306a30 tx=0x7ff01802bb70 comp rx=0 tx=0).stop 2024-04-20T11:23:17.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c19a8a0 msgr2=0x7ff02c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:17.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c19a8a0 0x7ff02c19ad50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff024009fd0 tx=0x7ff02400ee50 comp rx=0 tx=0).stop 2024-04-20T11:23:17.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 shutdown_connections 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff02c19a8a0 0x7ff02c19ad50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0100a3ed0 0x7ff0100a6390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c109350 0x7ff02c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 --2- 172.21.15.44:0/4290620644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff02c104280 0x7ff02c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 >> 172.21.15.44:0/4290620644 conn(0x7ff02c0ffaa0 msgr2=0x7ff02c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.943+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 shutdown_connections 2024-04-20T11:23:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:17.944+0000 7ff02be00640 1 -- 172.21.15.44:0/4290620644 wait complete. 2024-04-20T11:23:17.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:23:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:17 smithi165 ceph-mon[21975]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T11:23:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1491208682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:23:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 -- 172.21.15.44:0/2076082540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 msgr2=0x7f36d41128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 --2- 172.21.15.44:0/2076082540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 0x7f36d41128e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc02bcc0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 -- 172.21.15.44:0/2076082540 shutdown_connections 2024-04-20T11:23:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 --2- 172.21.15.44:0/2076082540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36d4112e20 0x7f36d4115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 --2- 172.21.15.44:0/2076082540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 0x7f36d41128e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 --2- 172.21.15.44:0/2076082540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4074300 0x7f36d40746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.166+0000 7f36d3200640 1 -- 172.21.15.44:0/2076082540 >> 172.21.15.44:0/2076082540 conn(0x7f36d40fd900 msgr2=0x7f36d40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.167+0000 7f36d3200640 1 -- 172.21.15.44:0/2076082540 shutdown_connections 2024-04-20T11:23:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.167+0000 7f36d3200640 1 -- 172.21.15.44:0/2076082540 wait complete. 2024-04-20T11:23:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.168+0000 7f36d3200640 1 Processor -- start 2024-04-20T11:23:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.168+0000 7f36d3200640 1 -- start start 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.168+0000 7f36d3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36d4074300 0x7f36d419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.168+0000 7f36d3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 0x7f36d419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.168+0000 7f36d3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 0x7f36d41a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.168+0000 7f36d3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4118940 con 0x7f36d4074300 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41187c0 con 0x7f36d4074c20 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4118ac0 con 0x7f36d4112e20 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 0x7f36d41a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36d4074300 0x7f36d419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 0x7f36d41a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51508/0 (socket says 172.21.15.44:51508) 2024-04-20T11:23:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 -- 172.21.15.44:0/2209701230 learned_addr learned my addr 172.21.15.44:0/2209701230 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36cbe00640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 0x7f36d419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 -- 172.21.15.44:0/2209701230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 msgr2=0x7f36d419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 0x7f36d419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 -- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36d4074300 msgr2=0x7f36d419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36d4074300 0x7f36d419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.169+0000 7f36d2800640 1 -- 172.21.15.44:0/2209701230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41a3e10 con 0x7f36d4112e20 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.170+0000 7f36d2800640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 0x7f36d41a36e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00eb40 tx=0x7f36cc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.170+0000 7f36caa00640 1 -- 172.21.15.44:0/2209701230 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004b20 con 0x7f36d4112e20 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.170+0000 7f36caa00640 1 -- 172.21.15.44:0/2209701230 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36cc004cc0 con 0x7f36d4112e20 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.170+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d41a4100 con 0x7f36d4112e20 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.170+0000 7f36caa00640 1 -- 172.21.15.44:0/2209701230 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0108f0 con 0x7f36d4112e20 2024-04-20T11:23:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.170+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41ab990 con 0x7f36d4112e20 2024-04-20T11:23:18.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.173+0000 7f36db7ea740 1 Processor -- start 2024-04-20T11:23:18.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.173+0000 7f36caa00640 1 -- 172.21.15.44:0/2209701230 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f36cc005020 con 0x7f36d4112e20 2024-04-20T11:23:18.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 -- start start 2024-04-20T11:23:18.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf8448b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84494130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf84496a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf843f0050 con 0x55bf8448ba90 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf84396490 con 0x55bf8448b170 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36db7ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf843ec010 con 0x55bf84494670 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84494130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf8448b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84494130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48288/0 (socket says 172.21.15.44:48288) 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36d1e00640 1 -- 172.21.15.44:0/4113310110 learned_addr learned my addr 172.21.15.44:0/4113310110 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.174+0000 7f36cbe00640 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf84496a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36d1e00640 1 -- 172.21.15.44:0/4113310110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 msgr2=0x55bf84496a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36d1e00640 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf84496a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36d1e00640 1 -- 172.21.15.44:0/4113310110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 msgr2=0x55bf8448b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36d1e00640 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf8448b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36d1e00640 1 -- 172.21.15.44:0/4113310110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8439e070 con 0x55bf8448ba90 2024-04-20T11:23:18.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36cbe00640 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf84496a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:18.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36d1e00640 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84494130 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009d20 tx=0x7f36c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a05b1366f81187 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:18.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36caa00640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f36b80a3f50 0x7f36b80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.175+0000 7f36c3000640 1 -- 172.21.15.44:0/4113310110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c401e070 con 0x55bf8448ba90 2024-04-20T11:23:18.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36caa00640 1 -- 172.21.15.44:0/2209701230 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f36cc0d10f0 con 0x7f36d4112e20 2024-04-20T11:23:18.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36c3000640 1 -- 172.21.15.44:0/4113310110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36c4019b30 con 0x55bf8448ba90 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36c3000640 1 -- 172.21.15.44:0/4113310110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c400ec00 con 0x55bf8448ba90 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36cbe00640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f36b80a3f50 0x7f36b80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36cbe00640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f36b80a3f50 0x7f36b80a6410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f36bc006010 tx=0x7f36bc0056d0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36db7ea740 1 -- 172.21.15.44:0/4113310110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 msgr2=0x55bf84494130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.176+0000 7f36db7ea740 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84494130 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009d20 tx=0x7f36c4011810 comp rx=0 tx=0).stop 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 -- 172.21.15.44:0/4113310110 shutdown_connections 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf84496a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84494130 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 --2- 172.21.15.44:0/4113310110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf8448b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 -- 172.21.15.44:0/4113310110 >> 172.21.15.44:0/4113310110 conn(0x55bf844858a0 msgr2=0x55bf84485c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:18.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 -- 172.21.15.44:0/4113310110 shutdown_connections 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 -- 172.21.15.44:0/4113310110 wait complete. 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 -- start start 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.177+0000 7f36db7ea740 1 Processor -- start 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 0x55bf84491650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448ba90 0x55bf84491b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf84497460 con 0x55bf8448b170 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf843a9860 con 0x55bf8448ba90 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf844975e0 con 0x55bf84494670 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36db7ea740 10 client.0 _ll_register_callbacks cb 0x55bf84377fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:18.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 0x55bf84491650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448ba90 0x55bf84491b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 0x55bf84491650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48300/0 (socket says 172.21.15.44:48300) 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448ba90 0x55bf84491b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43936/0 (socket says 172.21.15.44:43936) 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36d1e00640 1 -- 172.21.15.44:0/855953364 learned_addr learned my addr 172.21.15.44:0/855953364 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.178+0000 7f36d2800640 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36d1e00640 1 -- 172.21.15.44:0/855953364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 msgr2=0x55bf844920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36d1e00640 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36d1e00640 1 -- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448ba90 msgr2=0x55bf84491b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36d1e00640 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448ba90 0x55bf84491b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36d1e00640 1 -- 172.21.15.44:0/855953364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf843ad950 con 0x55bf8448b170 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36d1e00640 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 0x55bf84491650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e630 tx=0x7f36c400e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4ac463241e2d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36b7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36b7400640 1 -- 172.21.15.44:0/855953364 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4026020 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36b7400640 1 -- 172.21.15.44:0/855953364 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f36c400ed90 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.179+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c4019b30 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36b7400640 1 -- 172.21.15.44:0/855953364 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36c4031050 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36db7ea740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf8437b820 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36b7400640 1 -- 172.21.15.44:0/855953364 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36c4019b30 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36b7400640 1 -- 172.21.15.44:0/855953364 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f36c40046c0 con 0x55bf8448b170 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36db7ea740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36db7ea740 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.180+0000 7f36db7ea740 20 client.15522 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.181+0000 7f36db7ea740 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf844a45b0 0x55bf844a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.181+0000 7f36db7ea740 4 client.15522 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:18.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.181+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf843fda70 con 0x55bf844a45b0 2024-04-20T11:23:18.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.181+0000 7f36cbe00640 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf844a45b0 0x55bf844a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.182+0000 7f36cbe00640 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf844a45b0 0x55bf844a69a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23cef8b343529ca server_cookie=d50aae240c362d6c in_seq=0 out_seq=0 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.182+0000 7f36b7400640 10 client.15522 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36b7400640 1 -- 172.21.15.44:0/855953364 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf843fda70 con 0x55bf844a45b0 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36b7400640 10 client.15522 handle_command_reply: tid=0 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 client.15522 shutdown 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 msgr2=0x55bf84491650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 0x55bf84491650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e630 tx=0x7f36c400e660 comp rx=0 tx=0).stop 2024-04-20T11:23:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 shutdown_connections 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf844a45b0 0x55bf844a69a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448ba90 0x55bf84491b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 --2- 172.21.15.44:0/855953364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448b170 0x55bf84491650 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 >> 172.21.15.44:0/855953364 conn(0x55bf844858a0 msgr2=0x55bf844875f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36b7400640 0 client.15522 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 shutdown_connections 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 1 -- 172.21.15.44:0/855953364 wait complete. 2024-04-20T11:23:18.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.183+0000 7f36db7ea740 20 client.15522 trim_cache size 0 max 16384 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.198+0000 7f36db7ea740 1 Processor -- start 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.198+0000 7f36db7ea740 1 -- start start 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf844a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84490f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36db7ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b4050 con 0x55bf8448ba90 2024-04-20T11:23:18.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36db7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b5b90 con 0x55bf8448b170 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36db7ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b4720 con 0x55bf84494670 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84490f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf844a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36d1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf844a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:43946/0 (socket says 172.21.15.44:43946) 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36d1e00640 1 -- 172.21.15.44:0/1312361498 learned_addr learned my addr 172.21.15.44:0/1312361498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.199+0000 7f36d2800640 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d1e00640 1 -- 172.21.15.44:0/1312361498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 msgr2=0x55bf844920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d1e00640 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d1e00640 1 -- 172.21.15.44:0/1312361498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 msgr2=0x55bf84490f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d1e00640 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84490f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d1e00640 1 -- 172.21.15.44:0/1312361498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf841023f0 con 0x55bf8448b170 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d2800640 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36d1e00640 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf844a0320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36c4004a90 tx=0x7f36c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8b9883ed66fa59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36c2600640 1 -- 172.21.15.44:0/1312361498 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4026070 con 0x55bf8448b170 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36c2600640 1 -- 172.21.15.44:0/1312361498 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36c4009780 con 0x55bf8448b170 2024-04-20T11:23:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.200+0000 7f36c2600640 1 -- 172.21.15.44:0/1312361498 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c401e040 con 0x55bf8448b170 2024-04-20T11:23:18.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 -- 172.21.15.44:0/1312361498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 msgr2=0x55bf844a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf844a0320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f36c4004a90 tx=0x7f36c4011810 comp rx=0 tx=0).stop 2024-04-20T11:23:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 -- 172.21.15.44:0/1312361498 shutdown_connections 2024-04-20T11:23:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf84494670 0x55bf844920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf84490f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 --2- 172.21.15.44:0/1312361498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8448b170 0x55bf844a0320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 -- 172.21.15.44:0/1312361498 >> 172.21.15.44:0/1312361498 conn(0x55bf844858a0 msgr2=0x55bf84484960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 -- 172.21.15.44:0/1312361498 shutdown_connections 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 -- 172.21.15.44:0/1312361498 wait complete. 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.201+0000 7f36db7ea740 1 -- start start 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 Processor -- start 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf8448b170 0x55bf8449ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf8448df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf84494670 0x55bf8448e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b54c0 con 0x55bf8448ba90 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b4df0 con 0x55bf84494670 2024-04-20T11:23:18.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41b2020 con 0x55bf8448b170 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36db7ea740 10 client.0 _ll_register_callbacks cb 0x7f36c4009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.202+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf8448df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf8448df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48330/0 (socket says 172.21.15.44:48330) 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 -- 172.21.15.44:0/3672243367 learned_addr learned my addr 172.21.15.44:0/3672243367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36d2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf84494670 0x55bf8448e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36d1e00640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf8448b170 0x55bf8449ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 -- 172.21.15.44:0/3672243367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf8448b170 msgr2=0x55bf8449ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf8448b170 0x55bf8449ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 -- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf84494670 msgr2=0x55bf8448e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf84494670 0x55bf8448e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 -- 172.21.15.44:0/3672243367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf841cc7d0 con 0x55bf8448ba90 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36d2800640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf84494670 0x55bf8448e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36cbe00640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf8448df00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d20c0856f6068 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36b7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.203+0000 7f36b7400640 1 -- 172.21.15.44:0/3672243367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc05e070 con 0x55bf8448ba90 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36b7400640 1 -- 172.21.15.44:0/3672243367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f36bc02cc70 con 0x55bf8448ba90 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c4009780 con 0x55bf8448ba90 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36b7400640 1 -- 172.21.15.44:0/3672243367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36bc06a050 con 0x55bf8448ba90 2024-04-20T11:23:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36db7ea740 10 client.15528 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf8450a630 con 0x55bf8448ba90 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36b7400640 1 -- 172.21.15.44:0/3672243367 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36c4009780 con 0x55bf8448ba90 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36b7400640 1 -- 172.21.15.44:0/3672243367 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f36bc04d040 con 0x55bf8448ba90 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36db7ea740 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.204+0000 7f36db7ea740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36db7ea740 20 client.15528 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36db7ea740 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf8450ad80 0x55bf8450d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36db7ea740 4 client.15528 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf843f78e0 con 0x55bf8450ad80 2024-04-20T11:23:18.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36d1e00640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf8450ad80 0x55bf8450d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36d1e00640 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf8450ad80 0x55bf8450d1a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9180108a177dc6ae server_cookie=3d452bdb1b70c219 in_seq=0 out_seq=0 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.205+0000 7f36b7400640 10 client.15528 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36b7400640 1 -- 172.21.15.44:0/3672243367 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf843f78e0 con 0x55bf8450ad80 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36b7400640 10 client.15528 handle_command_reply: tid=0 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 client.15528 shutdown 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 msgr2=0x55bf8448df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf8448df00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc005e40 comp rx=0 tx=0).stop 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 shutdown_connections 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf8450ad80 0x55bf8450d1a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf84494670 0x55bf8448e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf8448ba90 0x55bf8448df00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36b7400640 0 client.15528 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 --2- 172.21.15.44:0/3672243367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf8448b170 0x55bf8449ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 >> 172.21.15.44:0/3672243367 conn(0x55bf844858a0 msgr2=0x55bf844868a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 shutdown_connections 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 1 -- 172.21.15.44:0/3672243367 wait complete. 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.206+0000 7f36db7ea740 20 client.15528 trim_cache size 0 max 16384 2024-04-20T11:23:18.216 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f36b80a3f50 msgr2=0x7f36b80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f36b80a3f50 0x7f36b80a6410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f36bc006010 tx=0x7f36bc0056d0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 msgr2=0x7f36d41a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 0x7f36d41a36e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00eb40 tx=0x7f36cc00c2d0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 shutdown_connections 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36d4112e20 0x7f36d41a36e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36d4074c20 0x7f36d419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f36b80a3f50 0x7f36b80a6410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 --2- 172.21.15.44:0/2209701230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36d4074300 0x7f36d419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 >> 172.21.15.44:0/2209701230 conn(0x7f36d40fd900 msgr2=0x7f36d40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:18.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 shutdown_connections 2024-04-20T11:23:18.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:18.207+0000 7f36d3200640 1 -- 172.21.15.44:0/2209701230 wait complete. 2024-04-20T11:23:19.226 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:23:19.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:23:19.227 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:23:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 -- 172.21.15.44:0/555369144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 msgr2=0x7f07ec108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 --2- 172.21.15.44:0/555369144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec108a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009930 tx=0x7f07d802bc30 comp rx=0 tx=0).stop 2024-04-20T11:23:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 -- 172.21.15.44:0/555369144 shutdown_connections 2024-04-20T11:23:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 --2- 172.21.15.44:0/555369144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec108f50 0x7f07ec1153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 --2- 172.21.15.44:0/555369144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec108a10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 --2- 172.21.15.44:0/555369144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07ec10e600 0x7f07ec10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 -- 172.21.15.44:0/555369144 >> 172.21.15.44:0/555369144 conn(0x7f07ec0764a0 msgr2=0x7f07ec0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 -- 172.21.15.44:0/555369144 shutdown_connections 2024-04-20T11:23:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.459+0000 7f07f3600640 1 -- 172.21.15.44:0/555369144 wait complete. 2024-04-20T11:23:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.460+0000 7f07f3600640 1 Processor -- start 2024-04-20T11:23:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 -- start start 2024-04-20T11:23:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07ec108f50 0x7f07ec19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 0x7f07ec1a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1189d0 con 0x7f07ec10e600 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec118850 con 0x7f07ec108590 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f3600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec118b50 con 0x7f07ec108f50 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 0x7f07ec1a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f0c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f0c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58550/0 (socket says 172.21.15.44:58550) 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 0x7f07ec1a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49844/0 (socket says 172.21.15.44:49844) 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07f0c00640 1 -- 172.21.15.44:0/1503372890 learned_addr learned my addr 172.21.15.44:0/1503372890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.461+0000 7f07ebe00640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07ec108f50 0x7f07ec19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f0c00640 1 -- 172.21.15.44:0/1503372890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07ec108f50 msgr2=0x7f07ec19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f0c00640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07ec108f50 0x7f07ec19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f0c00640 1 -- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 msgr2=0x7f07ec1a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f0c00640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 0x7f07ec1a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f0c00640 1 -- 172.21.15.44:0/1503372890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec1a3ec0 con 0x7f07ec108590 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f1600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 0x7f07ec1a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07f0c00640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec19eed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07e000e9f0 tx=0x7f07e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07e9600640 1 -- 172.21.15.44:0/1503372890 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e000cd80 con 0x7f07ec108590 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.462+0000 7f07e9600640 1 -- 172.21.15.44:0/1503372890 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07e0005520 con 0x7f07ec108590 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.463+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec1a41b0 con 0x7f07ec108590 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.463+0000 7f07e9600640 1 -- 172.21.15.44:0/1503372890 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0010690 con 0x7f07ec108590 2024-04-20T11:23:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.463+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec1aba40 con 0x7f07ec108590 2024-04-20T11:23:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.466+0000 7f07e9600640 1 -- 172.21.15.44:0/1503372890 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f07e00046c0 con 0x7f07ec108590 2024-04-20T11:23:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.466+0000 7f07f626f740 1 Processor -- start 2024-04-20T11:23:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.466+0000 7f07f626f740 1 -- start start 2024-04-20T11:23:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f626f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f626f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f626f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f626f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5647b4ebad20 con 0x5647b4f5f410 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f626f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647b4e613e0 con 0x5647b4f55f10 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f626f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647b4eb6ce0 con 0x5647b4f56830 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:58552/0 (socket says 172.21.15.44:58552) 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 -- 172.21.15.44:0/3561723333 learned_addr learned my addr 172.21.15.44:0/3561723333 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f0c00640 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 -- 172.21.15.44:0/3561723333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 msgr2=0x5647b4f5eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f5eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 -- 172.21.15.44:0/3561723333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 msgr2=0x5647b4f61800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.467+0000 7f07f1600640 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f61800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07f1600640 1 -- 172.21.15.44:0/3561723333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b4e80980 con 0x5647b4f55f10 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07ebe00640 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07f1600640 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f562f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07e40117e0 tx=0x7f07e4011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c521a64336b12b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07ddc00640 1 -- 172.21.15.44:0/3561723333 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4011310 con 0x5647b4f55f10 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07ddc00640 1 -- 172.21.15.44:0/3561723333 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07e400f9a0 con 0x5647b4f55f10 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07ddc00640 1 -- 172.21.15.44:0/3561723333 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4004620 con 0x5647b4f55f10 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.468+0000 7f07e9600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07d40a3f50 0x7f07d40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07e9600640 1 -- 172.21.15.44:0/1503372890 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f07e00d0e00 con 0x7f07ec108590 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f0c00640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07d40a3f50 0x7f07d40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 -- 172.21.15.44:0/3561723333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 msgr2=0x5647b4f562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f562f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07e40117e0 tx=0x7f07e4011ca0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 -- 172.21.15.44:0/3561723333 shutdown_connections 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f5eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 --2- 172.21.15.44:0/3561723333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f562f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 -- 172.21.15.44:0/3561723333 >> 172.21.15.44:0/3561723333 conn(0x5647b4f50640 msgr2=0x5647b4f50a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 -- 172.21.15.44:0/3561723333 shutdown_connections 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.469+0000 7f07f626f740 1 -- 172.21.15.44:0/3561723333 wait complete. 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f0c00640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07d40a3f50 0x7f07d40a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07e00ce3e0 tx=0x7f07e00d48b0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 -- start start 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 Processor -- start 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f5c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f56830 0x5647b4f5c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f5cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5647b4f62200 con 0x5647b4f5f410 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647b4e747a0 con 0x5647b4f56830 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.470+0000 7f07f626f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647b4f62380 con 0x5647b4f55f10 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07f626f740 10 client.0 _ll_register_callbacks cb 0x5647b4e76b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07f1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f5c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f5cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f56830 0x5647b4f5c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f5cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49856/0 (socket says 172.21.15.44:49856) 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07f1600640 1 -- 172.21.15.44:0/2481775566 learned_addr learned my addr 172.21.15.44:0/2481775566 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07ebe00640 1 -- 172.21.15.44:0/2481775566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 msgr2=0x5647b4f5c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07ebe00640 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f5c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07ebe00640 1 -- 172.21.15.44:0/2481775566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 msgr2=0x5647b4f5cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07ebe00640 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f5cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07ebe00640 1 -- 172.21.15.44:0/2481775566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b4c957e0 con 0x5647b4f56830 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.471+0000 7f07f1600640 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f5cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07ebe00640 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f56830 0x5647b4f5c9b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07d802c170 tx=0x7f07d802cd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968beab99d912c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07d2000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07d2000640 1 -- 172.21.15.44:0/2481775566 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8004310 con 0x5647b4f56830 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e400f9a0 con 0x5647b4f56830 2024-04-20T11:23:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07d2000640 1 -- 172.21.15.44:0/2481775566 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f07d8005030 con 0x5647b4f56830 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07d2000640 1 -- 172.21.15.44:0/2481775566 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07d804e050 con 0x5647b4f56830 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07f626f740 10 client.25661 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.472+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647b4cf5f70 con 0x5647b4f56830 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07d2000640 1 -- 172.21.15.44:0/2481775566 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07e400f9a0 con 0x5647b4f56830 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07d2000640 1 -- 172.21.15.44:0/2481775566 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f07d80035f0 con 0x5647b4f56830 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07f626f740 10 client.25661 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07f626f740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07f626f740 20 client.25661 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07f626f740 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4f6f800 0x5647b4f71bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07f626f740 4 client.25661 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.473+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647b4ec8740 con 0x5647b4f6f800 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.474+0000 7f07f1600640 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4f6f800 0x5647b4f71bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.474+0000 7f07f1600640 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4f6f800 0x5647b4f71bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5fcd98d9ea7dd3 server_cookie=6205a06cb6b991a7 in_seq=0 out_seq=0 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.474+0000 7f07d2000640 10 client.25661 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07d2000640 1 -- 172.21.15.44:0/2481775566 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647b4ec8740 con 0x5647b4f6f800 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07d2000640 10 client.25661 handle_command_reply: tid=0 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 client.25661 shutdown 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f56830 msgr2=0x5647b4f5c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f56830 0x5647b4f5c9b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07d802c170 tx=0x7f07d802cd40 comp rx=0 tx=0).stop 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 shutdown_connections 2024-04-20T11:23:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4f6f800 0x5647b4f71bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f5cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f56830 0x5647b4f5c9b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07d2000640 0 client.25661 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 --2- 172.21.15.44:0/2481775566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f5c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.475+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 >> 172.21.15.44:0/2481775566 conn(0x5647b4f50640 msgr2=0x5647b4f52390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.476+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 shutdown_connections 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.476+0000 7f07f626f740 1 -- 172.21.15.44:0/2481775566 wait complete. 2024-04-20T11:23:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.476+0000 7f07f626f740 20 client.25661 trim_cache size 0 max 16384 2024-04-20T11:23:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.495+0000 7f07f626f740 1 Processor -- start 2024-04-20T11:23:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.495+0000 7f07f626f740 1 -- start start 2024-04-20T11:23:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f626f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f6b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f626f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 0x5647b4f5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f626f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f5f410 0x5647b4f4f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f626f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b4190 con 0x5647b4f56830 2024-04-20T11:23:19.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f626f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b5cd0 con 0x5647b4f5f410 2024-04-20T11:23:19.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f626f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b4860 con 0x5647b4f55f10 2024-04-20T11:23:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 0x5647b4f5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f6b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 0x5647b4f5bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49858/0 (socket says 172.21.15.44:49858) 2024-04-20T11:23:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07ebe00640 1 -- 172.21.15.44:0/1876801278 learned_addr learned my addr 172.21.15.44:0/1876801278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.496+0000 7f07f1600640 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f5f410 0x5647b4f4f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07ebe00640 1 -- 172.21.15.44:0/1876801278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 msgr2=0x5647b4f6b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07ebe00640 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f6b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07ebe00640 1 -- 172.21.15.44:0/1876801278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f5f410 msgr2=0x5647b4f4f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07ebe00640 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f5f410 0x5647b4f4f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07ebe00640 1 -- 172.21.15.44:0/1876801278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647b4f56660 con 0x5647b4f56830 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07ebe00640 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 0x5647b4f5bd70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f07d8000c00 tx=0x7f07d80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09900db97adf868 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.497+0000 7f07dd200640 1 -- 172.21.15.44:0/1876801278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8057070 con 0x5647b4f56830 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07dd200640 1 -- 172.21.15.44:0/1876801278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07d8004bc0 con 0x5647b4f56830 2024-04-20T11:23:19.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07dd200640 1 -- 172.21.15.44:0/1876801278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8004e40 con 0x5647b4f56830 2024-04-20T11:23:19.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 -- 172.21.15.44:0/1876801278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 msgr2=0x5647b4f5bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 0x5647b4f5bd70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f07d8000c00 tx=0x7f07d80095f0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 -- 172.21.15.44:0/1876801278 shutdown_connections 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f5f410 0x5647b4f4f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f56830 0x5647b4f5bd70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 --2- 172.21.15.44:0/1876801278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f55f10 0x5647b4f6b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 -- 172.21.15.44:0/1876801278 >> 172.21.15.44:0/1876801278 conn(0x5647b4f50640 msgr2=0x5647b4f4edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 -- 172.21.15.44:0/1876801278 shutdown_connections 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.498+0000 7f07f626f740 1 -- 172.21.15.44:0/1876801278 wait complete. 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 -- start start 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 Processor -- start 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b5600 con 0x5647b4f5f410 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b4f30 con 0x5647b4f55f10 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1b2160 con 0x5647b4f56830 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.499+0000 7f07f626f740 10 client.0 _ll_register_callbacks cb 0x7f07d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f59600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48972/0 (socket says 172.21.15.44:48972) 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 -- 172.21.15.44:0/102180671 learned_addr learned my addr 172.21.15.44:0/102180671 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07f1600640 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 -- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 msgr2=0x5647b4f590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 -- 172.21.15.44:0/102180671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 msgr2=0x5647b4f59b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f59b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 -- 172.21.15.44:0/102180671 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647b4b8d170 con 0x5647b4f56830 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.500+0000 7f07ebe00640 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f59600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f07d802c170 tx=0x7f07d80038e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337804bd4b1fd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07d2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07d2000640 1 -- 172.21.15.44:0/102180671 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8057070 con 0x5647b4f56830 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d8004bc0 con 0x5647b4f56830 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07d2000640 1 -- 172.21.15.44:0/102180671 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f07d80046c0 con 0x5647b4f56830 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07d2000640 1 -- 172.21.15.44:0/102180671 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07d8005d50 con 0x5647b4f56830 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07f626f740 10 client.25669 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.501+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647b4fd5dc0 con 0x5647b4f56830 2024-04-20T11:23:19.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07d2000640 1 -- 172.21.15.44:0/102180671 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07d8004bc0 con 0x5647b4f56830 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07d2000640 1 -- 172.21.15.44:0/102180671 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f07d80344e0 con 0x5647b4f56830 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f626f740 10 client.25669 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f626f740 10 client.25669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f626f740 20 client.25669 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f626f740 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4fd63c0 0x5647b4fd87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f626f740 4 client.25669 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647b4ec25b0 con 0x5647b4fd63c0 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.502+0000 7f07f1600640 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4fd63c0 0x5647b4fd87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.503+0000 7f07f1600640 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4fd63c0 0x5647b4fd87b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628422b997f8a36b server_cookie=f7f972cc53dda51c in_seq=0 out_seq=0 2024-04-20T11:23:19.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.503+0000 7f07d2000640 10 client.25669 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07d2000640 1 -- 172.21.15.44:0/102180671 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647b4ec25b0 con 0x5647b4fd63c0 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07d2000640 10 client.25669 handle_command_reply: tid=0 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 client.25669 shutdown 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 msgr2=0x5647b4f59600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f59600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f07d802c170 tx=0x7f07d80038e0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 shutdown_connections 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647b4fd63c0 0x5647b4fd87b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647b4f5f410 0x5647b4f59b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647b4f56830 0x5647b4f59600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 --2- 172.21.15.44:0/102180671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647b4f55f10 0x5647b4f590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 >> 172.21.15.44:0/102180671 conn(0x5647b4f50640 msgr2=0x5647b4f5ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.504+0000 7f07d2000640 0 client.25669 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 shutdown_connections 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f626f740 1 -- 172.21.15.44:0/102180671 wait complete. 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f626f740 20 client.25669 trim_cache size 0 max 16384 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "13b83344-ac73-4f5a-9822-d0e8b40a59ad", 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:23:19.509 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:23:19.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07d40a3f50 msgr2=0x7f07d40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f3600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07d40a3f50 0x7f07d40a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07e00ce3e0 tx=0x7f07e00d48b0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 msgr2=0x7f07ec19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.505+0000 7f07f3600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec19eed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07e000e9f0 tx=0x7f07e000edf0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 shutdown_connections 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07d40a3f50 0x7f07d40a6410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07ec10e600 0x7f07ec1a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07ec108f50 0x7f07ec19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 --2- 172.21.15.44:0/1503372890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07ec108590 0x7f07ec19eed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 >> 172.21.15.44:0/1503372890 conn(0x7f07ec0764a0 msgr2=0x7f07ec077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 shutdown_connections 2024-04-20T11:23:19.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.506+0000 7f07f3600640 1 -- 172.21.15.44:0/1503372890 wait complete. 2024-04-20T11:23:19.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:23:19.525 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:23:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 -- 172.21.15.44:0/4153874849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854109340 msgr2=0x7fe854110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 --2- 172.21.15.44:0/4153874849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854109340 0x7fe854110f10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009900 tx=0x7fe84c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:23:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 -- 172.21.15.44:0/4153874849 shutdown_connections 2024-04-20T11:23:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 --2- 172.21.15.44:0/4153874849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854109340 0x7fe854110f10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 --2- 172.21.15.44:0/4153874849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854104c20 0x7fe854108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 --2- 172.21.15.44:0/4153874849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854104270 0x7fe854104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.736+0000 7fe859e00640 1 -- 172.21.15.44:0/4153874849 >> 172.21.15.44:0/4153874849 conn(0x7fe8540ffa70 msgr2=0x7fe854101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.737+0000 7fe859e00640 1 -- 172.21.15.44:0/4153874849 shutdown_connections 2024-04-20T11:23:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.737+0000 7fe859e00640 1 -- 172.21.15.44:0/4153874849 wait complete. 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 Processor -- start 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 -- start start 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 0x7fe8541a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854109340 0x7fe8541a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85419a770 0x7fe85419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe854114510 con 0x7fe854104270 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe854114390 con 0x7fe85419a770 2024-04-20T11:23:19.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe859e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe854114690 con 0x7fe854109340 2024-04-20T11:23:19.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.738+0000 7fe858a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 0x7fe8541a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854109340 0x7fe8541a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe859400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85419a770 0x7fe85419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe858a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 0x7fe8541a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49890/0 (socket says 172.21.15.44:49890) 2024-04-20T11:23:19.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe858a00640 1 -- 172.21.15.44:0/649420884 learned_addr learned my addr 172.21.15.44:0/649420884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 -- 172.21.15.44:0/649420884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85419a770 msgr2=0x7fe85419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85419a770 0x7fe85419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 -- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 msgr2=0x7fe8541a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 0x7fe8541a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 -- 172.21.15.44:0/649420884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85419b370 con 0x7fe854109340 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe858a00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 0x7fe8541a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.739+0000 7fe853e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854109340 0x7fe8541a0bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e9c0 tx=0x7fe84000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.740+0000 7fe851600640 1 -- 172.21.15.44:0/649420884 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84000ccc0 con 0x7fe854109340 2024-04-20T11:23:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.740+0000 7fe851600640 1 -- 172.21.15.44:0/649420884 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe840010930 con 0x7fe854109340 2024-04-20T11:23:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.740+0000 7fe851600640 1 -- 172.21.15.44:0/649420884 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8400189c0 con 0x7fe854109340 2024-04-20T11:23:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.740+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541a7450 con 0x7fe854109340 2024-04-20T11:23:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.740+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541a7800 con 0x7fe854109340 2024-04-20T11:23:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.743+0000 7fe851600640 1 -- 172.21.15.44:0/649420884 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fe840010ae0 con 0x7fe854109340 2024-04-20T11:23:19.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.743+0000 7fe85e02c740 1 Processor -- start 2024-04-20T11:23:19.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.743+0000 7fe85e02c740 1 -- start start 2024-04-20T11:23:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdbb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdbdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe85e02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55deffd18150 con 0x55deffdb2ca0 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe85e02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55deffcbe370 con 0x55deffdbb880 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe85e02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55deffd14110 con 0x55deffdb2380 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe858a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdbb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe859400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe853e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdbdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe859400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49010/0 (socket says 172.21.15.44:49010) 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.744+0000 7fe859400640 1 -- 172.21.15.44:0/1324779703 learned_addr learned my addr 172.21.15.44:0/1324779703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe859400640 1 -- 172.21.15.44:0/1324779703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 msgr2=0x55deffdbdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe859400640 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdbdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe859400640 1 -- 172.21.15.44:0/1324779703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 msgr2=0x55deffdbb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe859400640 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdbb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe859400640 1 -- 172.21.15.44:0/1324779703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deffaf47d0 con 0x55deffdb2380 2024-04-20T11:23:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe858a00640 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdbb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe853e00640 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdbdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe859400640 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb2760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe84c036040 tx=0x7fe84c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e376f5359fe0fd6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe851600640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe83c0a3ed0 0x7fe83c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe845c00640 1 -- 172.21.15.44:0/1324779703 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c02c660 con 0x55deffdb2380 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe845c00640 1 -- 172.21.15.44:0/1324779703 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe84c02c800 con 0x55deffdb2380 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe851600640 1 -- 172.21.15.44:0/649420884 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe8400d0cf0 con 0x7fe854109340 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.745+0000 7fe845c00640 1 -- 172.21.15.44:0/1324779703 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c005540 con 0x55deffdb2380 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe858a00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe83c0a3ed0 0x7fe83c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 -- 172.21.15.44:0/1324779703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 msgr2=0x55deffdb2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb2760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe84c036040 tx=0x7fe84c009560 comp rx=0 tx=0).stop 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe858a00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe83c0a3ed0 0x7fe83c0a6390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55deffdb1e50 tx=0x7fe848009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 -- 172.21.15.44:0/1324779703 shutdown_connections 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdbdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdbb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1324779703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb2760 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 -- 172.21.15.44:0/1324779703 >> 172.21.15.44:0/1324779703 conn(0x55deffdacab0 msgr2=0x55deffdace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 -- 172.21.15.44:0/1324779703 shutdown_connections 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.746+0000 7fe85e02c740 1 -- 172.21.15.44:0/1324779703 wait complete. 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.747+0000 7fe85e02c740 1 -- start start 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.747+0000 7fe85e02c740 1 Processor -- start 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdb8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55deffdbe610 con 0x55deffdb2ca0 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55deffcd1740 con 0x55deffdbb880 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55deffdbe790 con 0x55deffdb2380 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe85e02c740 10 client.0 _ll_register_callbacks cb 0x55deffcccb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe853e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdb8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe858a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe859400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe853e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdb8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49908/0 (socket says 172.21.15.44:49908) 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe853e00640 1 -- 172.21.15.44:0/3365429224 learned_addr learned my addr 172.21.15.44:0/3365429224 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe859400640 1 -- 172.21.15.44:0/3365429224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 msgr2=0x55deffdb8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe859400640 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe859400640 1 -- 172.21.15.44:0/3365429224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 msgr2=0x55deffdb8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe859400640 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdb8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.748+0000 7fe859400640 1 -- 172.21.15.44:0/3365429224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deffa2a3f0 con 0x55deffdbb880 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe859400640 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb9380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009900 tx=0x7fe84c02bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f40815a2b00f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe83a000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe83a000640 1 -- 172.21.15.44:0/3365429224 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0046c0 con 0x55deffdbb880 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c02c800 con 0x55deffdbb880 2024-04-20T11:23:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe83a000640 1 -- 172.21.15.44:0/3365429224 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe84c004860 con 0x55deffdbb880 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe83a000640 1 -- 172.21.15.44:0/3365429224 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe84c059050 con 0x55deffdbb880 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe85e02c740 10 client.25679 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55deff9ea0f0 con 0x55deffdbb880 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.749+0000 7fe83a000640 1 -- 172.21.15.44:0/3365429224 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe84c02c800 con 0x55deffdbb880 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe83a000640 1 -- 172.21.15.44:0/3365429224 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fe84c034c50 con 0x55deffdbb880 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe85e02c740 10 client.25679 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe85e02c740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe85e02c740 20 client.25679 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe85e02c740 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffdcba80 0x55deffdcde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe85e02c740 4 client.25679 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deffd25b70 con 0x55deffdcba80 2024-04-20T11:23:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.750+0000 7fe858a00640 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffdcba80 0x55deffdcde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.751+0000 7fe858a00640 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffdcba80 0x55deffdcde70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462de4884536f5e5 server_cookie=db145797bcb78b26 in_seq=0 out_seq=0 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.751+0000 7fe83a000640 10 client.25679 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.751+0000 7fe83a000640 1 -- 172.21.15.44:0/3365429224 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55deffd25b70 con 0x55deffdcba80 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.751+0000 7fe83a000640 10 client.25679 handle_command_reply: tid=0 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 client.25679 shutdown 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 msgr2=0x55deffdb9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb9380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009900 tx=0x7fe84c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 shutdown_connections 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffdcba80 0x55deffdcde70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb9380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdb8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 --2- 172.21.15.44:0/3365429224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdb8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe83a000640 0 client.25679 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 >> 172.21.15.44:0/3365429224 conn(0x55deffdacab0 msgr2=0x55deffdae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 shutdown_connections 2024-04-20T11:23:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.752+0000 7fe85e02c740 1 -- 172.21.15.44:0/3365429224 wait complete. 2024-04-20T11:23:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.753+0000 7fe85e02c740 20 client.25679 trim_cache size 0 max 16384 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 Processor -- start 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 -- start start 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdc74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541afb00 con 0x55deffdb2ca0 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541b1640 con 0x55deffdbb880 2024-04-20T11:23:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.772+0000 7fe85e02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541b01d0 con 0x55deffdb2380 2024-04-20T11:23:19.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdabb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49918/0 (socket says 172.21.15.44:49918) 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 -- 172.21.15.44:0/1318318758 learned_addr learned my addr 172.21.15.44:0/1318318758 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe858a00640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdc74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe859400640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 -- 172.21.15.44:0/1318318758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 msgr2=0x55deffdc74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdc74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 -- 172.21.15.44:0/1318318758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 msgr2=0x55deffdb91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe853e00640 1 -- 172.21.15.44:0/1318318758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deffdce1e0 con 0x55deffdb2ca0 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe859400640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.773+0000 7fe858a00640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.774+0000 7fe853e00640 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdabb10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe8400d4540 tx=0x7fe8400d47a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99d3c7f9d6bc896 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.774+0000 7fe845200640 1 -- 172.21.15.44:0/1318318758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84009c6b0 con 0x55deffdb2ca0 2024-04-20T11:23:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.774+0000 7fe845200640 1 -- 172.21.15.44:0/1318318758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe84009cb70 con 0x55deffdb2ca0 2024-04-20T11:23:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.774+0000 7fe845200640 1 -- 172.21.15.44:0/1318318758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84000ce60 con 0x55deffdb2ca0 2024-04-20T11:23:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 -- 172.21.15.44:0/1318318758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 msgr2=0x55deffdabb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdabb10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe8400d4540 tx=0x7fe8400d47a0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 -- 172.21.15.44:0/1318318758 shutdown_connections 2024-04-20T11:23:19.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdbb880 0x55deffdb91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2ca0 0x55deffdabb10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 --2- 172.21.15.44:0/1318318758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdb2380 0x55deffdc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 -- 172.21.15.44:0/1318318758 >> 172.21.15.44:0/1318318758 conn(0x55deffdacab0 msgr2=0x55deffdb7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 -- 172.21.15.44:0/1318318758 shutdown_connections 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 -- 172.21.15.44:0/1318318758 wait complete. 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.775+0000 7fe85e02c740 1 -- start start 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 Processor -- start 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 0x55deffdb5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdb2ca0 0x55deffdb5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdbb880 0x55deffdb60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541b0f70 con 0x55deffdb2380 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541b08a0 con 0x55deffdb2ca0 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541adb30 con 0x55deffdbb880 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe85e02c740 10 client.0 _ll_register_callbacks cb 0x7fe8400d4670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:19.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe858a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 0x55deffdb5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe858a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 0x55deffdb5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49934/0 (socket says 172.21.15.44:49934) 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe858a00640 1 -- 172.21.15.44:0/2349128373 learned_addr learned my addr 172.21.15.44:0/2349128373 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.776+0000 7fe853e00640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdb2ca0 0x55deffdb5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe859400640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdbb880 0x55deffdb60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe858a00640 1 -- 172.21.15.44:0/2349128373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdbb880 msgr2=0x55deffdb60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe858a00640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdbb880 0x55deffdb60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe858a00640 1 -- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdb2ca0 msgr2=0x55deffdb5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe858a00640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdb2ca0 0x55deffdb5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe858a00640 1 -- 172.21.15.44:0/2349128373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deffe30d90 con 0x55deffdb2380 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe858a00640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 0x55deffdb5670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe848012a70 tx=0x7fe848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b796f5a553a687 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:19.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe83a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe83a000640 1 -- 172.21.15.44:0/2349128373 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84802b850 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe83a000640 1 -- 172.21.15.44:0/2349128373 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe848003970 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.777+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84009cb70 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe83a000640 1 -- 172.21.15.44:0/2349128373 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe848003b70 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 10 client.15564 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55deffe320e0 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe83a000640 1 -- 172.21.15.44:0/2349128373 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe84009cb70 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe83a000640 1 -- 172.21.15.44:0/2349128373 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fe84803c050 con 0x55deffdb2380 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 20 client.15564 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffe32870 0x55deffe34c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 4 client.15564 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.778+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55deffd1f9e0 con 0x55deffe32870 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.779+0000 7fe853e00640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffe32870 0x55deffe34c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.779+0000 7fe853e00640 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffe32870 0x55deffe34c90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1c04f696a75714 server_cookie=5e060b7845a2339 in_seq=0 out_seq=0 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.779+0000 7fe83a000640 10 client.15564 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.779+0000 7fe83a000640 1 -- 172.21.15.44:0/2349128373 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55deffd1f9e0 con 0x55deffe32870 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.779+0000 7fe83a000640 10 client.15564 handle_command_reply: tid=0 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.779+0000 7fe85e02c740 1 client.15564 shutdown 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 msgr2=0x55deffdb5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 0x55deffdb5670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe848012a70 tx=0x7fe848004a40 comp rx=0 tx=0).stop 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 shutdown_connections 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55deffe32870 0x55deffe34c90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deffdbb880 0x55deffdb60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deffdb2ca0 0x55deffdb5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe83a000640 0 client.15564 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 --2- 172.21.15.44:0/2349128373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deffdb2380 0x55deffdb5670 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 >> 172.21.15.44:0/2349128373 conn(0x55deffdacab0 msgr2=0x55deffdabb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 shutdown_connections 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 1 -- 172.21.15.44:0/2349128373 wait complete. 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.780+0000 7fe85e02c740 20 client.15564 trim_cache size 0 max 16384 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout: "13b83344-ac73-4f5a-9822-d0e8b40a59ad": { 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "13b83344-ac73-4f5a-9822-d0e8b40a59ad", 2024-04-20T11:23:19.783 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:23:19.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:23:19.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:23:19.784 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:23:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe83c0a3ed0 msgr2=0x7fe83c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe83c0a3ed0 0x7fe83c0a6390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55deffdb1e50 tx=0x7fe848009500 comp rx=0 tx=0).stop 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854109340 msgr2=0x7fe8541a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854109340 0x7fe8541a0bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e9c0 tx=0x7fe84000ee80 comp rx=0 tx=0).stop 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 shutdown_connections 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe85419a770 0x7fe85419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe854109340 0x7fe8541a0bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe83c0a3ed0 0x7fe83c0a6390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 --2- 172.21.15.44:0/649420884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe854104270 0x7fe8541a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 >> 172.21.15.44:0/649420884 conn(0x7fe8540ffa70 msgr2=0x7fe854100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 shutdown_connections 2024-04-20T11:23:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:19.781+0000 7fe859e00640 1 -- 172.21.15.44:0/649420884 wait complete. 2024-04-20T11:23:19.798 INFO:tasks.cephfs.filesystem:scrub status for tag:13b83344-ac73-4f5a-9822-d0e8b40a59ad - {'path': '/', 'tag': '13b83344-ac73-4f5a-9822-d0e8b40a59ad', 'options': 'recursive,force'} 2024-04-20T11:23:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:19 smithi078 ceph-mon[21464]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-20T11:23:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:19 smithi044 ceph-mon[16855]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-20T11:23:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:19 smithi165 ceph-mon[21975]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-20T11:23:20.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:20 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:23:20.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:20 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:20.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:20 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:23:20.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:20 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:20.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:20 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:23:20.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:20 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:20.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:20 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:23:20.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:20 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:20 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:23:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:20 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:20 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:23:20.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:20 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:21 smithi078 ceph-mon[21464]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:23:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:21 smithi044 ceph-mon[16855]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:23:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:21 smithi165 ceph-mon[21975]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:23:22.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:22 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:23:22.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:22 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:23:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:22 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:23:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:23 smithi078 ceph-mon[21464]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-20T11:23:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:23 smithi044 ceph-mon[16855]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-20T11:23:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:23 smithi165 ceph-mon[21975]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-20T11:23:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:25 smithi078 ceph-mon[21464]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:23:25.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:25 smithi044 ceph-mon[16855]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:23:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:25 smithi165 ceph-mon[21975]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:23:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:27 smithi078 ceph-mon[21464]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:23:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:27 smithi044 ceph-mon[16855]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:23:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:27 smithi165 ceph-mon[21975]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:23:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:29 smithi078 ceph-mon[21464]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-20T11:23:29.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:29 smithi044 ceph-mon[16855]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-20T11:23:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:29 smithi165 ceph-mon[21975]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-20T11:23:31.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:31 smithi078 ceph-mon[21464]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-20T11:23:31.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:31 smithi044 ceph-mon[16855]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-20T11:23:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:31 smithi165 ceph-mon[21975]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-20T11:23:33.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:33 smithi078 ceph-mon[21464]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-20T11:23:33.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:33 smithi044 ceph-mon[16855]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-20T11:23:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:33 smithi165 ceph-mon[21975]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-20T11:23:35.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:35 smithi078 ceph-mon[21464]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-20T11:23:35.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:35 smithi044 ceph-mon[16855]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-20T11:23:35.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:35 smithi165 ceph-mon[21975]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-20T11:23:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:37 smithi078 ceph-mon[21464]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-20T11:23:37.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:37 smithi044 ceph-mon[16855]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-20T11:23:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:37 smithi165 ceph-mon[21975]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-20T11:23:38.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:38 smithi078 ceph-mon[21464]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-20T11:23:38.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:38 smithi044 ceph-mon[16855]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-20T11:23:38.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:38 smithi165 ceph-mon[21975]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-20T11:23:41.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:41 smithi165 ceph-mon[21975]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:41 smithi078 ceph-mon[21464]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:41.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:41 smithi044 ceph-mon[16855]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:43 smithi165 ceph-mon[21975]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-20T11:23:43.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:43 smithi078 ceph-mon[21464]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-20T11:23:43.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:43 smithi044 ceph-mon[16855]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-20T11:23:45.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:45 smithi165 ceph-mon[21975]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-20T11:23:45.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:45 smithi078 ceph-mon[21464]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-20T11:23:45.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:45 smithi044 ceph-mon[16855]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-20T11:23:46.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:46 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:23:46.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:46 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:23:46.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:46 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:23:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:47 smithi165 ceph-mon[21975]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:47 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:23:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:47 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:23:47.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:47 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:23:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:47 smithi078 ceph-mon[21464]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:47 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:23:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:47 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:23:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:47 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:23:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:47 smithi044 ceph-mon[16855]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:47 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:23:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:47 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:23:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:47 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:23:49.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:49 smithi165 ceph-mon[21975]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:23:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:49 smithi078 ceph-mon[21464]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:23:49.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:49 smithi044 ceph-mon[16855]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:23:49.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:23:49.799 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 -- 172.21.15.44:0/661716979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 msgr2=0x7f34e81150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 --2- 172.21.15.44:0/661716979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81150a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009c10 tx=0x7f34e002bef0 comp rx=0 tx=0).stop 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 -- 172.21.15.44:0/661716979 shutdown_connections 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 --2- 172.21.15.44:0/661716979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81150a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 --2- 172.21.15.44:0/661716979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e8076db0 0x7f34e8112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 --2- 172.21.15.44:0/661716979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 0x7f34e8076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 -- 172.21.15.44:0/661716979 >> 172.21.15.44:0/661716979 conn(0x7f34e80ffaa0 msgr2=0x7f34e8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 -- 172.21.15.44:0/661716979 shutdown_connections 2024-04-20T11:23:49.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.986+0000 7f34eca00640 1 -- 172.21.15.44:0/661716979 wait complete. 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.987+0000 7f34eca00640 1 Processor -- start 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.987+0000 7f34eca00640 1 -- start start 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.987+0000 7f34eca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 0x7f34e819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34eca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e8076db0 0x7f34e819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34eca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34eca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8118980 con 0x7f34e8076490 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34eca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8118800 con 0x7f34e8112cb0 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34eca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8118b00 con 0x7f34e8076db0 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 0x7f34e819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55354/0 (socket says 172.21.15.44:55354) 2024-04-20T11:23:49.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e7200640 1 -- 172.21.15.44:0/2783890113 learned_addr learned my addr 172.21.15.44:0/2783890113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e8076db0 0x7f34e819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e7200640 1 -- 172.21.15.44:0/2783890113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e8076db0 msgr2=0x7f34e819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e7200640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e8076db0 0x7f34e819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.988+0000 7f34e7200640 1 -- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 msgr2=0x7f34e819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:49.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34e7200640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 0x7f34e819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:49.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34e7200640 1 -- 172.21.15.44:0/2783890113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e81a3db0 con 0x7f34e8112cb0 2024-04-20T11:23:49.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34e6800640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 0x7f34e819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34e7200640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34e002c430 tx=0x7f34e0033720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34df400640 1 -- 172.21.15.44:0/2783890113 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0043020 con 0x7f34e8112cb0 2024-04-20T11:23:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34df400640 1 -- 172.21.15.44:0/2783890113 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f34e0004040 con 0x7f34e8112cb0 2024-04-20T11:23:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34df400640 1 -- 172.21.15.44:0/2783890113 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e003e6b0 con 0x7f34e8112cb0 2024-04-20T11:23:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.991+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e81a4040 con 0x7f34e8112cb0 2024-04-20T11:23:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.992+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81ab930 con 0x7f34e8112cb0 2024-04-20T11:23:49.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.995+0000 7f34df400640 1 -- 172.21.15.44:0/2783890113 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f34e0046020 con 0x7f34e8112cb0 2024-04-20T11:23:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.996+0000 7f34f01ee740 1 Processor -- start 2024-04-20T11:23:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 -- start start 2024-04-20T11:23:49.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34df400640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34cc0a3f50 0x7f34cc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34df400640 1 -- 172.21.15.44:0/2783890113 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f34e00f2eb0 con 0x7f34e8112cb0 2024-04-20T11:23:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bc8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bc8880 0x5594a1bcac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5594a1b25150 con 0x5594a1bc8880 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594a1acb370 con 0x5594a1bbfca0 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34f01ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5594a1b21110 con 0x5594a1bbf380 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.997+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34cc0a3f50 0x7f34cc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bc8880 0x5594a1bcac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bbf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35582/0 (socket says 172.21.15.44:35582) 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bc8880 0x5594a1bcac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39312/0 (socket says 172.21.15.44:39312) 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 -- 172.21.15.44:0/2880913804 learned_addr learned my addr 172.21.15.44:0/2880913804 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bc8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 -- 172.21.15.44:0/2880913804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 msgr2=0x5594a1bc8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bc8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 -- 172.21.15.44:0/2880913804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bc8880 msgr2=0x5594a1bcac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bc8880 0x5594a1bcac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34e6800640 1 -- 172.21.15.44:0/2880913804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594a19017d0 con 0x5594a1bbf380 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.998+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34cc0a3f50 0x7f34cc0a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f34e8103500 tx=0x7f34d0015850 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.999+0000 7f34e6800640 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bbf760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009cf0 tx=0x7f34d80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b82845f9da3ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.999+0000 7f34d7400640 1 -- 172.21.15.44:0/2880913804 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8009790 con 0x5594a1bbf380 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.999+0000 7f34d7400640 1 -- 172.21.15.44:0/2880913804 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f34d8012950 con 0x5594a1bbf380 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:49.999+0000 7f34d7400640 1 -- 172.21.15.44:0/2880913804 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800fc40 con 0x5594a1bbf380 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 -- 172.21.15.44:0/2880913804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 msgr2=0x5594a1bbf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bbf760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009cf0 tx=0x7f34d80117b0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 -- 172.21.15.44:0/2880913804 shutdown_connections 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bc8880 0x5594a1bcac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bc8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2880913804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bbf760 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 -- 172.21.15.44:0/2880913804 >> 172.21.15.44:0/2880913804 conn(0x5594a1bb9ab0 msgr2=0x5594a1bb9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 -- 172.21.15.44:0/2880913804 shutdown_connections 2024-04-20T11:23:50.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.000+0000 7f34f01ee740 1 -- 172.21.15.44:0/2880913804 wait complete. 2024-04-20T11:23:50.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.001+0000 7f34f01ee740 1 -- start start 2024-04-20T11:23:50.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.001+0000 7f34f01ee740 1 Processor -- start 2024-04-20T11:23:50.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbfca0 0x5594a1bc5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5594a1bcb670 con 0x5594a1bbf380 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594a1ade740 con 0x5594a1bc8880 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5594a1bcb7f0 con 0x5594a1bbfca0 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34f01ee740 10 client.0 _ll_register_callbacks cb 0x5594a1ad9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.002+0000 7f34e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39316/0 (socket says 172.21.15.44:39316) 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 -- 172.21.15.44:0/43540111 learned_addr learned my addr 172.21.15.44:0/43540111 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34dfe00640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbfca0 0x5594a1bc5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e7200640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 -- 172.21.15.44:0/43540111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbfca0 msgr2=0x5594a1bc5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbfca0 0x5594a1bc5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 -- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 msgr2=0x5594a1bc62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 -- 172.21.15.44:0/43540111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594a18373f0 con 0x5594a1bbf380 2024-04-20T11:23:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e7200640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.003+0000 7f34e6800640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc5820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f34d800f820 tx=0x7f34d80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37c741f116939da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34cbe00640 1 -- 172.21.15.44:0/43540111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8028070 con 0x5594a1bbf380 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d8012950 con 0x5594a1bbf380 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34cbe00640 1 -- 172.21.15.44:0/43540111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f34d8012450 con 0x5594a1bbf380 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34cbe00640 1 -- 172.21.15.44:0/43540111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34d8034050 con 0x5594a1bbf380 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34f01ee740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.004+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594a17f70f0 con 0x5594a1bbf380 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34cbe00640 1 -- 172.21.15.44:0/43540111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f34d8012950 con 0x5594a1bbf380 2024-04-20T11:23:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34cbe00640 1 -- 172.21.15.44:0/43540111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f34d80035f0 con 0x5594a1bbf380 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34f01ee740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34f01ee740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34f01ee740 20 client.15576 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34f01ee740 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1bd8b00 0x5594a1bdaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34f01ee740 4 client.15576 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.005+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594a1b32b70 con 0x5594a1bd8b00 2024-04-20T11:23:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.006+0000 7f34dfe00640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1bd8b00 0x5594a1bdaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.006+0000 7f34dfe00640 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1bd8b00 0x5594a1bdaef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aab0fa07e72655f server_cookie=476e212100cac668 in_seq=0 out_seq=0 2024-04-20T11:23:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.006+0000 7f34cbe00640 10 client.15576 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34cbe00640 1 -- 172.21.15.44:0/43540111 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594a1b32b70 con 0x5594a1bd8b00 2024-04-20T11:23:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34cbe00640 10 client.15576 handle_command_reply: tid=0 2024-04-20T11:23:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 client.15576 shutdown 2024-04-20T11:23:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 msgr2=0x5594a1bc5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc5820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f34d800f820 tx=0x7f34d80056d0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 shutdown_connections 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1bd8b00 0x5594a1bdaef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbfca0 0x5594a1bc5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 --2- 172.21.15.44:0/43540111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc5820 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 >> 172.21.15.44:0/43540111 conn(0x5594a1bb9ab0 msgr2=0x5594a1bbb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.007+0000 7f34cbe00640 0 client.15576 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.008+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 shutdown_connections 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.008+0000 7f34f01ee740 1 -- 172.21.15.44:0/43540111 wait complete. 2024-04-20T11:23:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.008+0000 7f34f01ee740 20 client.15576 trim_cache size 0 max 16384 2024-04-20T11:23:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.026+0000 7f34f01ee740 1 Processor -- start 2024-04-20T11:23:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 -- start start 2024-04-20T11:23:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bd4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bc8880 0x5594a1be05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b3fb0 con 0x5594a1bbf380 2024-04-20T11:23:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b5af0 con 0x5594a1bbfca0 2024-04-20T11:23:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34f01ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b4680 con 0x5594a1bc8880 2024-04-20T11:23:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.027+0000 7f34e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39324/0 (socket says 172.21.15.44:39324) 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 -- 172.21.15.44:0/1670083706 learned_addr learned my addr 172.21.15.44:0/1670083706 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34dfe00640 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bd4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e7200640 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bc8880 0x5594a1be05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 -- 172.21.15.44:0/1670083706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bc8880 msgr2=0x5594a1be05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bc8880 0x5594a1be05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 -- 172.21.15.44:0/1670083706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 msgr2=0x5594a1bd4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bd4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.028+0000 7f34e6800640 1 -- 172.21.15.44:0/1670083706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594a1bbace0 con 0x5594a1bbf380 2024-04-20T11:23:50.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.029+0000 7f34e6800640 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc7f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005a40 tx=0x7f34d80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204d14ed47fdf7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.029+0000 7f34d6a00640 1 -- 172.21.15.44:0/1670083706 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8028040 con 0x5594a1bbf380 2024-04-20T11:23:50.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.029+0000 7f34d6a00640 1 -- 172.21.15.44:0/1670083706 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f34d8009540 con 0x5594a1bbf380 2024-04-20T11:23:50.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.029+0000 7f34d6a00640 1 -- 172.21.15.44:0/1670083706 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8003f60 con 0x5594a1bbf380 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 -- 172.21.15.44:0/1670083706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 msgr2=0x5594a1bc7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc7f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005a40 tx=0x7f34d80117b0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 -- 172.21.15.44:0/1670083706 shutdown_connections 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bc8880 0x5594a1be05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bbfca0 0x5594a1bd4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 --2- 172.21.15.44:0/1670083706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbf380 0x5594a1bc7f50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 -- 172.21.15.44:0/1670083706 >> 172.21.15.44:0/1670083706 conn(0x5594a1bb9ab0 msgr2=0x5594a1bbb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 -- 172.21.15.44:0/1670083706 shutdown_connections 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.030+0000 7f34f01ee740 1 -- 172.21.15.44:0/1670083706 wait complete. 2024-04-20T11:23:50.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.031+0000 7f34f01ee740 1 -- start start 2024-04-20T11:23:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.031+0000 7f34f01ee740 1 Processor -- start 2024-04-20T11:23:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.031+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bc2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.031+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbfca0 0x5594a1bc2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34f01ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34f01ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b5420 con 0x5594a1bbfca0 2024-04-20T11:23:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34f01ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b4d50 con 0x5594a1bc8880 2024-04-20T11:23:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34f01ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81b1fa0 con 0x5594a1bbf380 2024-04-20T11:23:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34f01ee740 10 client.0 _ll_register_callbacks cb 0x7f34e0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bc2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bc2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35602/0 (socket says 172.21.15.44:35602) 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 -- 172.21.15.44:0/2794281806 learned_addr learned my addr 172.21.15.44:0/2794281806 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbfca0 0x5594a1bc2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 -- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 msgr2=0x5594a1bc2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 -- 172.21.15.44:0/2794281806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbfca0 msgr2=0x5594a1bc2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbfca0 0x5594a1bc2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.032+0000 7f34e6800640 1 -- 172.21.15.44:0/2794281806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594a1c3de10 con 0x5594a1bbf380 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.033+0000 7f34e6800640 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bc2330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009360 tx=0x7f34d8009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cb19d554edaec5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.033+0000 7f34cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.034+0000 7f34cbe00640 1 -- 172.21.15.44:0/2794281806 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8045060 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.034+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d8009540 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.034+0000 7f34cbe00640 1 -- 172.21.15.44:0/2794281806 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f34d8012650 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.034+0000 7f34cbe00640 1 -- 172.21.15.44:0/2794281806 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34d8028040 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.034+0000 7f34f01ee740 10 client.25705 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.034+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5594a1c3ef60 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.035+0000 7f34cbe00640 1 -- 172.21.15.44:0/2794281806 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f34d8009540 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.035+0000 7f34cbe00640 1 -- 172.21.15.44:0/2794281806 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f34d8048020 con 0x5594a1bbf380 2024-04-20T11:23:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.035+0000 7f34f01ee740 10 client.25705 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.035+0000 7f34f01ee740 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.035+0000 7f34f01ee740 20 client.25705 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.035+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1c3f510 0x5594a1c41900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.036+0000 7f34f01ee740 4 client.25705 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:23:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.036+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594a1b2c9e0 con 0x5594a1c3f510 2024-04-20T11:23:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.036+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1c3f510 0x5594a1c41900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34dfe00640 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1c3f510 0x5594a1c41900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55954ce0a38078c7 server_cookie=909bb36979d338df in_seq=0 out_seq=0 2024-04-20T11:23:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34cbe00640 10 client.25705 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34cbe00640 1 -- 172.21.15.44:0/2794281806 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594a1b2c9e0 con 0x5594a1c3f510 2024-04-20T11:23:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34cbe00640 10 client.25705 handle_command_reply: tid=0 2024-04-20T11:23:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 client.25705 shutdown 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 msgr2=0x5594a1bc2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bc2330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009360 tx=0x7f34d8009390 comp rx=0 tx=0).stop 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 shutdown_connections 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5594a1c3f510 0x5594a1c41900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594a1bc8880 0x5594a1bc2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5594a1bbfca0 0x5594a1bc2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 --2- 172.21.15.44:0/2794281806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594a1bbf380 0x5594a1bc2330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 >> 172.21.15.44:0/2794281806 conn(0x5594a1bb9ab0 msgr2=0x5594a1bc57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34cbe00640 0 client.25705 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.037+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 shutdown_connections 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.038+0000 7f34f01ee740 1 -- 172.21.15.44:0/2794281806 wait complete. 2024-04-20T11:23:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.038+0000 7f34f01ee740 20 client.25705 trim_cache size 0 max 16384 2024-04-20T11:23:50.040 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:23:50.040 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:23:50.040 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:23:50.040 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.038+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34cc0a3f50 msgr2=0x7f34cc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.038+0000 7f34eca00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34cc0a3f50 0x7f34cc0a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f34e8103500 tx=0x7f34d0015850 comp rx=0 tx=0).stop 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.038+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 msgr2=0x7f34e81a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.038+0000 7f34eca00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34e002c430 tx=0x7f34e0033720 comp rx=0 tx=0).stop 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 shutdown_connections 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8112cb0 0x7f34e81a3680 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e8076db0 0x7f34e819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34cc0a3f50 0x7f34cc0a6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 --2- 172.21.15.44:0/2783890113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34e8076490 0x7f34e819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 >> 172.21.15.44:0/2783890113 conn(0x7f34e80ffaa0 msgr2=0x7f34e8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 shutdown_connections 2024-04-20T11:23:50.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.039+0000 7f34eca00640 1 -- 172.21.15.44:0/2783890113 wait complete. 2024-04-20T11:23:50.065 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:23:50.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:23:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 -- 172.21.15.44:0/636425019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941063b0 msgr2=0x7f4d94106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 --2- 172.21.15.44:0/636425019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941063b0 0x7f4d94106860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009c10 tx=0x7f4d8c02bf80 comp rx=0 tx=0).stop 2024-04-20T11:23:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 -- 172.21.15.44:0/636425019 shutdown_connections 2024-04-20T11:23:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 --2- 172.21.15.44:0/636425019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941063b0 0x7f4d94106860 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 --2- 172.21.15.44:0/636425019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941018d0 0x7f4d94105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 --2- 172.21.15.44:0/636425019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d94101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.265+0000 7f4d9b400640 1 -- 172.21.15.44:0/636425019 >> 172.21.15.44:0/636425019 conn(0x7f4d940771e0 msgr2=0x7f4d940775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.266+0000 7f4d9b400640 1 -- 172.21.15.44:0/636425019 shutdown_connections 2024-04-20T11:23:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.266+0000 7f4d9b400640 1 -- 172.21.15.44:0/636425019 wait complete. 2024-04-20T11:23:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.267+0000 7f4d9b400640 1 Processor -- start 2024-04-20T11:23:50.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.267+0000 7f4d9b400640 1 -- start start 2024-04-20T11:23:50.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d9b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d9419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d9b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941018d0 0x7f4d9419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d9b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941063b0 0x7f4d941a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d9b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94118a90 con 0x7f4d941063b0 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d9b400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94118910 con 0x7f4d94100fb0 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d9b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94118c10 con 0x7f4d941018d0 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.268+0000 7f4d98a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d9419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d9419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55418/0 (socket says 172.21.15.44:55418) 2024-04-20T11:23:50.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 -- 172.21.15.44:0/1672610218 learned_addr learned my addr 172.21.15.44:0/1672610218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d93e00640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941018d0 0x7f4d9419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d99400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941063b0 0x7f4d941a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 -- 172.21.15.44:0/1672610218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941018d0 msgr2=0x7f4d9419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941018d0 0x7f4d9419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 -- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941063b0 msgr2=0x7f4d941a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941063b0 0x7f4d941a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d98a00640 1 -- 172.21.15.44:0/1672610218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d941a3f40 con 0x7f4d94100fb0 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.269+0000 7f4d99400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941063b0 0x7f4d941a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.270+0000 7f4d98a00640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d9419ef50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e9f0 tx=0x7f4d8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.270+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800ccc0 con 0x7f4d94100fb0 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.270+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4d88005520 con 0x7f4d94100fb0 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.270+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d941a4230 con 0x7f4d94100fb0 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.270+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88010690 con 0x7f4d94100fb0 2024-04-20T11:23:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.270+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d941abac0 con 0x7f4d94100fb0 2024-04-20T11:23:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.271+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9406ad80 con 0x7f4d94100fb0 2024-04-20T11:23:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.274+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f4d88004f70 con 0x7f4d94100fb0 2024-04-20T11:23:50.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.276+0000 7f4d91600640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4d7c0a3f50 0x7f4d7c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.276+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4d880d0f80 con 0x7f4d94100fb0 2024-04-20T11:23:50.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.276+0000 7f4d93e00640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4d7c0a3f50 0x7f4d7c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.277+0000 7f4d93e00640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4d7c0a3f50 0x7f4d7c0a6410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d94077a00 tx=0x7f4d80005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.277+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4d88053ff0 con 0x7f4d94100fb0 2024-04-20T11:23:50.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.525+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d940691d0 con 0x7f4d94100fb0 2024-04-20T11:23:50.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.526+0000 7f4d91600640 1 -- 172.21.15.44:0/1672610218 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f4d8809d450 con 0x7f4d94100fb0 2024-04-20T11:23:50.527 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:23:50.527 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:22:35.546711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:23:50.528 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-04-20T11:23:50.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4d7c0a3f50 msgr2=0x7f4d7c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4d7c0a3f50 0x7f4d7c0a6410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d94077a00 tx=0x7f4d80005f90 comp rx=0 tx=0).stop 2024-04-20T11:23:50.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 msgr2=0x7f4d9419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d9419ef50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e9f0 tx=0x7f4d8800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 shutdown_connections 2024-04-20T11:23:50.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4d7c0a3f50 0x7f4d7c0a6410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d941063b0 0x7f4d941a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d941018d0 0x7f4d9419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 --2- 172.21.15.44:0/1672610218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94100fb0 0x7f4d9419ef50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 >> 172.21.15.44:0/1672610218 conn(0x7f4d940771e0 msgr2=0x7f4d94103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.530+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 shutdown_connections 2024-04-20T11:23:50.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.531+0000 7f4d9b400640 1 -- 172.21.15.44:0/1672610218 wait complete. 2024-04-20T11:23:50.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 -- 172.21.15.44:0/925347931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102a20 msgr2=0x7fa7ec106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/925347931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102a20 0x7fa7ec106a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009900 tx=0x7fa7d802bc50 comp rx=0 tx=0).stop 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 -- 172.21.15.44:0/925347931 shutdown_connections 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/925347931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec107140 0x7fa7ec10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/925347931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102a20 0x7fa7ec106a60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/925347931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7ec102070 0x7fa7ec102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 -- 172.21.15.44:0/925347931 >> 172.21.15.44:0/925347931 conn(0x7fa7ec0fd930 msgr2=0x7fa7ec0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 -- 172.21.15.44:0/925347931 shutdown_connections 2024-04-20T11:23:50.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.742+0000 7fa7f2a00640 1 -- 172.21.15.44:0/925347931 wait complete. 2024-04-20T11:23:50.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.743+0000 7fa7f2a00640 1 Processor -- start 2024-04-20T11:23:50.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 -- start start 2024-04-20T11:23:50.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 0x7fa7ec1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7ec107140 0x7fa7ec1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec19a7f0 0x7fa7ec19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1122f0 con 0x7fa7ec102070 2024-04-20T11:23:50.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112170 con 0x7fa7ec19a7f0 2024-04-20T11:23:50.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f2a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112470 con 0x7fa7ec107140 2024-04-20T11:23:50.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.744+0000 7fa7f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 0x7fa7ec1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 0x7fa7ec1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39360/0 (socket says 172.21.15.44:39360) 2024-04-20T11:23:50.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 -- 172.21.15.44:0/2674980521 learned_addr learned my addr 172.21.15.44:0/2674980521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7ec107140 0x7fa7ec1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f2000640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec19a7f0 0x7fa7ec19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 -- 172.21.15.44:0/2674980521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7ec107140 msgr2=0x7fa7ec1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7ec107140 0x7fa7ec1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 -- 172.21.15.44:0/2674980521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec19a7f0 msgr2=0x7fa7ec19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec19a7f0 0x7fa7ec19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 -- 172.21.15.44:0/2674980521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec19b380 con 0x7fa7ec102070 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f2000640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec19a7f0 0x7fa7ec19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.745+0000 7fa7f1600640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 0x7fa7ec1a0680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000d950 tx=0x7fa7e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.746+0000 7fa7ea000640 1 -- 172.21.15.44:0/2674980521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0017030 con 0x7fa7ec102070 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.746+0000 7fa7ea000640 1 -- 172.21.15.44:0/2674980521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7e0004020 con 0x7fa7ec102070 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.746+0000 7fa7ea000640 1 -- 172.21.15.44:0/2674980521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0004a00 con 0x7fa7ec102070 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.746+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec19b670 con 0x7fa7ec102070 2024-04-20T11:23:50.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.746+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec1a7870 con 0x7fa7ec102070 2024-04-20T11:23:50.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.750+0000 7fa7f6dc8740 1 Processor -- start 2024-04-20T11:23:50.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.750+0000 7fa7f6dc8740 1 -- start start 2024-04-20T11:23:50.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.751+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x55751440c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.751+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514415130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.751+0000 7fa7f2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x55751440c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.751+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514417a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557514371050 con 0x55751440ca90 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557514317490 con 0x557514415670 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55751436d010 con 0x55751440c170 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x55751440c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35654/0 (socket says 172.21.15.44:35654) 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 -- 172.21.15.44:0/1798559163 learned_addr learned my addr 172.21.15.44:0/1798559163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514417a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 -- 172.21.15.44:0/1798559163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 msgr2=0x557514417a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514417a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 -- 172.21.15.44:0/1798559163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 msgr2=0x557514415130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:23:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514415130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7f2000640 1 -- 172.21.15.44:0/1798559163 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751431f070 con 0x55751440c170 2024-04-20T11:23:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.752+0000 7fa7ea000640 1 -- 172.21.15.44:0/2674980521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fa7e00041d0 con 0x7fa7ec102070 2024-04-20T11:23:50.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.753+0000 7fa7f2000640 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x55751440c550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009ee0 tx=0x7fa7e400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50fda61635509e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.753+0000 7fa7de600640 1 -- 172.21.15.44:0/1798559163 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4011640 con 0x55751440c170 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.753+0000 7fa7de600640 1 -- 172.21.15.44:0/1798559163 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7e4011ce0 con 0x55751440c170 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.753+0000 7fa7de600640 1 -- 172.21.15.44:0/1798559163 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40103d0 con 0x55751440c170 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1798559163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 msgr2=0x55751440c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x55751440c550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009ee0 tx=0x7fa7e400f730 comp rx=0 tx=0).stop 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1798559163 shutdown_connections 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514417a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514415130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1798559163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x55751440c550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1798559163 >> 172.21.15.44:0/1798559163 conn(0x5575144068a0 msgr2=0x557514406c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1798559163 shutdown_connections 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.754+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1798559163 wait complete. 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.755+0000 7fa7ea000640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa7d40a3ee0 0x7fa7d40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.755+0000 7fa7f6dc8740 1 -- start start 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.755+0000 7fa7ea000640 1 -- 172.21.15.44:0/2674980521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa7e00d0430 con 0x7fa7ec102070 2024-04-20T11:23:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.755+0000 7fa7f6dc8740 1 Processor -- start 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.755+0000 7fa7f2000640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa7d40a3ee0 0x7fa7d40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.756+0000 7fa7f2000640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa7d40a3ee0 0x7fa7d40a63a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009d20 tx=0x7fa7e401ba40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.756+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x5575144127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.756+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514412d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.756+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514413270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557514418460 con 0x55751440ca90 2024-04-20T11:23:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55751432a860 con 0x557514415670 2024-04-20T11:23:50.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5575144185e0 con 0x55751440c170 2024-04-20T11:23:50.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f6dc8740 10 client.0 _ll_register_callbacks cb 0x5575142f8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514412d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514412d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39368/0 (socket says 172.21.15.44:39368) 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f0c00640 1 -- 172.21.15.44:0/2768946222 learned_addr learned my addr 172.21.15.44:0/2768946222 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f2000640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514413270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.757+0000 7fa7f1600640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x5575144127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7f0c00640 1 -- 172.21.15.44:0/2768946222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 msgr2=0x5575144127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x5575144127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7f0c00640 1 -- 172.21.15.44:0/2768946222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 msgr2=0x557514413270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514413270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7f0c00640 1 -- 172.21.15.44:0/2768946222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751432e950 con 0x55751440ca90 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514412d30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8005ec0 tx=0x7fa7d802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38ee7132790dd93 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:50.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.758+0000 7fa7d2a00640 1 -- 172.21.15.44:0/2768946222 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8043070 con 0x55751440ca90 2024-04-20T11:23:50.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e4011ce0 con 0x55751440ca90 2024-04-20T11:23:50.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7d2a00640 1 -- 172.21.15.44:0/2768946222 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa7d8004250 con 0x55751440ca90 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7d2a00640 1 -- 172.21.15.44:0/2768946222 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7d804f050 con 0x55751440ca90 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7f6dc8740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575142fc820 con 0x55751440ca90 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7d2a00640 1 -- 172.21.15.44:0/2768946222 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7e4011ce0 con 0x55751440ca90 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.759+0000 7fa7d2a00640 1 -- 172.21.15.44:0/2768946222 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fa7d8004b30 con 0x55751440ca90 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f6dc8740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f6dc8740 10 client.15594 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f6dc8740 20 client.15594 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5575144254e0 0x5575144278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f6dc8740 4 client.15594 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55751437ea70 con 0x5575144254e0 2024-04-20T11:23:50.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.760+0000 7fa7f1600640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5575144254e0 0x5575144278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.761+0000 7fa7f1600640 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5575144254e0 0x5575144278d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39f295d962ce5c1c server_cookie=df7f5dc9c7d2f022 in_seq=0 out_seq=0 2024-04-20T11:23:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.761+0000 7fa7d2a00640 10 client.15594 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.761+0000 7fa7d2a00640 1 -- 172.21.15.44:0/2768946222 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55751437ea70 con 0x5575144254e0 2024-04-20T11:23:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.761+0000 7fa7d2a00640 10 client.15594 handle_command_reply: tid=0 2024-04-20T11:23:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 client.15594 shutdown 2024-04-20T11:23:50.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 msgr2=0x557514412d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514412d30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8005ec0 tx=0x7fa7d802cea0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 shutdown_connections 2024-04-20T11:23:50.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5575144254e0 0x5575144278d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557514415670 0x557514413270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x557514412d30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/2768946222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55751440c170 0x5575144127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 >> 172.21.15.44:0/2768946222 conn(0x5575144068a0 msgr2=0x5575144085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7d2a00640 0 client.15594 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 shutdown_connections 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.762+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/2768946222 wait complete. 2024-04-20T11:23:50.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.763+0000 7fa7f6dc8740 20 client.15594 trim_cache size 0 max 16384 2024-04-20T11:23:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.790+0000 7fa7f6dc8740 1 Processor -- start 2024-04-20T11:23:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.790+0000 7fa7f6dc8740 1 -- start start 2024-04-20T11:23:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 0x557514405960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440ca90 0x55751442d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751442d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1afd40 con 0x55751440c170 2024-04-20T11:23:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1b1880 con 0x55751440ca90 2024-04-20T11:23:50.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1b0410 con 0x557514415670 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 0x557514405960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 0x557514405960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39370/0 (socket says 172.21.15.44:39370) 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f1600640 1 -- 172.21.15.44:0/1511440574 learned_addr learned my addr 172.21.15.44:0/1511440574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440ca90 0x55751442d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.791+0000 7fa7f2000640 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751442d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f1600640 1 -- 172.21.15.44:0/1511440574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 msgr2=0x55751442d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f1600640 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751442d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f1600640 1 -- 172.21.15.44:0/1511440574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440ca90 msgr2=0x55751442d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f1600640 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440ca90 0x55751442d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f1600640 1 -- 172.21.15.44:0/1511440574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557514417dd0 con 0x55751440c170 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f2000640 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751442d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.792+0000 7fa7f1600640 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 0x557514405960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0098bd0 tx=0x7fa7e0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c08a7efe87aa9b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.793+0000 7fa7ddc00640 1 -- 172.21.15.44:0/1511440574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00a4e70 con 0x55751440c170 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.793+0000 7fa7ddc00640 1 -- 172.21.15.44:0/1511440574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7e0014040 con 0x55751440c170 2024-04-20T11:23:50.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.793+0000 7fa7ddc00640 1 -- 172.21.15.44:0/1511440574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00a0150 con 0x55751440c170 2024-04-20T11:23:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.793+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1511440574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 msgr2=0x557514405960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.793+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 0x557514405960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0098bd0 tx=0x7fa7e0005db0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1511440574 shutdown_connections 2024-04-20T11:23:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751442d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440ca90 0x55751442d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/1511440574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440c170 0x557514405960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1511440574 >> 172.21.15.44:0/1511440574 conn(0x5575144068a0 msgr2=0x557514413ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1511440574 shutdown_connections 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/1511440574 wait complete. 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 -- start start 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.794+0000 7fa7f6dc8740 1 Processor -- start 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 0x55751441d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x55751441dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751441a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1b11b0 con 0x55751440ca90 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1b0ae0 con 0x55751440c170 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1add10 con 0x557514415670 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f6dc8740 10 client.0 _ll_register_callbacks cb 0x7fa7d802cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 0x55751441d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 0x55751441d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55484/0 (socket says 172.21.15.44:55484) 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f1600640 1 -- 172.21.15.44:0/681700332 learned_addr learned my addr 172.21.15.44:0/681700332 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.795+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x55751441dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f2000640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751441a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f1600640 1 -- 172.21.15.44:0/681700332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 msgr2=0x55751441a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f1600640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751441a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f1600640 1 -- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 msgr2=0x55751441dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f1600640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x55751441dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f1600640 1 -- 172.21.15.44:0/681700332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557514427c40 con 0x55751440c170 2024-04-20T11:23:50.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f2000640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751441a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f1600640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 0x55751441d830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00a4940 tx=0x7fa7e009c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea3e30ef8932013 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7d2a00640 1 -- 172.21.15.44:0/681700332 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e007f070 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7d2a00640 1 -- 172.21.15.44:0/681700332 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa7e009a500 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.796+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e0014040 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7d2a00640 1 -- 172.21.15.44:0/681700332 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7e0098870 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 10 client.25724 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575140833f0 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7d2a00640 1 -- 172.21.15.44:0/681700332 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7e0014040 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7d2a00640 1 -- 172.21.15.44:0/681700332 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fa7e0082020 con 0x55751440c170 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 10 client.25724 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 10 client.25724 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 20 client.25724 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:50.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55751448c660 0x55751448ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 4 client.25724 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.797+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575143788e0 con 0x55751448c660 2024-04-20T11:23:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.798+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55751448c660 0x55751448ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.798+0000 7fa7f0c00640 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55751448c660 0x55751448ea80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b2311fdca5a9819 server_cookie=326e152068a273ec in_seq=0 out_seq=0 2024-04-20T11:23:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.798+0000 7fa7d2a00640 10 client.25724 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7d2a00640 1 -- 172.21.15.44:0/681700332 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575143788e0 con 0x55751448c660 2024-04-20T11:23:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7d2a00640 10 client.25724 handle_command_reply: tid=0 2024-04-20T11:23:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 client.25724 shutdown 2024-04-20T11:23:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 msgr2=0x55751441d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 0x55751441d830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00a4940 tx=0x7fa7e009c340 comp rx=0 tx=0).stop 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 shutdown_connections 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557514415670 0x55751441a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55751440ca90 0x55751441dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55751448c660 0x55751448ea80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 --2- 172.21.15.44:0/681700332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751440c170 0x55751441d830 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 >> 172.21.15.44:0/681700332 conn(0x5575144068a0 msgr2=0x5575144057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7d2a00640 0 client.25724 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 shutdown_connections 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 1 -- 172.21.15.44:0/681700332 wait complete. 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.799+0000 7fa7f6dc8740 20 client.25724 trim_cache size 0 max 16384 2024-04-20T11:23:50.803 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa7d40a3ee0 msgr2=0x7fa7d40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa7d40a3ee0 0x7fa7d40a63a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009d20 tx=0x7fa7e401ba40 comp rx=0 tx=0).stop 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 msgr2=0x7fa7ec1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 0x7fa7ec1a0680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000d950 tx=0x7fa7e000de10 comp rx=0 tx=0).stop 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 shutdown_connections 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7ec19a7f0 0x7fa7ec19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7ec107140 0x7fa7ec1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa7d40a3ee0 0x7fa7d40a63a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.800+0000 7fa7f2a00640 1 --2- 172.21.15.44:0/2674980521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7ec102070 0x7fa7ec1a0680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.801+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 >> 172.21.15.44:0/2674980521 conn(0x7fa7ec0fd930 msgr2=0x7fa7ec10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.801+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 shutdown_connections 2024-04-20T11:23:50.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.801+0000 7fa7f2a00640 1 -- 172.21.15.44:0/2674980521 wait complete. 2024-04-20T11:23:50.819 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:23:50.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.992+0000 7f8c64e00640 1 -- 172.21.15.44:0/417615213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60102070 msgr2=0x7f8c60102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.992+0000 7f8c64e00640 1 --2- 172.21.15.44:0/417615213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60102070 0x7f8c60102450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009860 tx=0x7f8c5002bb70 comp rx=0 tx=0).stop 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 -- 172.21.15.44:0/417615213 shutdown_connections 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 --2- 172.21.15.44:0/417615213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60107140 0x7f8c6010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 --2- 172.21.15.44:0/417615213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60102a20 0x7f8c60106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 --2- 172.21.15.44:0/417615213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60102070 0x7f8c60102450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 -- 172.21.15.44:0/417615213 >> 172.21.15.44:0/417615213 conn(0x7f8c600fd930 msgr2=0x7f8c600ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 -- 172.21.15.44:0/417615213 shutdown_connections 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.993+0000 7f8c64e00640 1 -- 172.21.15.44:0/417615213 wait complete. 2024-04-20T11:23:50.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.994+0000 7f8c64e00640 1 Processor -- start 2024-04-20T11:23:50.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 -- start start 2024-04-20T11:23:50.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 0x7f8c601a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60107140 0x7f8c601a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6010df10 0x7f8c6010e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:50.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60112330 con 0x7f8c60107140 2024-04-20T11:23:50.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601121b0 con 0x7f8c60102070 2024-04-20T11:23:50.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.995+0000 7f8c64e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601124b0 con 0x7f8c6010df10 2024-04-20T11:23:50.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60107140 0x7f8c601a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 0x7f8c601a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60107140 0x7f8c601a4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39386/0 (socket says 172.21.15.44:39386) 2024-04-20T11:23:50.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5ea00640 1 -- 172.21.15.44:0/1714845968 learned_addr learned my addr 172.21.15.44:0/1714845968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:50.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 0x7f8c601a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55502/0 (socket says 172.21.15.44:55502) 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6010df10 0x7f8c6010e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 -- 172.21.15.44:0/1714845968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6010df10 msgr2=0x7f8c6010e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6010df10 0x7f8c6010e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 -- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60107140 msgr2=0x7f8c601a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60107140 0x7f8c601a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.996+0000 7f8c5f400640 1 -- 172.21.15.44:0/1714845968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6010eac0 con 0x7f8c60102070 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.997+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 0x7f8c601a3b40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c5002c0b0 tx=0x7f8c500045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.997+0000 7f8c57e00640 1 -- 172.21.15.44:0/1714845968 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50041020 con 0x7f8c60102070 2024-04-20T11:23:50.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.997+0000 7f8c57e00640 1 -- 172.21.15.44:0/1714845968 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c5002ca10 con 0x7f8c60102070 2024-04-20T11:23:50.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.997+0000 7f8c57e00640 1 -- 172.21.15.44:0/1714845968 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50039040 con 0x7f8c60102070 2024-04-20T11:23:50.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.997+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6010ed50 con 0x7f8c60102070 2024-04-20T11:23:50.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:50.997+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601abc90 con 0x7f8c60102070 2024-04-20T11:23:51.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.001+0000 7f8c69048740 1 Processor -- start 2024-04-20T11:23:51.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.001+0000 7f8c69048740 1 -- start start 2024-04-20T11:23:51.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c69048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce56550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c69048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56a90 0x555c1ce5f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c69048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce61a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c69048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555c1cdbb050 con 0x555c1ce5f670 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c69048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555c1cd61490 con 0x555c1ce56170 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c69048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555c1cdb7010 con 0x555c1ce56a90 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce56550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce56550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55512/0 (socket says 172.21.15.44:55512) 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c5fe00640 1 -- 172.21.15.44:0/4053395170 learned_addr learned my addr 172.21.15.44:0/4053395170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.002+0000 7f8c57e00640 1 -- 172.21.15.44:0/1714845968 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f8c50044020 con 0x7f8c60102070 2024-04-20T11:23:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5fe00640 1 -- 172.21.15.44:0/4053395170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56a90 msgr2=0x555c1ce5f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5f400640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56a90 0x555c1ce5f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce61a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56a90 0x555c1ce5f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5fe00640 1 -- 172.21.15.44:0/4053395170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 msgr2=0x555c1ce61a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce61a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5fe00640 1 -- 172.21.15.44:0/4053395170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1cd69070 con 0x555c1ce56170 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5f400640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56a90 0x555c1ce5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce61a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.003+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce56550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009f40 tx=0x7f8c58010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41efd70cca8898e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.004+0000 7f8c54c00640 1 -- 172.21.15.44:0/4053395170 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58003b70 con 0x555c1ce56170 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.004+0000 7f8c54c00640 1 -- 172.21.15.44:0/4053395170 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c58003d10 con 0x555c1ce56170 2024-04-20T11:23:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.004+0000 7f8c54c00640 1 -- 172.21.15.44:0/4053395170 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58011b30 con 0x555c1ce56170 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 -- 172.21.15.44:0/4053395170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 msgr2=0x555c1ce56550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce56550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009f40 tx=0x7f8c58010860 comp rx=0 tx=0).stop 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c57e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c440a3f50 0x7f8c440a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 -- 172.21.15.44:0/4053395170 shutdown_connections 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce61a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56a90 0x555c1ce5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 --2- 172.21.15.44:0/4053395170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce56550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 -- 172.21.15.44:0/4053395170 >> 172.21.15.44:0/4053395170 conn(0x555c1ce508a0 msgr2=0x555c1ce50c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 -- 172.21.15.44:0/4053395170 shutdown_connections 2024-04-20T11:23:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c57e00640 1 -- 172.21.15.44:0/1714845968 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8c500f1ff0 con 0x7f8c60102070 2024-04-20T11:23:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c69048740 1 -- 172.21.15.44:0/4053395170 wait complete. 2024-04-20T11:23:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.005+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c440a3f50 0x7f8c440a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.006+0000 7f8c69048740 1 -- start start 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.006+0000 7f8c69048740 1 Processor -- start 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c440a3f50 0x7f8c440a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c580117d0 tx=0x7f8c58011630 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce5cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce5f670 0x555c1ce5d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555c1ce62460 con 0x555c1ce56a90 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555c1cd74860 con 0x555c1ce5f670 2024-04-20T11:23:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555c1ce625e0 con 0x555c1ce56170 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c69048740 10 client.0 _ll_register_callbacks cb 0x555c1cd42fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce5cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.007+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce5cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39414/0 (socket says 172.21.15.44:39414) 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5ea00640 1 -- 172.21.15.44:0/262861087 learned_addr learned my addr 172.21.15.44:0/262861087 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce5f670 0x555c1ce5d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 -- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce5f670 msgr2=0x555c1ce5d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce5f670 0x555c1ce5d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 -- 172.21.15.44:0/262861087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 msgr2=0x555c1ce5cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce5cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 -- 172.21.15.44:0/262861087 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1cd78950 con 0x555c1ce56170 2024-04-20T11:23:51.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c5f400640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5c670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ef680 tx=0x7f8c500bdd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c8b7596bd97e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c49600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.008+0000 7f8c49600640 1 -- 172.21.15.44:0/262861087 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500d7060 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c49600640 1 -- 172.21.15.44:0/262861087 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8c500346c0 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c58003d10 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c49600640 1 -- 172.21.15.44:0/262861087 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c50034970 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c69048740 10 client.25738 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c1cd46820 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c49600640 1 -- 172.21.15.44:0/262861087 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c58003d10 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.009+0000 7f8c49600640 1 -- 172.21.15.44:0/262861087 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f8c5003c030 con 0x555c1ce56170 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c69048740 10 client.25738 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c69048740 10 client.25738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c69048740 20 client.25738 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c69048740 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ce6f5b0 0x555c1ce71a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c69048740 4 client.25738 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1cdc8a70 con 0x555c1ce6f5b0 2024-04-20T11:23:51.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.010+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ce6f5b0 0x555c1ce71a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.011+0000 7f8c49600640 10 client.25738 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.011+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ce6f5b0 0x555c1ce71a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=beedbef1d971d8fd server_cookie=a9e167677316e4c in_seq=0 out_seq=0 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.011+0000 7f8c49600640 1 -- 172.21.15.44:0/262861087 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c1cdc8a70 con 0x555c1ce6f5b0 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.011+0000 7f8c49600640 10 client.25738 handle_command_reply: tid=0 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 client.25738 shutdown 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 msgr2=0x555c1ce5c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5c670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ef680 tx=0x7f8c500bdd20 comp rx=0 tx=0).stop 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 shutdown_connections 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce5f670 0x555c1ce5d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ce6f5b0 0x555c1ce71a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce5cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 --2- 172.21.15.44:0/262861087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5c670 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c49600640 0 client.25738 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:51.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 >> 172.21.15.44:0/262861087 conn(0x555c1ce508a0 msgr2=0x555c1ce525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 shutdown_connections 2024-04-20T11:23:51.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 1 -- 172.21.15.44:0/262861087 wait complete. 2024-04-20T11:23:51.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.012+0000 7f8c69048740 20 client.25738 trim_cache size 0 max 16384 2024-04-20T11:23:51.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.032+0000 7f8c69048740 1 Processor -- start 2024-04-20T11:23:51.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.032+0000 7f8c69048740 1 -- start start 2024-04-20T11:23:51.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c69048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c69048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56a90 0x555c1ce6b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c69048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c69048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601b3fe0 con 0x555c1ce5f670 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c69048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601b5b20 con 0x555c1ce56a90 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c69048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601b46b0 con 0x555c1ce56170 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35726/0 (socket says 172.21.15.44:35726) 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c5f400640 1 -- 172.21.15.44:0/3772478404 learned_addr learned my addr 172.21.15.44:0/3772478404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.033+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56a90 0x555c1ce6b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5f400640 1 -- 172.21.15.44:0/3772478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56a90 msgr2=0x555c1ce6b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5f400640 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56a90 0x555c1ce6b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5f400640 1 -- 172.21.15.44:0/3772478404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 msgr2=0x555c1ce524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5f400640 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5f400640 1 -- 172.21.15.44:0/3772478404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1ce5b640 con 0x555c1ce56170 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c5f400640 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5eda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ba330 tx=0x7f8c50032890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9d010bf2ef3549 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c4be00640 1 -- 172.21.15.44:0/3772478404 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50034970 con 0x555c1ce56170 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c4be00640 1 -- 172.21.15.44:0/3772478404 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c500be5d0 con 0x555c1ce56170 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.034+0000 7f8c4be00640 1 -- 172.21.15.44:0/3772478404 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500c66b0 con 0x555c1ce56170 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 -- 172.21.15.44:0/3772478404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 msgr2=0x555c1ce5eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5eda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ba330 tx=0x7f8c50032890 comp rx=0 tx=0).stop 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 -- 172.21.15.44:0/3772478404 shutdown_connections 2024-04-20T11:23:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce5f670 0x555c1ce524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56a90 0x555c1ce6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 --2- 172.21.15.44:0/3772478404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce56170 0x555c1ce5eda0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 -- 172.21.15.44:0/3772478404 >> 172.21.15.44:0/3772478404 conn(0x555c1ce508a0 msgr2=0x555c1ce50c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.035+0000 7f8c69048740 1 -- 172.21.15.44:0/3772478404 shutdown_connections 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.037+0000 7f8c69048740 1 -- 172.21.15.44:0/3772478404 wait complete. 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.037+0000 7f8c69048740 1 -- start start 2024-04-20T11:23:51.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.037+0000 7f8c69048740 1 Processor -- start 2024-04-20T11:23:51.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce59650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce59b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce5f670 0x555c1ced4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601b5450 con 0x555c1ce56a90 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601b4d80 con 0x555c1ce56170 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601b1fb0 con 0x555c1ce5f670 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c69048740 10 client.0 _ll_register_callbacks cb 0x7f8c500ba170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce59650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce59b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce59650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55534/0 (socket says 172.21.15.44:55534) 2024-04-20T11:23:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c5f400640 1 -- 172.21.15.44:0/1420364500 learned_addr learned my addr 172.21.15.44:0/1420364500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.038+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce5f670 0x555c1ced4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5f400640 1 -- 172.21.15.44:0/1420364500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce5f670 msgr2=0x555c1ced4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce5f670 0x555c1ced4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5f400640 1 -- 172.21.15.44:0/1420364500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 msgr2=0x555c1ce59b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce59b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5f400640 1 -- 172.21.15.44:0/1420364500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1cacd3f0 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce5f670 0x555c1ced4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c5f400640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce59650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c500f22e0 tx=0x7f8c50034740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=806b42da02393049 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c49600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c49600640 1 -- 172.21.15.44:0/1420364500 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500d7400 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c49600640 1 -- 172.21.15.44:0/1420364500 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8c500c66b0 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.039+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c500be5d0 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c49600640 1 -- 172.21.15.44:0/1420364500 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c500c6b40 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 10 client.25742 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c1cb977d0 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c49600640 1 -- 172.21.15.44:0/1420364500 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c500be5d0 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c49600640 1 -- 172.21.15.44:0/1420364500 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f8c5003c040 con 0x555c1ce56170 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 10 client.25742 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 10 client.25742 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:23:51.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 20 client.25742 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ced6120 0x555c1ced8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 4 client.25742 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:51.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c1cdc28e0 con 0x555c1ced6120 2024-04-20T11:23:51.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.040+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ced6120 0x555c1ced8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ced6120 0x555c1ced8510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee829a41c6b4659 server_cookie=2e2fcc114f3ea43c in_seq=0 out_seq=0 2024-04-20T11:23:51.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c49600640 10 client.25742 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:51.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c49600640 1 -- 172.21.15.44:0/1420364500 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c1cdc28e0 con 0x555c1ced6120 2024-04-20T11:23:51.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c49600640 10 client.25742 handle_command_reply: tid=0 2024-04-20T11:23:51.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 client.25742 shutdown 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 msgr2=0x555c1ce59650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce59650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c500f22e0 tx=0x7f8c50034740 comp rx=0 tx=0).stop 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 shutdown_connections 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c1ce5f670 0x555c1ced4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555c1ced6120 0x555c1ced8510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c1ce56a90 0x555c1ce59b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c49600640 0 client.25742 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.041+0000 7f8c69048740 1 --2- 172.21.15.44:0/1420364500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c1ce56170 0x555c1ce59650 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.042+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 >> 172.21.15.44:0/1420364500 conn(0x555c1ce508a0 msgr2=0x555c1ce5cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.042+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 shutdown_connections 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.042+0000 7f8c69048740 1 -- 172.21.15.44:0/1420364500 wait complete. 2024-04-20T11:23:51.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.042+0000 7f8c69048740 20 client.25742 trim_cache size 0 max 16384 2024-04-20T11:23:51.045 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c440a3f50 msgr2=0x7f8c440a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c440a3f50 0x7f8c440a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c580117d0 tx=0x7f8c58011630 comp rx=0 tx=0).stop 2024-04-20T11:23:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 msgr2=0x7f8c601a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 0x7f8c601a3b40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c5002c0b0 tx=0x7f8c500045b0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 shutdown_connections 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6010df10 0x7f8c6010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c440a3f50 0x7f8c440a6410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c60107140 0x7f8c601a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 --2- 172.21.15.44:0/1714845968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c60102070 0x7f8c601a3b40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 >> 172.21.15.44:0/1714845968 conn(0x7f8c600fd930 msgr2=0x7f8c600fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 shutdown_connections 2024-04-20T11:23:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.043+0000 7f8c64e00640 1 -- 172.21.15.44:0/1714845968 wait complete. 2024-04-20T11:23:51.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:23:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 -- 172.21.15.44:0/1527052880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 msgr2=0x7f0780104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 --2- 172.21.15.44:0/1527052880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f0780104660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0774009900 tx=0x7f077402bc40 comp rx=0 tx=0).stop 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 -- 172.21.15.44:0/1527052880 shutdown_connections 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 --2- 172.21.15.44:0/1527052880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0780109350 0x7f0780110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 --2- 172.21.15.44:0/1527052880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780104c30 0x7f0780108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 --2- 172.21.15.44:0/1527052880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f0780104660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.283+0000 7f0787200640 1 -- 172.21.15.44:0/1527052880 >> 172.21.15.44:0/1527052880 conn(0x7f07800ffaa0 msgr2=0x7f0780101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.284+0000 7f0787200640 1 -- 172.21.15.44:0/1527052880 shutdown_connections 2024-04-20T11:23:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.284+0000 7f0787200640 1 -- 172.21.15.44:0/1527052880 wait complete. 2024-04-20T11:23:51.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.285+0000 7f0787200640 1 Processor -- start 2024-04-20T11:23:51.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.285+0000 7f0787200640 1 -- start start 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0787200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f07801a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0787200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780109350 0x7f07801a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0787200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f078019a800 0x7f078019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0787200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0780114520 con 0x7f0780104280 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0787200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07801143a0 con 0x7f0780109350 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0787200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07801146a0 con 0x7f078019a800 2024-04-20T11:23:51.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0785e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f07801a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0785e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f07801a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39442/0 (socket says 172.21.15.44:39442) 2024-04-20T11:23:51.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0785e00640 1 -- 172.21.15.44:0/3962537235 learned_addr learned my addr 172.21.15.44:0/3962537235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.286+0000 7f0785400640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780109350 0x7f07801a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0786800640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f078019a800 0x7f078019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785400640 1 -- 172.21.15.44:0/3962537235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f078019a800 msgr2=0x7f078019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785400640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f078019a800 0x7f078019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785400640 1 -- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 msgr2=0x7f07801a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785400640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f07801a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785400640 1 -- 172.21.15.44:0/3962537235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078019b390 con 0x7f0780109350 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0786800640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f078019a800 0x7f078019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785e00640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f07801a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.287+0000 7f0785400640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780109350 0x7f07801a0bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f077000e9c0 tx=0x7f077000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.288+0000 7f077aa00640 1 -- 172.21.15.44:0/3962537235 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077000ccc0 con 0x7f0780109350 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.288+0000 7f077aa00640 1 -- 172.21.15.44:0/3962537235 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0770010930 con 0x7f0780109350 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.288+0000 7f077aa00640 1 -- 172.21.15.44:0/3962537235 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700189c0 con 0x7f0780109350 2024-04-20T11:23:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.289+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078019b680 con 0x7f0780109350 2024-04-20T11:23:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.289+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07801a77e0 con 0x7f0780109350 2024-04-20T11:23:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.294+0000 7f078b418740 1 Processor -- start 2024-04-20T11:23:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.294+0000 7f077aa00640 1 -- 172.21.15.44:0/3962537235 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f077001f020 con 0x7f0780109350 2024-04-20T11:23:51.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.294+0000 7f078b418740 1 -- start start 2024-04-20T11:23:51.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.295+0000 7f078b418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.295+0000 7f078b418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f078b418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f078b418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563282010050 con 0x5632820b42a0 2024-04-20T11:23:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f078b418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563281fb6490 con 0x5632820b3980 2024-04-20T11:23:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f078b418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56328200c010 con 0x5632820bce80 2024-04-20T11:23:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f0785400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f0785400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820bf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35762/0 (socket says 172.21.15.44:35762) 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f0785400640 1 -- 172.21.15.44:0/3378311063 learned_addr learned my addr 172.21.15.44:0/3378311063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f077aa00640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07680a3fd0 0x7f07680a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f077aa00640 1 -- 172.21.15.44:0/3962537235 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f07700d0be0 con 0x7f0780109350 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.296+0000 7f0786800640 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785e00640 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0786800640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07680a3fd0 0x7f07680a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785400640 1 -- 172.21.15.44:0/3378311063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 msgr2=0x5632820b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785400640 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785400640 1 -- 172.21.15.44:0/3378311063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 msgr2=0x5632820bc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785400640 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820bc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785400640 1 -- 172.21.15.44:0/3378311063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563281fbe070 con 0x5632820bce80 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.297+0000 7f0785e00640 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.298+0000 7f0785400640 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820bf270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07700ce100 tx=0x7f0770018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4269ebfc66617590 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.298+0000 7f0786800640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07680a3fd0 0x7f07680a6490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f077c005c40 tx=0x7f077c004ab0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.298+0000 7f076f000640 1 -- 172.21.15.44:0/3378311063 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077009c9b0 con 0x5632820bce80 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.298+0000 7f076f000640 1 -- 172.21.15.44:0/3378311063 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07700a0d60 con 0x5632820bce80 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.298+0000 7f076f000640 1 -- 172.21.15.44:0/3378311063 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077009cea0 con 0x5632820bce80 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 -- 172.21.15.44:0/3378311063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 msgr2=0x5632820bf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820bf270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07700ce100 tx=0x7f0770018420 comp rx=0 tx=0).stop 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 -- 172.21.15.44:0/3378311063 shutdown_connections 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820bf270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 --2- 172.21.15.44:0/3378311063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 -- 172.21.15.44:0/3378311063 >> 172.21.15.44:0/3378311063 conn(0x5632820ae0b0 msgr2=0x5632820ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 -- 172.21.15.44:0/3378311063 shutdown_connections 2024-04-20T11:23:51.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.299+0000 7f078b418740 1 -- 172.21.15.44:0/3378311063 wait complete. 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.300+0000 7f078b418740 1 -- start start 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.300+0000 7f078b418740 1 Processor -- start 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.300+0000 7f078b418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f078b418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820b42a0 0x5632820ba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f078b418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 0x5632820ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f078b418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5632820bfbb0 con 0x5632820bce80 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f078b418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563281fc9860 con 0x5632820b3980 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f078b418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632820bfd30 con 0x5632820b42a0 2024-04-20T11:23:51.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f078b418740 10 client.0 _ll_register_callbacks cb 0x563281f97fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f0786800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 0x5632820ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f0786800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 0x5632820ba980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39460/0 (socket says 172.21.15.44:39460) 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f0786800640 1 -- 172.21.15.44:0/1192233657 learned_addr learned my addr 172.21.15.44:0/1192233657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f0785400640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820b42a0 0x5632820ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.301+0000 7f0785e00640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0786800640 1 -- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820b42a0 msgr2=0x5632820ba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0786800640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820b42a0 0x5632820ba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0786800640 1 -- 172.21.15.44:0/1192233657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 msgr2=0x5632820b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0786800640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0786800640 1 -- 172.21.15.44:0/1192233657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563281fcd950 con 0x5632820bce80 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0786800640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 0x5632820ba980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f077c009f80 tx=0x7f077c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade5bea854fc15c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0763400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:51.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.302+0000 7f0763400640 1 -- 172.21.15.44:0/1192233657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c025070 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f0763400640 1 -- 172.21.15.44:0/1192233657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f077c021660 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07700a0d60 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f0763400640 1 -- 172.21.15.44:0/1192233657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f077c021860 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f078b418740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563281f9b820 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f0763400640 1 -- 172.21.15.44:0/1192233657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07700a0d60 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.303+0000 7f0763400640 1 -- 172.21.15.44:0/1192233657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f077c032050 con 0x5632820bce80 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.304+0000 7f078b418740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.304+0000 7f078b418740 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.304+0000 7f078b418740 20 client.15627 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.304+0000 7f078b418740 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632820ccde0 0x5632820cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.304+0000 7f078b418740 4 client.15627 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:51.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.304+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328201da70 con 0x5632820ccde0 2024-04-20T11:23:51.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.305+0000 7f0785e00640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632820ccde0 0x5632820cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.305+0000 7f0785e00640 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632820ccde0 0x5632820cf1f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4f383cd2ad96b2e server_cookie=cab4f6aeb28dfa5b in_seq=0 out_seq=0 2024-04-20T11:23:51.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.305+0000 7f0763400640 10 client.15627 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.306+0000 7f0763400640 1 -- 172.21.15.44:0/1192233657 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56328201da70 con 0x5632820ccde0 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.306+0000 7f0763400640 10 client.15627 handle_command_reply: tid=0 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.306+0000 7f078b418740 1 client.15627 shutdown 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.306+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 msgr2=0x5632820ba980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.306+0000 7f078b418740 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 0x5632820ba980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f077c009f80 tx=0x7f077c004a40 comp rx=0 tx=0).stop 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 shutdown_connections 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632820ccde0 0x5632820cf1f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820bce80 0x5632820ba980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820b42a0 0x5632820ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 --2- 172.21.15.44:0/1192233657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 >> 172.21.15.44:0/1192233657 conn(0x5632820ae0b0 msgr2=0x5632820afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f0763400640 0 client.15627 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:51.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 shutdown_connections 2024-04-20T11:23:51.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 1 -- 172.21.15.44:0/1192233657 wait complete. 2024-04-20T11:23:51.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.307+0000 7f078b418740 20 client.15627 trim_cache size 0 max 16384 2024-04-20T11:23:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.321+0000 7f078b418740 1 Processor -- start 2024-04-20T11:23:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.321+0000 7f078b418740 1 -- start start 2024-04-20T11:23:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f078b418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 0x5632820bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f078b418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b42a0 0x5632820c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f078b418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f078b418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07801ab900 con 0x5632820b3980 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f078b418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07801ad440 con 0x5632820b42a0 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f078b418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07801abfd0 con 0x5632820bce80 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f0785e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 0x5632820bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f0785e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 0x5632820bc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39470/0 (socket says 172.21.15.44:39470) 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f0785e00640 1 -- 172.21.15.44:0/2662902227 learned_addr learned my addr 172.21.15.44:0/2662902227 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.322+0000 7f0786800640 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0785400640 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b42a0 0x5632820c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0786800640 1 -- 172.21.15.44:0/2662902227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b42a0 msgr2=0x5632820c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0786800640 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b42a0 0x5632820c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0786800640 1 -- 172.21.15.44:0/2662902227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 msgr2=0x5632820bc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0786800640 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 0x5632820bc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0786800640 1 -- 172.21.15.44:0/2662902227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632820b40d0 con 0x5632820bce80 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0785e00640 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 0x5632820bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.323+0000 7f0786800640 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820a47e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f077c013010 tx=0x7f077c009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5da1775930c0b303 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.324+0000 7f076e600640 1 -- 172.21.15.44:0/2662902227 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c025040 con 0x5632820bce80 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.324+0000 7f076e600640 1 -- 172.21.15.44:0/2662902227 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f077c038070 con 0x5632820bce80 2024-04-20T11:23:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.324+0000 7f076e600640 1 -- 172.21.15.44:0/2662902227 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c004e10 con 0x5632820bce80 2024-04-20T11:23:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.324+0000 7f078b418740 1 -- 172.21.15.44:0/2662902227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 msgr2=0x5632820a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.324+0000 7f078b418740 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820a47e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f077c013010 tx=0x7f077c009480 comp rx=0 tx=0).stop 2024-04-20T11:23:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 -- 172.21.15.44:0/2662902227 shutdown_connections 2024-04-20T11:23:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820a47e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b42a0 0x5632820c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 --2- 172.21.15.44:0/2662902227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b3980 0x5632820bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 -- 172.21.15.44:0/2662902227 >> 172.21.15.44:0/2662902227 conn(0x5632820ae0b0 msgr2=0x5632820a40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 -- 172.21.15.44:0/2662902227 shutdown_connections 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 -- 172.21.15.44:0/2662902227 wait complete. 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 -- start start 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.325+0000 7f078b418740 1 Processor -- start 2024-04-20T11:23:51.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820b7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07801acd70 con 0x5632820b42a0 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07801ac6a0 con 0x5632820b3980 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07801a9910 con 0x5632820bce80 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f078b418740 10 client.0 _ll_register_callbacks cb 0x563281f97fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.326+0000 7f0785e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0785400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0785e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55574/0 (socket says 172.21.15.44:55574) 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0785e00640 1 -- 172.21.15.44:0/4116660136 learned_addr learned my addr 172.21.15.44:0/4116660136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 -- 172.21.15.44:0/4116660136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 msgr2=0x5632820b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 -- 172.21.15.44:0/4116660136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 msgr2=0x5632820b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 -- 172.21.15.44:0/4116660136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563281d223f0 con 0x5632820bce80 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0785400640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0785e00640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0786800640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820b7720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f077c02ae00 tx=0x7f077c009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486e80307b8b306a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.327+0000 7f0763400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f0763400640 1 -- 172.21.15.44:0/4116660136 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c021660 con 0x5632820bce80 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f0763400640 1 -- 172.21.15.44:0/4116660136 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f077c01b070 con 0x5632820bce80 2024-04-20T11:23:51.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f077c038070 con 0x5632820bce80 2024-04-20T11:23:51.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f0763400640 1 -- 172.21.15.44:0/4116660136 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f077c012a10 con 0x5632820bce80 2024-04-20T11:23:51.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f078b418740 10 client.25759 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563281dec7d0 con 0x5632820bce80 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.328+0000 7f0763400640 1 -- 172.21.15.44:0/4116660136 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f077c038070 con 0x5632820bce80 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.329+0000 7f0763400640 1 -- 172.21.15.44:0/4116660136 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f077c032070 con 0x5632820bce80 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f078b418740 10 client.25759 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f078b418740 10 client.25759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f078b418740 20 client.25759 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f078b418740 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632821339b0 0x563282135da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f078b418740 4 client.25759 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632820178e0 con 0x5632821339b0 2024-04-20T11:23:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.330+0000 7f0785e00640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632821339b0 0x563282135da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f0785e00640 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632821339b0 0x563282135da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aded3f702e0d65f8 server_cookie=aca0d19db1c2ee4a in_seq=0 out_seq=0 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f0763400640 10 client.25759 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f0763400640 1 -- 172.21.15.44:0/4116660136 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632820178e0 con 0x5632821339b0 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f0763400640 10 client.25759 handle_command_reply: tid=0 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 client.25759 shutdown 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 msgr2=0x5632820b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820b7720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f077c02ae00 tx=0x7f077c009fd0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 shutdown_connections 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5632821339b0 0x563282135da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632820bce80 0x5632820b7720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632820b42a0 0x5632820b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 --2- 172.21.15.44:0/4116660136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632820b3980 0x5632820b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.331+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 >> 172.21.15.44:0/4116660136 conn(0x5632820ae0b0 msgr2=0x5632820afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.332+0000 7f0763400640 0 client.25759 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.332+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 shutdown_connections 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.332+0000 7f078b418740 1 -- 172.21.15.44:0/4116660136 wait complete. 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.332+0000 7f078b418740 20 client.25759 trim_cache size 0 max 16384 2024-04-20T11:23:51.336 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:51.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07680a3fd0 msgr2=0x7f07680a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07680a3fd0 0x7f07680a6490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f077c005c40 tx=0x7f077c004ab0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780109350 msgr2=0x7f07801a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780109350 0x7f07801a0bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f077000e9c0 tx=0x7f077000ee80 comp rx=0 tx=0).stop 2024-04-20T11:23:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 shutdown_connections 2024-04-20T11:23:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f078019a800 0x7f078019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0780109350 0x7f07801a0bd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f07680a3fd0 0x7f07680a6490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 --2- 172.21.15.44:0/3962537235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0780104280 0x7f07801a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 >> 172.21.15.44:0/3962537235 conn(0x7f07800ffaa0 msgr2=0x7f0780101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 shutdown_connections 2024-04-20T11:23:51.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.334+0000 7f0787200640 1 -- 172.21.15.44:0/3962537235 wait complete. 2024-04-20T11:23:51.356 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:23:51.410 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:51 smithi044 ceph-mon[16855]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:51.410 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:51 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1672610218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:23:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:51 smithi165 ceph-mon[21975]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:51.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:51 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1672610218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 -- 172.21.15.44:0/1114737537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 msgr2=0x7ff0cc10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1114737537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc10ed10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009be0 tx=0x7ff0c402bef0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 -- 172.21.15.44:0/1114737537 shutdown_connections 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1114737537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc10ed10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1114737537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc102a20 0x7ff0cc106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1114737537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0cc102070 0x7ff0cc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 -- 172.21.15.44:0/1114737537 >> 172.21.15.44:0/1114737537 conn(0x7ff0cc0fd930 msgr2=0x7ff0cc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 -- 172.21.15.44:0/1114737537 shutdown_connections 2024-04-20T11:23:51.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.531+0000 7ff0d2200640 1 -- 172.21.15.44:0/1114737537 wait complete. 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.532+0000 7ff0d2200640 1 Processor -- start 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.532+0000 7ff0d2200640 1 -- start start 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0cc102070 0x7ff0cc074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d2200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d2200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc074ce0 0x7ff0cc1a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d2200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc112350 con 0x7ff0cc102070 2024-04-20T11:23:51.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d2200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1121d0 con 0x7ff0cc107140 2024-04-20T11:23:51.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d2200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1124d0 con 0x7ff0cc074ce0 2024-04-20T11:23:51.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0cc102070 0x7ff0cc074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc0747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55578/0 (socket says 172.21.15.44:55578) 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0cbe00640 1 -- 172.21.15.44:0/1017269324 learned_addr learned my addr 172.21.15.44:0/1017269324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.533+0000 7ff0d1800640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc074ce0 0x7ff0cc1a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0cbe00640 1 -- 172.21.15.44:0/1017269324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc074ce0 msgr2=0x7ff0cc1a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc074ce0 0x7ff0cc1a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0cbe00640 1 -- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0cc102070 msgr2=0x7ff0cc074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0cc102070 0x7ff0cc074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0cbe00640 1 -- 172.21.15.44:0/1017269324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc1a7040 con 0x7ff0cc107140 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc0747a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800e9c0 tx=0x7ff0b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0c9600640 1 -- 172.21.15.44:0/1017269324 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800ccc0 con 0x7ff0cc107140 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.534+0000 7ff0c9600640 1 -- 172.21.15.44:0/1017269324 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0b8005520 con 0x7ff0cc107140 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.535+0000 7ff0c9600640 1 -- 172.21.15.44:0/1017269324 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8010690 con 0x7ff0cc107140 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.535+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc1a7330 con 0x7ff0cc107140 2024-04-20T11:23:51.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.535+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc1a7830 con 0x7ff0cc107140 2024-04-20T11:23:51.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.539+0000 7ff0d647e740 1 Processor -- start 2024-04-20T11:23:51.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.539+0000 7ff0d647e740 1 -- start start 2024-04-20T11:23:51.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.539+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdde360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.539+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.539+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d647e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e96cd43050 con 0x55e96cdddf80 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d647e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e96cce9490 con 0x55e96cdde930 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d647e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e96cd3f010 con 0x55e96cdefe80 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdde360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdde360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39506/0 (socket says 172.21.15.44:39506) 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d1800640 1 -- 172.21.15.44:0/2996788822 learned_addr learned my addr 172.21.15.44:0/2996788822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.540+0000 7ff0c9600640 1 -- 172.21.15.44:0/1017269324 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7ff0b80046c0 con 0x7ff0cc107140 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.541+0000 7ff0d1800640 1 -- 172.21.15.44:0/2996788822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 msgr2=0x55e96cdf2270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.541+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf2270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.541+0000 7ff0d1800640 1 -- 172.21.15.44:0/2996788822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 msgr2=0x55e96cdef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.541+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.541+0000 7ff0d1800640 1 -- 172.21.15.44:0/2996788822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96ccf1070 con 0x55e96cdddf80 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.541+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdde360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009be0 tx=0x7ff0c4009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f52880fa9a28ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.542+0000 7ff0be000640 1 -- 172.21.15.44:0/2996788822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4043030 con 0x55e96cdddf80 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.542+0000 7ff0be000640 1 -- 172.21.15.44:0/2996788822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0c4033c40 con 0x55e96cdddf80 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.542+0000 7ff0be000640 1 -- 172.21.15.44:0/2996788822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c402c890 con 0x55e96cdddf80 2024-04-20T11:23:51.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.542+0000 7ff0c9600640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0b40a3f80 0x7ff0b40a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0c9600640 1 -- 172.21.15.44:0/1017269324 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff0b80d0b80 con 0x7ff0cc107140 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 -- 172.21.15.44:0/2996788822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 msgr2=0x55e96cdde360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdde360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009be0 tx=0x7ff0c4009330 comp rx=0 tx=0).stop 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0b40a3f80 0x7ff0b40a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 -- 172.21.15.44:0/2996788822 shutdown_connections 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2996788822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdde360 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.543+0000 7ff0d647e740 1 -- 172.21.15.44:0/2996788822 >> 172.21.15.44:0/2996788822 conn(0x55e96cdd88a0 msgr2=0x55e96cdd8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.544+0000 7ff0d647e740 1 -- 172.21.15.44:0/2996788822 shutdown_connections 2024-04-20T11:23:51.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.544+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0b40a3f80 0x7ff0b40a6440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55e96cdd9b30 tx=0x7ff0c0005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.544+0000 7ff0d647e740 1 -- 172.21.15.44:0/2996788822 wait complete. 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.545+0000 7ff0d647e740 1 -- start start 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.545+0000 7ff0d647e740 1 Processor -- start 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cded450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cded990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e96cdf2be0 con 0x55e96cdddf80 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e96ccfc860 con 0x55e96cdde930 2024-04-20T11:23:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e96cdf2d60 con 0x55e96cdefe80 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d647e740 10 client.0 _ll_register_callbacks cb 0x55e96cccafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0d0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cded450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cded450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55602/0 (socket says 172.21.15.44:55602) 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.546+0000 7ff0cbe00640 1 -- 172.21.15.44:0/2953569112 learned_addr learned my addr 172.21.15.44:0/2953569112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cded990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0cbe00640 1 -- 172.21.15.44:0/2953569112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 msgr2=0x55e96cded990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cded990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0cbe00640 1 -- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 msgr2=0x55e96cdecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0cbe00640 1 -- 172.21.15.44:0/2953569112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96cd00950 con 0x55e96cdde930 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.547+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.548+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cded450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80ce1f0 tx=0x7ff0b809c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=802f3c293972ade1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.548+0000 7ff0b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:51.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.548+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2953569112 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b809c950 con 0x55e96cdde930 2024-04-20T11:23:51.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.548+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2953569112 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff0b80870b0 con 0x55e96cdde930 2024-04-20T11:23:51.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.548+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c4033c40 con 0x55e96cdde930 2024-04-20T11:23:51.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.549+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2953569112 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0b80a0d00 con 0x55e96cdde930 2024-04-20T11:23:51.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.549+0000 7ff0d647e740 10 client.25769 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.549+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e96ccce820 con 0x55e96cdde930 2024-04-20T11:23:51.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.549+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2953569112 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0c4033c40 con 0x55e96cdde930 2024-04-20T11:23:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.549+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2953569112 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7ff0b80bbb10 con 0x55e96cdde930 2024-04-20T11:23:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d647e740 10 client.25769 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d647e740 10 client.25769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:23:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d647e740 20 client.25769 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96cdffe80 0x55e96ce02270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d647e740 4 client.25769 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e96cd50a70 con 0x55e96cdffe80 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.550+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96cdffe80 0x55e96ce02270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.551+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96cdffe80 0x55e96ce02270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a77062c1e9088b05 server_cookie=8e88ccfcb472d43c in_seq=0 out_seq=0 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.551+0000 7ff0b2a00640 10 client.25769 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2953569112 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e96cd50a70 con 0x55e96cdffe80 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0b2a00640 10 client.25769 handle_command_reply: tid=0 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0d647e740 1 client.25769 shutdown 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 msgr2=0x55e96cded450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cded450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80ce1f0 tx=0x7ff0b809c7e0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 shutdown_connections 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.552+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96cdffe80 0x55e96ce02270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cded990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cded450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2953569112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 >> 172.21.15.44:0/2953569112 conn(0x55e96cdd88a0 msgr2=0x55e96cdda5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0b2a00640 0 client.25769 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 shutdown_connections 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 1 -- 172.21.15.44:0/2953569112 wait complete. 2024-04-20T11:23:51.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.553+0000 7ff0d647e740 20 client.25769 trim_cache size 0 max 16384 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.581+0000 7ff0d647e740 1 Processor -- start 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.581+0000 7ff0d647e740 1 -- start start 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdddf80 0x55e96cdef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdde930 0x55e96cdfbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96ce07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d647e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ab720 con 0x55e96cdde930 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d647e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ad260 con 0x55e96cdddf80 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d647e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1abdf0 con 0x55e96cdefe80 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdde930 0x55e96cdfbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96ce07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdddf80 0x55e96cdef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96ce07b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35850/0 (socket says 172.21.15.44:35850) 2024-04-20T11:23:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.582+0000 7ff0d1800640 1 -- 172.21.15.44:0/2547773819 learned_addr learned my addr 172.21.15.44:0/2547773819 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d0e00640 1 -- 172.21.15.44:0/2547773819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 msgr2=0x55e96ce07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96ce07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d0e00640 1 -- 172.21.15.44:0/2547773819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdde930 msgr2=0x55e96cdfbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdde930 0x55e96cdfbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d0e00640 1 -- 172.21.15.44:0/2547773819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96cdde6d0 con 0x55e96cdddf80 2024-04-20T11:23:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96ce07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdddf80 0x55e96cdef640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff0b809ac00 tx=0x7ff0c0022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb1d6fb626e38a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.583+0000 7ff0bd600640 1 -- 172.21.15.44:0/2547773819 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c002b530 con 0x55e96cdddf80 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0bd600640 1 -- 172.21.15.44:0/2547773819 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0c0005520 con 0x55e96cdddf80 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0bd600640 1 -- 172.21.15.44:0/2547773819 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0004ce0 con 0x55e96cdddf80 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 -- 172.21.15.44:0/2547773819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdddf80 msgr2=0x55e96cdef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdddf80 0x55e96cdef640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff0b809ac00 tx=0x7ff0c0022040 comp rx=0 tx=0).stop 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 -- 172.21.15.44:0/2547773819 shutdown_connections 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96ce07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdde930 0x55e96cdfbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2547773819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdddf80 0x55e96cdef640 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.584+0000 7ff0d647e740 1 -- 172.21.15.44:0/2547773819 >> 172.21.15.44:0/2547773819 conn(0x55e96cdd88a0 msgr2=0x55e96cdda530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.585+0000 7ff0d647e740 1 -- 172.21.15.44:0/2547773819 shutdown_connections 2024-04-20T11:23:51.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.585+0000 7ff0d647e740 1 -- 172.21.15.44:0/2547773819 wait complete. 2024-04-20T11:23:51.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.585+0000 7ff0d647e740 1 -- start start 2024-04-20T11:23:51.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.585+0000 7ff0d647e740 1 Processor -- start 2024-04-20T11:23:51.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdf3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdf40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1acb90 con 0x55e96cdddf80 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ac4c0 con 0x55e96cdde930 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1a9730 con 0x55e96cdefe80 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d647e740 10 client.0 _ll_register_callbacks cb 0x7ff0b80056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdf3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35858/0 (socket says 172.21.15.44:35858) 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.586+0000 7ff0d1800640 1 -- 172.21.15.44:0/2402495845 learned_addr learned my addr 172.21.15.44:0/2402495845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0d1800640 1 -- 172.21.15.44:0/2402495845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 msgr2=0x55e96cdf40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdf40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdf40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0d1800640 1 -- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 msgr2=0x55e96cdf3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdf3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0d1800640 1 -- 172.21.15.44:0/2402495845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96ca553f0 con 0x55e96cdefe80 2024-04-20T11:23:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0cbe00640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdf40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0d1800640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf45f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004ab0 tx=0x7ff0c4003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818dab12f1803b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.587+0000 7ff0b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2402495845 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4043040 con 0x55e96cdefe80 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c0005520 con 0x55e96cdefe80 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2402495845 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff0c40336f0 con 0x55e96cdefe80 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2402495845 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0c4058050 con 0x55e96cdefe80 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0d647e740 10 client.25774 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e96cb1f7d0 con 0x55e96cdefe80 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2402495845 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0c0005520 con 0x55e96cdefe80 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.588+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2402495845 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7ff0c4004290 con 0x55e96cdefe80 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d647e740 10 client.25774 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d647e740 10 client.25774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d647e740 20 client.25774 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96ce667e0 0x55e96ce68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d647e740 4 client.25774 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e96cd4a8e0 con 0x55e96ce667e0 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.589+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96ce667e0 0x55e96ce68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.590+0000 7ff0d0e00640 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96ce667e0 0x55e96ce68bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18a334eaa341e7db server_cookie=b264e6ca3346d891 in_seq=0 out_seq=0 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.590+0000 7ff0b2a00640 10 client.25774 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.590+0000 7ff0b2a00640 1 -- 172.21.15.44:0/2402495845 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e96cd4a8e0 con 0x55e96ce667e0 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.590+0000 7ff0b2a00640 10 client.25774 handle_command_reply: tid=0 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.590+0000 7ff0d647e740 1 client.25774 shutdown 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 msgr2=0x55e96cdf45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf45f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004ab0 tx=0x7ff0c4003d80 comp rx=0 tx=0).stop 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 shutdown_connections 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55e96ce667e0 0x55e96ce68bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e96cdefe80 0x55e96cdf45f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e96cdde930 0x55e96cdf40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 --2- 172.21.15.44:0/2402495845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e96cdddf80 0x55e96cdf3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 >> 172.21.15.44:0/2402495845 conn(0x55e96cdd88a0 msgr2=0x55e96cdeceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0b2a00640 0 client.25774 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 shutdown_connections 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.591+0000 7ff0d647e740 1 -- 172.21.15.44:0/2402495845 wait complete. 2024-04-20T11:23:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.592+0000 7ff0d647e740 20 client.25774 trim_cache size 0 max 16384 2024-04-20T11:23:51.594 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.592+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0b40a3f80 msgr2=0x7ff0b40a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.592+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0b40a3f80 0x7ff0b40a6440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55e96cdd9b30 tx=0x7ff0c0005f90 comp rx=0 tx=0).stop 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.592+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 msgr2=0x7ff0cc0747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.592+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc0747a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800e9c0 tx=0x7ff0b800ee80 comp rx=0 tx=0).stop 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 shutdown_connections 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc074ce0 0x7ff0cc1a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0cc107140 0x7ff0cc0747a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff0b40a3f80 0x7ff0b40a6440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 --2- 172.21.15.44:0/1017269324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0cc102070 0x7ff0cc074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 >> 172.21.15.44:0/1017269324 conn(0x7ff0cc0fd930 msgr2=0x7ff0cc0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 shutdown_connections 2024-04-20T11:23:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.593+0000 7ff0d2200640 1 -- 172.21.15.44:0/1017269324 wait complete. 2024-04-20T11:23:51.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:23:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:51 smithi078 ceph-mon[21464]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:51 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1672610218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 -- 172.21.15.44:0/1724743018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9010c3e0 msgr2=0x7fcc9010e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1724743018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9010c3e0 0x7fcc9010e880 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ca0 tx=0x7fcc8802c190 comp rx=0 tx=0).stop 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 -- 172.21.15.44:0/1724743018 shutdown_connections 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1724743018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9010c3e0 0x7fcc9010e880 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1724743018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc901038e0 0x7fcc9010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1724743018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc90102fc0 0x7fcc901033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 -- 172.21.15.44:0/1724743018 >> 172.21.15.44:0/1724743018 conn(0x7fcc900fc7e0 msgr2=0x7fcc900fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 -- 172.21.15.44:0/1724743018 shutdown_connections 2024-04-20T11:23:51.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.828+0000 7fcc96c00640 1 -- 172.21.15.44:0/1724743018 wait complete. 2024-04-20T11:23:51.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 Processor -- start 2024-04-20T11:23:51.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 -- start start 2024-04-20T11:23:51.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 0x7fcc901a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc901038e0 0x7fcc901a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9010c3e0 0x7fcc9019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90112300 con 0x7fcc9010c3e0 2024-04-20T11:23:51.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90112180 con 0x7fcc901038e0 2024-04-20T11:23:51.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.830+0000 7fcc96c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90112480 con 0x7fcc90102fc0 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc95800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 0x7fcc901a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc94e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc901038e0 0x7fcc901a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc95800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 0x7fcc901a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35866/0 (socket says 172.21.15.44:35866) 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc95800640 1 -- 172.21.15.44:0/1042483410 learned_addr learned my addr 172.21.15.44:0/1042483410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc96200640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9010c3e0 0x7fcc9019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc95800640 1 -- 172.21.15.44:0/1042483410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc901038e0 msgr2=0x7fcc901a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc95800640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc901038e0 0x7fcc901a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.831+0000 7fcc95800640 1 -- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9010c3e0 msgr2=0x7fcc9019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc95800640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9010c3e0 0x7fcc9019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc95800640 1 -- 172.21.15.44:0/1042483410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9019afe0 con 0x7fcc90102fc0 2024-04-20T11:23:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc94e00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc901038e0 0x7fcc901a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc96200640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9010c3e0 0x7fcc9019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc95800640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 0x7fcc901a06a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d950 tx=0x7fcc8400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc8e000640 1 -- 172.21.15.44:0/1042483410 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84017030 con 0x7fcc90102fc0 2024-04-20T11:23:51.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc8e000640 1 -- 172.21.15.44:0/1042483410 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc84004020 con 0x7fcc90102fc0 2024-04-20T11:23:51.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.832+0000 7fcc8e000640 1 -- 172.21.15.44:0/1042483410 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84004a00 con 0x7fcc90102fc0 2024-04-20T11:23:51.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.833+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9019b2d0 con 0x7fcc90102fc0 2024-04-20T11:23:51.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.833+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc901a74f0 con 0x7fcc90102fc0 2024-04-20T11:23:51.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.838+0000 7fcc9af84740 1 Processor -- start 2024-04-20T11:23:51.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.838+0000 7fcc9af84740 1 -- start start 2024-04-20T11:23:51.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.838+0000 7fcc8e000640 1 -- 172.21.15.44:0/1042483410 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fcc840041d0 con 0x7fcc90102fc0 2024-04-20T11:23:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2ca8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cb1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc9af84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5575d2c0d050 con 0x5575d2ca8a90 2024-04-20T11:23:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc9af84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575d2bb3490 con 0x5575d2ca8170 2024-04-20T11:23:51.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc9af84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5575d2c09010 con 0x5575d2cb1670 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.839+0000 7fcc94e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc94e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cb3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35882/0 (socket says 172.21.15.44:35882) 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc94e00640 1 -- 172.21.15.44:0/1324896862 learned_addr learned my addr 172.21.15.44:0/1324896862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc96200640 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2ca8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc95800640 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cb1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc96200640 1 -- 172.21.15.44:0/1324896862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 msgr2=0x5575d2cb3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc96200640 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cb3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc96200640 1 -- 172.21.15.44:0/1324896862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 msgr2=0x5575d2cb1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc96200640 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cb1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc96200640 1 -- 172.21.15.44:0/1324896862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d2bbb070 con 0x5575d2ca8170 2024-04-20T11:23:51.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.840+0000 7fcc8e000640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc780a3f50 0x7fcc780a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.841+0000 7fcc95800640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc780a3f50 0x7fcc780a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.841+0000 7fcc8e000640 1 -- 172.21.15.44:0/1042483410 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc840d0420 con 0x7fcc90102fc0 2024-04-20T11:23:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.841+0000 7fcc96200640 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2ca8550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009bf0 tx=0x7fcc88009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67af0f880d8ec7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.842+0000 7fcc82a00640 1 -- 172.21.15.44:0/1324896862 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88043020 con 0x5575d2ca8170 2024-04-20T11:23:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.842+0000 7fcc82a00640 1 -- 172.21.15.44:0/1324896862 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc88032720 con 0x5575d2ca8170 2024-04-20T11:23:51.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.842+0000 7fcc82a00640 1 -- 172.21.15.44:0/1324896862 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88039040 con 0x5575d2ca8170 2024-04-20T11:23:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 -- 172.21.15.44:0/1324896862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 msgr2=0x5575d2ca8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc95800640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc780a3f50 0x7fcc780a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc840cda00 tx=0x7fcc840045d0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2ca8550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009bf0 tx=0x7fcc88009890 comp rx=0 tx=0).stop 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 -- 172.21.15.44:0/1324896862 shutdown_connections 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cb3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cb1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 --2- 172.21.15.44:0/1324896862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2ca8550 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 -- 172.21.15.44:0/1324896862 >> 172.21.15.44:0/1324896862 conn(0x5575d2ca28a0 msgr2=0x5575d2ca2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 -- 172.21.15.44:0/1324896862 shutdown_connections 2024-04-20T11:23:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.843+0000 7fcc9af84740 1 -- 172.21.15.44:0/1324896862 wait complete. 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.844+0000 7fcc9af84740 1 -- start start 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.844+0000 7fcc9af84740 1 Processor -- start 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2caed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2caf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5575d2cb4400 con 0x5575d2ca8a90 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575d2bc6860 con 0x5575d2ca8170 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5575d2cb4580 con 0x5575d2cb1670 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc9af84740 10 client.0 _ll_register_callbacks cb 0x5575d2b94fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc96200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc96200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55670/0 (socket says 172.21.15.44:55670) 2024-04-20T11:23:51.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc96200640 1 -- 172.21.15.44:0/2142031668 learned_addr learned my addr 172.21.15.44:0/2142031668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2caed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.845+0000 7fcc96200640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2caf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc96200640 1 -- 172.21.15.44:0/2142031668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 msgr2=0x5575d2caf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc96200640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2caf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc96200640 1 -- 172.21.15.44:0/2142031668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 msgr2=0x5575d2caed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc96200640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2caed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc96200640 1 -- 172.21.15.44:0/2142031668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d2bca950 con 0x5575d2ca8170 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc96200640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cae820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ca0 tx=0x7fcc88036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8155c7dcc813eb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.846+0000 7fcc77400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc77400640 1 -- 172.21.15.44:0/2142031668 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88039070 con 0x5575d2ca8170 2024-04-20T11:23:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc77400640 1 -- 172.21.15.44:0/2142031668 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc88004730 con 0x5575d2ca8170 2024-04-20T11:23:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc88032720 con 0x5575d2ca8170 2024-04-20T11:23:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc77400640 1 -- 172.21.15.44:0/2142031668 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc88043040 con 0x5575d2ca8170 2024-04-20T11:23:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc9af84740 10 client.25784 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575d2b98820 con 0x5575d2ca8170 2024-04-20T11:23:51.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.847+0000 7fcc77400640 1 -- 172.21.15.44:0/2142031668 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc88032720 con 0x5575d2ca8170 2024-04-20T11:23:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc77400640 1 -- 172.21.15.44:0/2142031668 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fcc8803f880 con 0x5575d2ca8170 2024-04-20T11:23:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc9af84740 10 client.25784 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc9af84740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc9af84740 20 client.25784 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2cc1610 0x5575d2cc3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc9af84740 4 client.25784 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.848+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575d2c1aa70 con 0x5575d2cc1610 2024-04-20T11:23:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.849+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2cc1610 0x5575d2cc3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.850+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2cc1610 0x5575d2cc3a00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e0c2ecd4ba1847 server_cookie=b79bd829d2d6d3 in_seq=0 out_seq=0 2024-04-20T11:23:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.850+0000 7fcc77400640 10 client.25784 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.850+0000 7fcc77400640 1 -- 172.21.15.44:0/2142031668 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575d2c1aa70 con 0x5575d2cc1610 2024-04-20T11:23:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.850+0000 7fcc77400640 10 client.25784 handle_command_reply: tid=0 2024-04-20T11:23:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 client.25784 shutdown 2024-04-20T11:23:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 msgr2=0x5575d2cae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cae820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ca0 tx=0x7fcc88036040 comp rx=0 tx=0).stop 2024-04-20T11:23:51.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 shutdown_connections 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2cc1610 0x5575d2cc3a00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2caf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2caed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2142031668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cae820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc77400640 0 client.25784 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 >> 172.21.15.44:0/2142031668 conn(0x5575d2ca28a0 msgr2=0x5575d2ca45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 shutdown_connections 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.851+0000 7fcc9af84740 1 -- 172.21.15.44:0/2142031668 wait complete. 2024-04-20T11:23:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.852+0000 7fcc9af84740 20 client.25784 trim_cache size 0 max 16384 2024-04-20T11:23:51.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 Processor -- start 2024-04-20T11:23:51.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 -- start start 2024-04-20T11:23:51.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8170 0x5575d2cb0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 0x5575d2cc6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901afc20 con 0x5575d2ca8170 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b1760 con 0x5575d2ca8a90 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.880+0000 7fcc9af84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b02f0 con 0x5575d2cb1670 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 0x5575d2cc6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 0x5575d2cc6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55676/0 (socket says 172.21.15.44:55676) 2024-04-20T11:23:51.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 -- 172.21.15.44:0/2897602333 learned_addr learned my addr 172.21.15.44:0/2897602333 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc96200640 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8170 0x5575d2cb0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 -- 172.21.15.44:0/2897602333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 msgr2=0x5575d2cc6f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:23:51.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc96200640 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 -- 172.21.15.44:0/2897602333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8170 msgr2=0x5575d2cb0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8170 0x5575d2cb0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.881+0000 7fcc94e00640 1 -- 172.21.15.44:0/2897602333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d291f3f0 con 0x5575d2ca8a90 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.882+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 0x5575d2cc6a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5575d2cb45f0 tx=0x7fcc7c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a39160aaaecb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.882+0000 7fcc82000640 1 -- 172.21.15.44:0/2897602333 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c023030 con 0x5575d2ca8a90 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.882+0000 7fcc82000640 1 -- 172.21.15.44:0/2897602333 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc7c005520 con 0x5575d2ca8a90 2024-04-20T11:23:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.882+0000 7fcc82000640 1 -- 172.21.15.44:0/2897602333 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c01f370 con 0x5575d2ca8a90 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 -- 172.21.15.44:0/2897602333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 msgr2=0x5575d2cc6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 0x5575d2cc6a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5575d2cb45f0 tx=0x7fcc7c005f90 comp rx=0 tx=0).stop 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 -- 172.21.15.44:0/2897602333 shutdown_connections 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8a90 0x5575d2cc6a20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2897602333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8170 0x5575d2cb0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.883+0000 7fcc9af84740 1 -- 172.21.15.44:0/2897602333 >> 172.21.15.44:0/2897602333 conn(0x5575d2ca28a0 msgr2=0x5575d2cbd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.884+0000 7fcc9af84740 1 -- 172.21.15.44:0/2897602333 shutdown_connections 2024-04-20T11:23:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.884+0000 7fcc9af84740 1 -- 172.21.15.44:0/2897602333 wait complete. 2024-04-20T11:23:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.884+0000 7fcc9af84740 1 -- start start 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.884+0000 7fcc9af84740 1 Processor -- start 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cc6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cc40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b1090 con 0x5575d2ca8a90 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b09c0 con 0x5575d2ca8170 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901adbf0 con 0x5575d2cb1670 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc9af84740 10 client.0 _ll_register_callbacks cb 0x7fcc88003ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.885+0000 7fcc96200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cc6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cc6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55692/0 (socket says 172.21.15.44:55692) 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 -- 172.21.15.44:0/2904001542 learned_addr learned my addr 172.21.15.44:0/2904001542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cc40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 -- 172.21.15.44:0/2904001542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 msgr2=0x5575d2cc4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 -- 172.21.15.44:0/2904001542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 msgr2=0x5575d2cc40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cc40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 -- 172.21.15.44:0/2904001542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d29e97d0 con 0x5575d2ca8170 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc96200640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cc6ce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc88002230 tx=0x7fcc88058040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26afc08b9e55b5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:51.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.886+0000 7fcc77400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc77400640 1 -- 172.21.15.44:0/2904001542 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8804e070 con 0x5575d2ca8170 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc77400640 1 -- 172.21.15.44:0/2904001542 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc88005c60 con 0x5575d2ca8170 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7c005520 con 0x5575d2ca8170 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc77400640 1 -- 172.21.15.44:0/2904001542 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc88039070 con 0x5575d2ca8170 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc9af84740 10 client.25790 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575d2d27570 con 0x5575d2ca8170 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc77400640 1 -- 172.21.15.44:0/2904001542 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc7c005520 con 0x5575d2ca8170 2024-04-20T11:23:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.887+0000 7fcc77400640 1 -- 172.21.15.44:0/2904001542 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fcc8805f040 con 0x5575d2ca8170 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc9af84740 10 client.25790 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc9af84740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc9af84740 20 client.25790 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2d27b80 0x5575d2d29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc9af84740 4 client.25790 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575d2c148e0 con 0x5575d2d27b80 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.888+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2d27b80 0x5575d2d29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc94e00640 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2d27b80 0x5575d2d29f70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8321655cfe8835ef server_cookie=50a6ccdd2b0da835 in_seq=0 out_seq=0 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc77400640 10 client.25790 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc77400640 1 -- 172.21.15.44:0/2904001542 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575d2c148e0 con 0x5575d2d27b80 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc77400640 10 client.25790 handle_command_reply: tid=0 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 client.25790 shutdown 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 msgr2=0x5575d2cc6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cc6ce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc88002230 tx=0x7fcc88058040 comp rx=0 tx=0).stop 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 shutdown_connections 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5575d2d27b80 0x5575d2d29f70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5575d2cb1670 0x5575d2cc4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575d2ca8a90 0x5575d2cc40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc77400640 0 client.25790 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 --2- 172.21.15.44:0/2904001542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575d2ca8170 0x5575d2cc6ce0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.889+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 >> 172.21.15.44:0/2904001542 conn(0x5575d2ca28a0 msgr2=0x5575d2cbd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.890+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 shutdown_connections 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.890+0000 7fcc9af84740 1 -- 172.21.15.44:0/2904001542 wait complete. 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.890+0000 7fcc9af84740 20 client.25790 trim_cache size 0 max 16384 2024-04-20T11:23:51.893 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc780a3f50 msgr2=0x7fcc780a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc780a3f50 0x7fcc780a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc840cda00 tx=0x7fcc840045d0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 msgr2=0x7fcc901a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 0x7fcc901a06a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d950 tx=0x7fcc8400de10 comp rx=0 tx=0).stop 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 shutdown_connections 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc780a3f50 0x7fcc780a6410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9010c3e0 0x7fcc9019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc901038e0 0x7fcc901a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 --2- 172.21.15.44:0/1042483410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc90102fc0 0x7fcc901a06a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 >> 172.21.15.44:0/1042483410 conn(0x7fcc900fc7e0 msgr2=0x7fcc900fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 shutdown_connections 2024-04-20T11:23:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:51.891+0000 7fcc96c00640 1 -- 172.21.15.44:0/1042483410 wait complete. 2024-04-20T11:23:52.918 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:23:52.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:23:52.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:23:53.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 -- 172.21.15.44:0/2607563454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 msgr2=0x7fc1c4074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 --2- 172.21.15.44:0/2607563454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c4074710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009860 tx=0x7fc1b802bc20 comp rx=0 tx=0).stop 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 -- 172.21.15.44:0/2607563454 shutdown_connections 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 --2- 172.21.15.44:0/2607563454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1c410e6a0 0x7fc1c4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 --2- 172.21.15.44:0/2607563454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c4074c50 0x7fc1c410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 --2- 172.21.15.44:0/2607563454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c4074710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.106+0000 7fc1ca200640 1 -- 172.21.15.44:0/2607563454 >> 172.21.15.44:0/2607563454 conn(0x7fc1c40fd930 msgr2=0x7fc1c40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.107+0000 7fc1ca200640 1 -- 172.21.15.44:0/2607563454 shutdown_connections 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.107+0000 7fc1ca200640 1 -- 172.21.15.44:0/2607563454 wait complete. 2024-04-20T11:23:53.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.108+0000 7fc1ca200640 1 Processor -- start 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.108+0000 7fc1ca200640 1 -- start start 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.108+0000 7fc1ca200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.108+0000 7fc1ca200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1c4074c50 0x7fc1c41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1ca200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c410e6a0 0x7fc1c419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1ca200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4114540 con 0x7fc1c410e6a0 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1ca200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41143c0 con 0x7fc1c4074c50 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1ca200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41146c0 con 0x7fc1c4074330 2024-04-20T11:23:53.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c41a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35912/0 (socket says 172.21.15.44:35912) 2024-04-20T11:23:53.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 -- 172.21.15.44:0/3155907345 learned_addr learned my addr 172.21.15.44:0/3155907345 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c9800640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c410e6a0 0x7fc1c419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1c4074c50 0x7fc1c41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 -- 172.21.15.44:0/3155907345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1c4074c50 msgr2=0x7fc1c41a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1c4074c50 0x7fc1c41a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 -- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c410e6a0 msgr2=0x7fc1c419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c410e6a0 0x7fc1c419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.109+0000 7fc1c8e00640 1 -- 172.21.15.44:0/3155907345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c419b050 con 0x7fc1c4074330 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.110+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c41a06c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80057c0 tx=0x7fc1b802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.110+0000 7fc1c1600640 1 -- 172.21.15.44:0/3155907345 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8041020 con 0x7fc1c4074330 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.110+0000 7fc1c1600640 1 -- 172.21.15.44:0/3155907345 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1b8034940 con 0x7fc1c4074330 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.110+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c419b2e0 con 0x7fc1c4074330 2024-04-20T11:23:53.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.110+0000 7fc1c1600640 1 -- 172.21.15.44:0/3155907345 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8039040 con 0x7fc1c4074330 2024-04-20T11:23:53.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.110+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c41a7510 con 0x7fc1c4074330 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.113+0000 7fc1ce5b7740 1 Processor -- start 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 -- start start 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e37bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b0e3714d20 con 0x55b0e37b95f0 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b0e36bb3e0 con 0x55b0e37b0a10 2024-04-20T11:23:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b0e3710ce0 con 0x55b0e37b00f0 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e37bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1c9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e37bb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39630/0 (socket says 172.21.15.44:39630) 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.114+0000 7fc1c3e00640 1 -- 172.21.15.44:0/2688738187 learned_addr learned my addr 172.21.15.44:0/2688738187 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c9800640 1 -- 172.21.15.44:0/2688738187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 msgr2=0x55b0e37b90b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c9800640 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b90b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c9800640 1 -- 172.21.15.44:0/2688738187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 msgr2=0x55b0e37bb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c9800640 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e37bb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c1600640 1 -- 172.21.15.44:0/3155907345 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fc1b8044020 con 0x7fc1c4074330 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c9800640 1 -- 172.21.15.44:0/2688738187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e36da980 con 0x55b0e37b00f0 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.115+0000 7fc1c9800640 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b04d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4075280 tx=0x7fc1bc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b1dd60a1b5c6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.117+0000 7fc1b6000640 1 -- 172.21.15.44:0/2688738187 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc003b70 con 0x55b0e37b00f0 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.117+0000 7fc1b6000640 1 -- 172.21.15.44:0/2688738187 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1bc003d10 con 0x55b0e37b00f0 2024-04-20T11:23:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.117+0000 7fc1b6000640 1 -- 172.21.15.44:0/2688738187 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011a20 con 0x55b0e37b00f0 2024-04-20T11:23:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.117+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2688738187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 msgr2=0x55b0e37b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.117+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b04d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4075280 tx=0x7fc1bc0107a0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.118+0000 7fc1c1600640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc1ac0a3fd0 0x7fc1ac0a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.118+0000 7fc1c1600640 1 -- 172.21.15.44:0/3155907345 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc1b80f2190 con 0x7fc1c4074330 2024-04-20T11:23:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.118+0000 7fc1c9800640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc1ac0a3fd0 0x7fc1ac0a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.119+0000 7fc1c9800640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc1ac0a3fd0 0x7fc1ac0a6490 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4075280 tx=0x7fc1bc009bd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.119+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2688738187 shutdown_connections 2024-04-20T11:23:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.119+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e37bb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.119+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.119+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2688738187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b04d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.119+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2688738187 >> 172.21.15.44:0/2688738187 conn(0x55b0e37aa820 msgr2=0x55b0e37aac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.120+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2688738187 shutdown_connections 2024-04-20T11:23:53.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.120+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2688738187 wait complete. 2024-04-20T11:23:53.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.120+0000 7fc1ce5b7740 1 -- start start 2024-04-20T11:23:53.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 Processor -- start 2024-04-20T11:23:53.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 0x55b0e37b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b0e37bc430 con 0x55b0e37b00f0 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b0e36ce7a0 con 0x55b0e37b0a10 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b0e37bc5b0 con 0x55b0e37b95f0 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1ce5b7740 10 client.0 _ll_register_callbacks cb 0x55b0e36d0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 0x55b0e37b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 0x55b0e37b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39646/0 (socket says 172.21.15.44:39646) 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.121+0000 7fc1c8e00640 1 -- 172.21.15.44:0/1234832866 learned_addr learned my addr 172.21.15.44:0/1234832866 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c9800640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c8e00640 1 -- 172.21.15.44:0/1234832866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 msgr2=0x55b0e37b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c8e00640 1 -- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 msgr2=0x55b0e37b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c8e00640 1 -- 172.21.15.44:0/1234832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e34ef7e0 con 0x55b0e37b00f0 2024-04-20T11:23:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c9800640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 0x55b0e37b66b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8005e00 tx=0x7fc1b804d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535210d99459876f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.122+0000 7fc1aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1aaa00640 1 -- 172.21.15.44:0/1234832866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80c2420 con 0x55b0e37b00f0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1aaa00640 1 -- 172.21.15.44:0/1234832866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc1b80035f0 con 0x55b0e37b00f0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1bc003d10 con 0x55b0e37b00f0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1aaa00640 1 -- 172.21.15.44:0/1234832866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1b80c6ce0 con 0x55b0e37b00f0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1ce5b7740 10 client.15678 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0e354ff70 con 0x55b0e37b00f0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.123+0000 7fc1aaa00640 1 -- 172.21.15.44:0/1234832866 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1bc003d10 con 0x55b0e37b00f0 2024-04-20T11:23:53.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1aaa00640 1 -- 172.21.15.44:0/1234832866 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fc1b803c030 con 0x55b0e37b00f0 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1ce5b7740 10 client.15678 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1ce5b7740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1ce5b7740 20 client.15678 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e37c98e0 0x55b0e37cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1ce5b7740 4 client.15678 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.124+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0e3722740 con 0x55b0e37c98e0 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.125+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e37c98e0 0x55b0e37cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.126+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e37c98e0 0x55b0e37cbcd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b52a4d3103df872 server_cookie=4ece628ce417fb31 in_seq=0 out_seq=0 2024-04-20T11:23:53.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.126+0000 7fc1aaa00640 10 client.15678 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.126+0000 7fc1aaa00640 1 -- 172.21.15.44:0/1234832866 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0e3722740 con 0x55b0e37c98e0 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.126+0000 7fc1aaa00640 10 client.15678 handle_command_reply: tid=0 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.127+0000 7fc1ce5b7740 1 client.15678 shutdown 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 msgr2=0x55b0e37b66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 0x55b0e37b66b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8005e00 tx=0x7fc1b804d300 comp rx=0 tx=0).stop 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 shutdown_connections 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e37c98e0 0x55b0e37cbcd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/1234832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b00f0 0x55b0e37b66b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 >> 172.21.15.44:0/1234832866 conn(0x55b0e37aa820 msgr2=0x55b0e37ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1aaa00640 0 client.15678 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 shutdown_connections 2024-04-20T11:23:53.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/1234832866 wait complete. 2024-04-20T11:23:53.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.128+0000 7fc1ce5b7740 20 client.15678 trim_cache size 0 max 16384 2024-04-20T11:23:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 Processor -- start 2024-04-20T11:23:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 -- start start 2024-04-20T11:23:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 0x55b0e37b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b0a10 0x55b0e37c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41afb10 con 0x55b0e37b0a10 2024-04-20T11:23:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41b1650 con 0x55b0e37b00f0 2024-04-20T11:23:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.148+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41b01e0 con 0x55b0e37b95f0 2024-04-20T11:23:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 0x55b0e37b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 0x55b0e37b8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55728/0 (socket says 172.21.15.44:55728) 2024-04-20T11:23:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 -- 172.21.15.44:0/2656150499 learned_addr learned my addr 172.21.15.44:0/2656150499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c9800640 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b0a10 0x55b0e37c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 -- 172.21.15.44:0/2656150499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 msgr2=0x55b0e37d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 -- 172.21.15.44:0/2656150499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b0a10 msgr2=0x55b0e37c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b0a10 0x55b0e37c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c8e00640 1 -- 172.21.15.44:0/2656150499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e33e7170 con 0x55b0e37b00f0 2024-04-20T11:23:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.149+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b0a10 0x55b0e37c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:23:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.150+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 0x55b0e37b8de0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80c68a0 tx=0x7fc1b80bc580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3db8067502be0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.150+0000 7fc1b5600640 1 -- 172.21.15.44:0/2656150499 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80c2420 con 0x55b0e37b00f0 2024-04-20T11:23:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.150+0000 7fc1b5600640 1 -- 172.21.15.44:0/2656150499 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1b80035f0 con 0x55b0e37b00f0 2024-04-20T11:23:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.150+0000 7fc1b5600640 1 -- 172.21.15.44:0/2656150499 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80bd720 con 0x55b0e37b00f0 2024-04-20T11:23:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2656150499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 msgr2=0x55b0e37b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 0x55b0e37b8de0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80c68a0 tx=0x7fc1b80bc580 comp rx=0 tx=0).stop 2024-04-20T11:23:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2656150499 shutdown_connections 2024-04-20T11:23:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b95f0 0x55b0e37d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b0a10 0x55b0e37c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/2656150499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b00f0 0x55b0e37b8de0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2656150499 >> 172.21.15.44:0/2656150499 conn(0x55b0e37aa820 msgr2=0x55b0e37a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2656150499 shutdown_connections 2024-04-20T11:23:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.151+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/2656150499 wait complete. 2024-04-20T11:23:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.152+0000 7fc1ce5b7740 1 -- start start 2024-04-20T11:23:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.152+0000 7fc1ce5b7740 1 Processor -- start 2024-04-20T11:23:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e382edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41b0f80 con 0x55b0e37b95f0 2024-04-20T11:23:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41b08b0 con 0x55b0e37b0a10 2024-04-20T11:23:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41adae0 con 0x55b0e37b00f0 2024-04-20T11:23:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1ce5b7740 10 client.0 _ll_register_callbacks cb 0x7fc1b8003f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1c8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35934/0 (socket says 172.21.15.44:35934) 2024-04-20T11:23:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1c8e00640 1 -- 172.21.15.44:0/657465065 learned_addr learned my addr 172.21.15.44:0/657465065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.153+0000 7fc1c9800640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e382edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c8e00640 1 -- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 msgr2=0x55b0e37b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c8e00640 1 -- 172.21.15.44:0/657465065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 msgr2=0x55b0e382edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e382edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c8e00640 1 -- 172.21.15.44:0/657465065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e382f330 con 0x55b0e37b00f0 2024-04-20T11:23:53.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:53.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c9800640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e382edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:23:53.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1c8e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b3a80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80bdd70 tx=0x7fc1b80095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12cc3a5870e47c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:23:53.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1aaa00640 1 -- 172.21.15.44:0/657465065 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80c6a90 con 0x55b0e37b00f0 2024-04-20T11:23:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.154+0000 7fc1aaa00640 1 -- 172.21.15.44:0/657465065 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc1b80d2b10 con 0x55b0e37b00f0 2024-04-20T11:23:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.155+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b80035f0 con 0x55b0e37b00f0 2024-04-20T11:23:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.155+0000 7fc1aaa00640 1 -- 172.21.15.44:0/657465065 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1b80d2d10 con 0x55b0e37b00f0 2024-04-20T11:23:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.155+0000 7fc1ce5b7740 10 client.25807 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.155+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0e382f330 con 0x55b0e37b00f0 2024-04-20T11:23:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.155+0000 7fc1aaa00640 1 -- 172.21.15.44:0/657465065 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1b80035f0 con 0x55b0e37b00f0 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1aaa00640 1 -- 172.21.15.44:0/657465065 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fc1b80c35b0 con 0x55b0e37b00f0 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1ce5b7740 10 client.25807 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1ce5b7740 10 client.25807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1ce5b7740 20 client.25807 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e3830940 0x55b0e3832d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1ce5b7740 4 client.25807 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0e371c5b0 con 0x55b0e3830940 2024-04-20T11:23:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.156+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e3830940 0x55b0e3832d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.157+0000 7fc1c3e00640 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e3830940 0x55b0e3832d30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa77faa02bf10e4c server_cookie=46ee7e9c0bbbbc73 in_seq=0 out_seq=0 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.157+0000 7fc1aaa00640 10 client.25807 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.158+0000 7fc1aaa00640 1 -- 172.21.15.44:0/657465065 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0e371c5b0 con 0x55b0e3830940 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.158+0000 7fc1aaa00640 10 client.25807 handle_command_reply: tid=0 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.158+0000 7fc1ce5b7740 1 client.25807 shutdown 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 msgr2=0x55b0e37b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b3a80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80bdd70 tx=0x7fc1b80095b0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 shutdown_connections 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b0e3830940 0x55b0e3832d30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b0e37b95f0 0x55b0e382edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0e37b0a10 0x55b0e37b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1aaa00640 0 client.25807 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 --2- 172.21.15.44:0/657465065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b0e37b00f0 0x55b0e37b3a80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 >> 172.21.15.44:0/657465065 conn(0x55b0e37aa820 msgr2=0x55b0e37b6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 shutdown_connections 2024-04-20T11:23:53.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 1 -- 172.21.15.44:0/657465065 wait complete. 2024-04-20T11:23:53.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.159+0000 7fc1ce5b7740 20 client.25807 trim_cache size 0 max 16384 2024-04-20T11:23:53.172 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:23:53.172 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:23:53.172 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "858adc39-36fd-4a0c-b15f-d55389b27222", 2024-04-20T11:23:53.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:23:53.173 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:23:53.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc1ac0a3fd0 msgr2=0x7fc1ac0a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc1ac0a3fd0 0x7fc1ac0a6490 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4075280 tx=0x7fc1bc009bd0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 msgr2=0x7fc1c41a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c41a06c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80057c0 tx=0x7fc1b802c640 comp rx=0 tx=0).stop 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 shutdown_connections 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc1ac0a3fd0 0x7fc1ac0a6490 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1c410e6a0 0x7fc1c419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1c4074c50 0x7fc1c41a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 --2- 172.21.15.44:0/3155907345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1c4074330 0x7fc1c41a06c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.160+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 >> 172.21.15.44:0/3155907345 conn(0x7fc1c40fd930 msgr2=0x7fc1c40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.161+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 shutdown_connections 2024-04-20T11:23:53.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.161+0000 7fc1ca200640 1 -- 172.21.15.44:0/3155907345 wait complete. 2024-04-20T11:23:53.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:23:53.185 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:23:53.243 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:53 smithi044 ceph-mon[16855]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-20T11:23:53.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 -- 172.21.15.44:0/4067294304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102a20 msgr2=0x7f5458106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 --2- 172.21.15.44:0/4067294304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102a20 0x7f5458106a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5444009860 tx=0x7f544402bb70 comp rx=0 tx=0).stop 2024-04-20T11:23:53.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 -- 172.21.15.44:0/4067294304 shutdown_connections 2024-04-20T11:23:53.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 --2- 172.21.15.44:0/4067294304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458107140 0x7f545810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 --2- 172.21.15.44:0/4067294304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102a20 0x7f5458106a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 --2- 172.21.15.44:0/4067294304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458102070 0x7f5458102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 -- 172.21.15.44:0/4067294304 >> 172.21.15.44:0/4067294304 conn(0x7f54580fd930 msgr2=0x7f54580ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 -- 172.21.15.44:0/4067294304 shutdown_connections 2024-04-20T11:23:53.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.379+0000 7f545ca00640 1 -- 172.21.15.44:0/4067294304 wait complete. 2024-04-20T11:23:53.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.380+0000 7f545ca00640 1 Processor -- start 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 -- start start 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102070 0x7f545819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 0x7f545819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458198610 0x7f5458198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5458112310 con 0x7f5458198610 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5458112190 con 0x7f5458107140 2024-04-20T11:23:53.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.381+0000 7f545ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5458112490 con 0x7f5458102070 2024-04-20T11:23:53.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5457400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102070 0x7f545819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 0x7f545819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5457e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458198610 0x7f5458198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5457400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102070 0x7f545819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35958/0 (socket says 172.21.15.44:35958) 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 0x7f545819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55754/0 (socket says 172.21.15.44:55754) 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5457400640 1 -- 172.21.15.44:0/1651688647 learned_addr learned my addr 172.21.15.44:0/1651688647 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 -- 172.21.15.44:0/1651688647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102070 msgr2=0x7f545819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102070 0x7f545819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 -- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458198610 msgr2=0x7f5458198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458198610 0x7f5458198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5456a00640 1 -- 172.21.15.44:0/1651688647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5458199130 con 0x7f5458107140 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.382+0000 7f5457e00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458198610 0x7f5458198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.383+0000 7f5456a00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 0x7f545819e9a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5444037040 tx=0x7f54440045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.383+0000 7f544fe00640 1 -- 172.21.15.44:0/1651688647 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444041020 con 0x7f5458107140 2024-04-20T11:23:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.383+0000 7f544fe00640 1 -- 172.21.15.44:0/1651688647 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f544402ca10 con 0x7f5458107140 2024-04-20T11:23:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.383+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581993c0 con 0x7f5458107140 2024-04-20T11:23:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.383+0000 7f544fe00640 1 -- 172.21.15.44:0/1651688647 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444039040 con 0x7f5458107140 2024-04-20T11:23:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.384+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a55b0 con 0x7f5458107140 2024-04-20T11:23:53.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.387+0000 7f5460bf6740 1 Processor -- start 2024-04-20T11:23:53.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.387+0000 7f5460bf6740 1 -- start start 2024-04-20T11:23:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.387+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 0x55883396a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.387+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396aca0 0x558833973340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.387+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x558833975c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5460bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5588338d0150 con 0x55883396aca0 2024-04-20T11:23:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5460bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558833876370 con 0x558833973880 2024-04-20T11:23:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5460bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5588338cc110 con 0x55883396a380 2024-04-20T11:23:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 0x55883396a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 0x55883396a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35972/0 (socket says 172.21.15.44:35972) 2024-04-20T11:23:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 -- 172.21.15.44:0/700907100 learned_addr learned my addr 172.21.15.44:0/700907100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 -- 172.21.15.44:0/700907100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 msgr2=0x558833975c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x558833975c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 -- 172.21.15.44:0/700907100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396aca0 msgr2=0x558833973340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396aca0 0x558833973340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.388+0000 7f5457e00640 1 -- 172.21.15.44:0/700907100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588336ac7d0 con 0x55883396a380 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.389+0000 7f544fe00640 1 -- 172.21.15.44:0/1651688647 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f5444044020 con 0x7f5458107140 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.389+0000 7f5457e00640 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 0x55883396a760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5450009fc0 tx=0x7f545000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2e9aa7689d1c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.389+0000 7f5443e00640 1 -- 172.21.15.44:0/700907100 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545001d070 con 0x55883396a380 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.389+0000 7f5443e00640 1 -- 172.21.15.44:0/700907100 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f545000cce0 con 0x55883396a380 2024-04-20T11:23:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.389+0000 7f5443e00640 1 -- 172.21.15.44:0/700907100 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450004860 con 0x55883396a380 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 -- 172.21.15.44:0/700907100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 msgr2=0x55883396a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 0x55883396a760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5450009fc0 tx=0x7f545000ec30 comp rx=0 tx=0).stop 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 -- 172.21.15.44:0/700907100 shutdown_connections 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x558833975c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396aca0 0x558833973340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 --2- 172.21.15.44:0/700907100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396a380 0x55883396a760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.390+0000 7f5460bf6740 1 -- 172.21.15.44:0/700907100 >> 172.21.15.44:0/700907100 conn(0x558833964ab0 msgr2=0x558833964e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.391+0000 7f544fe00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f543c0a3f50 0x7f543c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.392+0000 7f544fe00640 1 -- 172.21.15.44:0/1651688647 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f54440f19c0 con 0x7f5458107140 2024-04-20T11:23:53.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.392+0000 7f5457400640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f543c0a3f50 0x7f543c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.392+0000 7f5460bf6740 1 -- 172.21.15.44:0/700907100 shutdown_connections 2024-04-20T11:23:53.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.393+0000 7f5457400640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f543c0a3f50 0x7f543c0a6410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x558833969e50 tx=0x7f54480046c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.393+0000 7f5460bf6740 1 -- 172.21.15.44:0/700907100 wait complete. 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 -- start start 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 Processor -- start 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833970940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x558833970e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x5588339713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55883397aa80 con 0x55883396a380 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558833889740 con 0x558833973880 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55883397ac00 con 0x55883396aca0 2024-04-20T11:23:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.394+0000 7f5460bf6740 10 client.0 _ll_register_callbacks cb 0x558833884b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x5588339713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x5588339713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55758/0 (socket says 172.21.15.44:55758) 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 -- 172.21.15.44:0/791736538 learned_addr learned my addr 172.21.15.44:0/791736538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5456a00640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x558833970e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457400640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833970940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 -- 172.21.15.44:0/791736538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 msgr2=0x558833970e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x558833970e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 -- 172.21.15.44:0/791736538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 msgr2=0x558833970940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833970940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 -- 172.21.15.44:0/791736538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588335e23f0 con 0x558833973880 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5457e00640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x5588339713c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5450003c20 tx=0x7f5450003c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31c5fab2b2be386 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.395+0000 7f5440c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5440c00640 1 -- 172.21.15.44:0/791736538 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450026020 con 0x558833973880 2024-04-20T11:23:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f545000cce0 con 0x558833973880 2024-04-20T11:23:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5440c00640 1 -- 172.21.15.44:0/791736538 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f5450022410 con 0x558833973880 2024-04-20T11:23:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5440c00640 1 -- 172.21.15.44:0/791736538 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5450031050 con 0x558833973880 2024-04-20T11:23:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5460bf6740 10 client.25814 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588335a20f0 con 0x558833973880 2024-04-20T11:23:53.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.396+0000 7f5440c00640 1 -- 172.21.15.44:0/791736538 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f545000cce0 con 0x558833973880 2024-04-20T11:23:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5440c00640 1 -- 172.21.15.44:0/791736538 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f545000b040 con 0x558833973880 2024-04-20T11:23:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5460bf6740 10 client.25814 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5460bf6740 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5460bf6740 20 client.25814 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5460bf6740 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558833987fe0 0x55883398a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5460bf6740 4 client.25814 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:23:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.397+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588338ddb70 con 0x558833987fe0 2024-04-20T11:23:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.398+0000 7f5457400640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558833987fe0 0x55883398a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.399+0000 7f5457400640 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558833987fe0 0x55883398a3d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2d51fcb4119f8c server_cookie=55c623a3af8497fd in_seq=0 out_seq=0 2024-04-20T11:23:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.399+0000 7f5440c00640 10 client.25814 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.399+0000 7f5440c00640 1 -- 172.21.15.44:0/791736538 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588338ddb70 con 0x558833987fe0 2024-04-20T11:23:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.399+0000 7f5440c00640 10 client.25814 handle_command_reply: tid=0 2024-04-20T11:23:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.399+0000 7f5460bf6740 1 client.25814 shutdown 2024-04-20T11:23:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 msgr2=0x5588339713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x5588339713c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5450003c20 tx=0x7f5450003c50 comp rx=0 tx=0).stop 2024-04-20T11:23:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 shutdown_connections 2024-04-20T11:23:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558833987fe0 0x55883398a3d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x5588339713c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x558833970e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 --2- 172.21.15.44:0/791736538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833970940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5440c00640 0 client.25814 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 >> 172.21.15.44:0/791736538 conn(0x558833964ab0 msgr2=0x558833966800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 shutdown_connections 2024-04-20T11:23:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 1 -- 172.21.15.44:0/791736538 wait complete. 2024-04-20T11:23:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.400+0000 7f5460bf6740 20 client.25814 trim_cache size 0 max 16384 2024-04-20T11:23:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 Processor -- start 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 -- start start 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833983940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x5588339711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x55883398f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ab7a0 con 0x55883396a380 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ad2e0 con 0x558833973880 2024-04-20T11:23:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.419+0000 7f5460bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f54581abe70 con 0x55883396aca0 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833983940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5456a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x5588339711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833983940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39714/0 (socket says 172.21.15.44:39714) 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457400640 1 -- 172.21.15.44:0/2337798530 learned_addr learned my addr 172.21.15.44:0/2337798530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x55883398f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 -- 172.21.15.44:0/2337798530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 msgr2=0x5588339711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x5588339711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 -- 172.21.15.44:0/2337798530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 msgr2=0x558833983940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833983940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 -- 172.21.15.44:0/2337798530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883398a740 con 0x558833973880 2024-04-20T11:23:53.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.420+0000 7f5457e00640 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x55883398f9d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f545000ec70 tx=0x7f5450009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15846d55fc3af495 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.421+0000 7f5443400640 1 -- 172.21.15.44:0/2337798530 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545003a070 con 0x558833973880 2024-04-20T11:23:53.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.421+0000 7f5443400640 1 -- 172.21.15.44:0/2337798530 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54500041c0 con 0x558833973880 2024-04-20T11:23:53.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.421+0000 7f5443400640 1 -- 172.21.15.44:0/2337798530 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450026070 con 0x558833973880 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.421+0000 7f5460bf6740 1 -- 172.21.15.44:0/2337798530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 msgr2=0x55883398f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.421+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x55883398f9d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f545000ec70 tx=0x7f5450009ec0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 -- 172.21.15.44:0/2337798530 shutdown_connections 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558833973880 0x55883398f9d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55883396aca0 0x5588339711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2337798530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833983940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 -- 172.21.15.44:0/2337798530 >> 172.21.15.44:0/2337798530 conn(0x558833964ab0 msgr2=0x558833963b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 -- 172.21.15.44:0/2337798530 shutdown_connections 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 -- 172.21.15.44:0/2337798530 wait complete. 2024-04-20T11:23:53.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 -- start start 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.422+0000 7f5460bf6740 1 Processor -- start 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833976ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 0x558833977420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558833973880 0x55883397ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54581acc10 con 0x55883396a380 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54581ac540 con 0x55883396aca0 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f54581a97d0 con 0x558833973880 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5460bf6740 10 client.0 _ll_register_callbacks cb 0x7f545001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 0x558833977420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 0x558833977420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55784/0 (socket says 172.21.15.44:55784) 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 -- 172.21.15.44:0/2525583596 learned_addr learned my addr 172.21.15.44:0/2525583596 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5457e00640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558833973880 0x55883397ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5457400640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833976ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 -- 172.21.15.44:0/2525583596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558833973880 msgr2=0x55883397ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558833973880 0x55883397ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 -- 172.21.15.44:0/2525583596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 msgr2=0x558833976ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833976ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.423+0000 7f5456a00640 1 -- 172.21.15.44:0/2525583596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883397f380 con 0x55883396aca0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5456a00640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 0x558833977420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54440ef140 tx=0x7f54440bd4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bba7ad610c3b403 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5440c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5440c00640 1 -- 172.21.15.44:0/2525583596 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54440bd8a0 con 0x55883396aca0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5440c00640 1 -- 172.21.15.44:0/2525583596 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f54440346c0 con 0x55883396aca0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588338d79e0 con 0x55883396aca0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5440c00640 1 -- 172.21.15.44:0/2525583596 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54440c1c50 con 0x55883396aca0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5460bf6740 10 client.25820 fetch_fsmap learned FSMap version 27 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.424+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558833980680 con 0x55883396aca0 2024-04-20T11:23:53.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5440c00640 1 -- 172.21.15.44:0/2525583596 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x5588338d79e0 con 0x55883396aca0 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5440c00640 1 -- 172.21.15.44:0/2525583596 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f544403c030 con 0x55883396aca0 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5460bf6740 10 client.25820 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5460bf6740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5460bf6740 20 client.25820 populate_metadata read hostname 'smithi044' 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5588339f2e60 0x5588339f5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5460bf6740 4 client.25820 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.425+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558833853450 con 0x5588339f2e60 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.426+0000 7f5457400640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5588339f2e60 0x5588339f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5440c00640 10 client.25820 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5457400640 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5588339f2e60 0x5588339f5280 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c827a31023e9b06c server_cookie=91616f88e23fb945 in_seq=0 out_seq=0 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5440c00640 1 -- 172.21.15.44:0/2525583596 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558833853450 con 0x5588339f2e60 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5440c00640 10 client.25820 handle_command_reply: tid=0 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 client.25820 shutdown 2024-04-20T11:23:53.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 msgr2=0x558833977420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 0x558833977420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54440ef140 tx=0x7f54440bd4f0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 shutdown_connections 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5588339f2e60 0x5588339f5280 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558833973880 0x55883397ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55883396aca0 0x558833977420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 --2- 172.21.15.44:0/2525583596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55883396a380 0x558833976ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 >> 172.21.15.44:0/2525583596 conn(0x558833964ab0 msgr2=0x558833970ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5440c00640 0 client.25820 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 shutdown_connections 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.427+0000 7f5460bf6740 1 -- 172.21.15.44:0/2525583596 wait complete. 2024-04-20T11:23:53.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.428+0000 7f5460bf6740 20 client.25820 trim_cache size 0 max 16384 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: "858adc39-36fd-4a0c-b15f-d55389b27222": { 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "858adc39-36fd-4a0c-b15f-d55389b27222", 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:23:53.433 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.428+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f543c0a3f50 msgr2=0x7f543c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.428+0000 7f545ca00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f543c0a3f50 0x7f543c0a6410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x558833969e50 tx=0x7f54480046c0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.428+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 msgr2=0x7f545819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.428+0000 7f545ca00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 0x7f545819e9a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5444037040 tx=0x7f54440045b0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 shutdown_connections 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f543c0a3f50 0x7f543c0a6410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5458198610 0x7f5458198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5458107140 0x7f545819e9a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 --2- 172.21.15.44:0/1651688647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5458102070 0x7f545819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:23:53.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 >> 172.21.15.44:0/1651688647 conn(0x7f54580fd930 msgr2=0x7f54580ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:23:53.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 shutdown_connections 2024-04-20T11:23:53.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:23:53.429+0000 7f545ca00640 1 -- 172.21.15.44:0/1651688647 wait complete. 2024-04-20T11:23:53.453 INFO:tasks.cephfs.filesystem:scrub status for tag:858adc39-36fd-4a0c-b15f-d55389b27222 - {'path': '/', 'tag': '858adc39-36fd-4a0c-b15f-d55389b27222', 'options': 'recursive,force'} 2024-04-20T11:23:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:53 smithi078 ceph-mon[21464]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-20T11:23:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:53 smithi165 ceph-mon[21975]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-20T11:23:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:54 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:23:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:54 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:54 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:23:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:54 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:54.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:54 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:23:54.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:54 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:54.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:54 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:23:54.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:54 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:54 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:23:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:54 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:54 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:23:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:54 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:23:55.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:55 smithi078 ceph-mon[21464]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-20T11:23:55.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:55 smithi044 ceph-mon[16855]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-20T11:23:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:55 smithi165 ceph-mon[21975]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-20T11:23:57.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:57 smithi078 ceph-mon[21464]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:57.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:57 smithi044 ceph-mon[16855]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:57 smithi165 ceph-mon[21975]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:23:58.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:58 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:23:58.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:58 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:23:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:58 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:23:59.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:23:59 smithi078 ceph-mon[21464]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:23:59.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:23:59 smithi044 ceph-mon[16855]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:23:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:23:59 smithi165 ceph-mon[21975]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:24:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:01 smithi078 ceph-mon[21464]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:24:01.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:01 smithi044 ceph-mon[16855]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:24:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:01 smithi165 ceph-mon[21975]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:24:03.531 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:03 smithi165 ceph-mon[21975]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-20T11:24:03.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:03 smithi078 ceph-mon[21464]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-20T11:24:03.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:03 smithi044 ceph-mon[16855]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-20T11:24:05.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:05 smithi078 ceph-mon[21464]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-20T11:24:05.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:05 smithi044 ceph-mon[16855]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-20T11:24:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:05 smithi165 ceph-mon[21975]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-20T11:24:07.445 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:07 smithi044 ceph-mon[16855]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:24:07.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:07 smithi078 ceph-mon[21464]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:24:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:07 smithi165 ceph-mon[21975]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:24:09.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:09 smithi078 ceph-mon[21464]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-20T11:24:09.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:09 smithi078 ceph-mon[21464]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-20T11:24:09.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:09 smithi044 ceph-mon[16855]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-20T11:24:09.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:09 smithi044 ceph-mon[16855]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-20T11:24:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:09 smithi165 ceph-mon[21975]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-20T11:24:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:09 smithi165 ceph-mon[21975]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-20T11:24:11.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:11 smithi078 ceph-mon[21464]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-20T11:24:11.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:11 smithi044 ceph-mon[16855]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-20T11:24:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:11 smithi165 ceph-mon[21975]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-20T11:24:13.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:13 smithi078 ceph-mon[21464]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-20T11:24:13.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:13 smithi044 ceph-mon[16855]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-20T11:24:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:13 smithi165 ceph-mon[21975]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-20T11:24:15.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:15 smithi078 ceph-mon[21464]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-04-20T11:24:15.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:15 smithi044 ceph-mon[16855]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-04-20T11:24:15.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:15 smithi165 ceph-mon[21975]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-04-20T11:24:17.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:17 smithi078 ceph-mon[21464]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-20T11:24:17.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:17 smithi044 ceph-mon[16855]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-20T11:24:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:17 smithi165 ceph-mon[21975]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-20T11:24:17.975 INFO:tasks.workunit.client.0.smithi044.stderr:Updating files: 58% (7224/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 95% (11734/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T11:24:19.036 INFO:tasks.workunit.client.0.smithi044.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T11:24:19.036 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.036 INFO:tasks.workunit.client.0.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: git switch -c 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr:Or undo this operation with: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: git switch - 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T11:24:19.037 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-20T11:24:19.038 INFO:tasks.workunit.client.0.smithi044.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T11:24:19.051 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T11:24:19.109 INFO:tasks.workunit.client.0.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T11:24:19.116 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T11:24:19.116 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T11:24:19.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:19 smithi078 ceph-mon[21464]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-20T11:24:19.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:19 smithi044 ceph-mon[16855]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-20T11:24:19.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:19 smithi165 ceph-mon[21975]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-20T11:24:19.991 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T11:24:20.098 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T11:24:20.184 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T11:24:20.190 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T11:24:20.190 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T11:24:20.282 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T11:24:20.291 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:24:20.291 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T11:24:20.343 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-04-20T11:24:20.344 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-04-20T11:24:20.344 DEBUG:teuthology.orchestra.run.smithi044:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-04-20T11:24:20.422 INFO:tasks.workunit.client.0.smithi044.stderr:+ KERNEL=linux-4.0.5 2024-04-20T11:24:20.422 INFO:tasks.workunit.client.0.smithi044.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-04-20T11:24:21.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:21 smithi078 ceph-mon[21464]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-20T11:24:21.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:21 smithi044 ceph-mon[16855]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-20T11:24:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:21 smithi165 ceph-mon[21975]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-20T11:24:23.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:24:23.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:24:23.509 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir untar_tar 2024-04-20T11:24:23.514 INFO:tasks.workunit.client.0.smithi044.stderr:+ cd untar_tar 2024-04-20T11:24:23.514 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-04-20T11:24:23.550 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ 2024-04-20T11:24:23.553 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/00-INDEX 2024-04-20T11:24:23.555 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/ 2024-04-20T11:24:23.557 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/README 2024-04-20T11:24:23.559 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-04-20T11:24:23.560 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-04-20T11:24:23.562 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-04-20T11:24:23.564 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-04-20T11:24:23.566 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-04-20T11:24:23.568 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-04-20T11:24:23.570 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-04-20T11:24:23.572 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-04-20T11:24:23.574 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-04-20T11:24:23.576 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-04-20T11:24:23.578 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-04-20T11:24:23.580 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-04-20T11:24:23.583 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-04-20T11:24:23.585 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-04-20T11:24:23.588 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-04-20T11:24:23.590 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-04-20T11:24:23.592 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-04-20T11:24:23.593 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-04-20T11:24:23.595 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-04-20T11:24:23.597 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-04-20T11:24:23.600 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-04-20T11:24:23.601 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-04-20T11:24:23.603 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-04-20T11:24:23.605 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-04-20T11:24:23.607 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-04-20T11:24:23.609 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-04-20T11:24:23.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-04-20T11:24:23.613 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-04-20T11:24:23.615 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-04-20T11:24:23.617 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-04-20T11:24:23.619 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-04-20T11:24:23.622 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-04-20T11:24:23.624 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-04-20T11:24:23.626 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-04-20T11:24:23.628 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-04-20T11:24:23.631 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-04-20T11:24:23.633 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-04-20T11:24:23.635 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-04-20T11:24:23.637 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-04-20T11:24:23.641 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-04-20T11:24:23.642 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-04-20T11:24:23.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-04-20T11:24:23.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-04-20T11:24:23.648 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-04-20T11:24:23.651 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-04-20T11:24:23.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-04-20T11:24:23.655 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-04-20T11:24:23.657 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-04-20T11:24:23.659 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-04-20T11:24:23.662 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-04-20T11:24:23.664 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-04-20T11:24:23.666 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-04-20T11:24:23.668 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-04-20T11:24:23.671 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-04-20T11:24:23.673 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-04-20T11:24:23.675 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-04-20T11:24:23.677 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-04-20T11:24:23.679 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-04-20T11:24:23.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.681+0000 7fa068800640 1 -- 172.21.15.44:0/154398051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06006b180 msgr2=0x7fa060106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.681+0000 7fa068800640 1 --2- 172.21.15.44:0/154398051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06006b180 0x7fa060106bb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa050009900 tx=0x7fa05002bc30 comp rx=0 tx=0).stop 2024-04-20T11:24:23.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 -- 172.21.15.44:0/154398051 shutdown_connections 2024-04-20T11:24:23.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 --2- 172.21.15.44:0/154398051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060107290 0x7fa06010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 --2- 172.21.15.44:0/154398051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06006b180 0x7fa060106bb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 --2- 172.21.15.44:0/154398051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06006a7d0 0x7fa06006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 -- 172.21.15.44:0/154398051 >> 172.21.15.44:0/154398051 conn(0x7fa060076c60 msgr2=0x7fa060077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.682 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-04-20T11:24:23.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 -- 172.21.15.44:0/154398051 shutdown_connections 2024-04-20T11:24:23.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.682+0000 7fa068800640 1 -- 172.21.15.44:0/154398051 wait complete. 2024-04-20T11:24:23.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 Processor -- start 2024-04-20T11:24:23.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 -- start start 2024-04-20T11:24:23.683 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06006a7d0 0x7fa06019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa060107290 0x7fa06019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 0x7fa060198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601123f0 con 0x7fa060107290 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa060112270 con 0x7fa060198750 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.683+0000 7fa068800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa060112570 con 0x7fa06006a7d0 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 0x7fa060198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa065400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa060107290 0x7fa06019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 0x7fa060198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:52978/0 (socket says 172.21.15.44:52978) 2024-04-20T11:24:23.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 -- 172.21.15.44:0/1156437646 learned_addr learned my addr 172.21.15.44:0/1156437646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa065e00640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06006a7d0 0x7fa06019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 -- 172.21.15.44:0/1156437646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06006a7d0 msgr2=0x7fa06019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06006a7d0 0x7fa06019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 -- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa060107290 msgr2=0x7fa06019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa060107290 0x7fa06019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 -- 172.21.15.44:0/1156437646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa060199350 con 0x7fa060198750 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.684+0000 7fa066800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 0x7fa060198bd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00efd0 tx=0x7fa05c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.685+0000 7fa05aa00640 1 -- 172.21.15.44:0/1156437646 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c003b70 con 0x7fa060198750 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.685+0000 7fa05aa00640 1 -- 172.21.15.44:0/1156437646 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c003d10 con 0x7fa060198750 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.685+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0601a53e0 con 0x7fa060198750 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.685+0000 7fa05aa00640 1 -- 172.21.15.44:0/1156437646 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c010a90 con 0x7fa060198750 2024-04-20T11:24:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.685+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a5790 con 0x7fa060198750 2024-04-20T11:24:23.687 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-04-20T11:24:23.687 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-04-20T11:24:23.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:23 smithi078 ceph-mon[21464]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-20T11:24:23.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.688+0000 7fa05aa00640 1 -- 172.21.15.44:0/1156437646 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa05c004020 con 0x7fa060198750 2024-04-20T11:24:23.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.688+0000 7fa06b5cf740 1 Processor -- start 2024-04-20T11:24:23.688 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-04-20T11:24:23.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.688+0000 7fa06b5cf740 1 -- start start 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f450802390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45080f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450811ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f450768150 con 0x55f450802960 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f45070e370 con 0x55f45080fab0 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f450764110 con 0x55f450801fb0 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45080f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45080f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40842/0 (socket says 172.21.15.44:40842) 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065e00640 1 -- 172.21.15.44:0/4127323865 learned_addr learned my addr 172.21.15.44:0/4127323865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065400640 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450811ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065400640 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f450802390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065e00640 1 -- 172.21.15.44:0/4127323865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 msgr2=0x55f450802390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.689+0000 7fa065e00640 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f450802390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa065e00640 1 -- 172.21.15.44:0/4127323865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 msgr2=0x55f450811ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa065e00640 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450811ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa065e00640 1 -- 172.21.15.44:0/4127323865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4505447d0 con 0x55f450802960 2024-04-20T11:24:23.691 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa065e00640 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45080f570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa054009cf0 tx=0x7fa054011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d9c98e0f19393e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa05aa00640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0480a3da0 0x7fa0480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa04f000640 1 -- 172.21.15.44:0/4127323865 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054004230 con 0x55f450802960 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa04f000640 1 -- 172.21.15.44:0/4127323865 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa0540043d0 con 0x55f450802960 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa05aa00640 1 -- 172.21.15.44:0/1156437646 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa05c0d0b50 con 0x7fa060198750 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa04f000640 1 -- 172.21.15.44:0/4127323865 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054019a20 con 0x55f450802960 2024-04-20T11:24:23.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.690+0000 7fa065400640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0480a3da0 0x7fa0480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.693 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 -- 172.21.15.44:0/4127323865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 msgr2=0x55f45080f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45080f570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa054009cf0 tx=0x7fa054011810 comp rx=0 tx=0).stop 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa065400640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0480a3da0 0x7fa0480a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa050009230 tx=0x7fa050005e80 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 -- 172.21.15.44:0/4127323865 shutdown_connections 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450811ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45080f570 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/4127323865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f450802390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 -- 172.21.15.44:0/4127323865 >> 172.21.15.44:0/4127323865 conn(0x55f4507fc8d0 msgr2=0x55f4507fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 -- 172.21.15.44:0/4127323865 shutdown_connections 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.691+0000 7fa06b5cf740 1 -- 172.21.15.44:0/4127323865 wait complete. 2024-04-20T11:24:23.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.692+0000 7fa06b5cf740 1 -- start start 2024-04-20T11:24:23.695 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-04-20T11:24:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.692+0000 7fa06b5cf740 1 Processor -- start 2024-04-20T11:24:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.692+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f45080cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f45080d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f450812870 con 0x55f450801fb0 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f450721740 con 0x55f45080fab0 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4508129f0 con 0x55f450802960 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa06b5cf740 10 client.0 _ll_register_callbacks cb 0x55f45071cb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53008/0 (socket says 172.21.15.44:53008) 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 -- 172.21.15.44:0/1290481460 learned_addr learned my addr 172.21.15.44:0/1290481460 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa065e00640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f45080cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa065400640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f45080d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 -- 172.21.15.44:0/1290481460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 msgr2=0x55f45080d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f45080d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 -- 172.21.15.44:0/1290481460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 msgr2=0x55f45080cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f45080cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.693+0000 7fa066800640 1 -- 172.21.15.44:0/1290481460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f45047a3f0 con 0x55f45080fab0 2024-04-20T11:24:23.698 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-04-20T11:24:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa066800640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080d540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00cba0 tx=0x7fa05c09c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca3808d8a0bcb7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa043400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa043400640 1 -- 172.21.15.44:0/1290481460 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0b6060 con 0x55f45080fab0 2024-04-20T11:24:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa043400640 1 -- 172.21.15.44:0/1290481460 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa05c004310 con 0x55f45080fab0 2024-04-20T11:24:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0540043d0 con 0x55f45080fab0 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa043400640 1 -- 172.21.15.44:0/1290481460 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa05c09d260 con 0x55f45080fab0 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa06b5cf740 10 client.25832 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f45043a0f0 con 0x55f45080fab0 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa043400640 1 -- 172.21.15.44:0/1290481460 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa0540043d0 con 0x55f45080fab0 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.694+0000 7fa043400640 1 -- 172.21.15.44:0/1290481460 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fa05c01f020 con 0x55f45080fab0 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa06b5cf740 10 client.25832 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa06b5cf740 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:23.700 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-04-20T11:24:23.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa06b5cf740 20 client.25832 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:23.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f45081fe20 0x55f450822210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa06b5cf740 4 client.25832 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:23.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f450775b70 con 0x55f45081fe20 2024-04-20T11:24:23.701 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:23 smithi044 ceph-mon[16855]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-20T11:24:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa065e00640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f45081fe20 0x55f450822210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa065e00640 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f45081fe20 0x55f450822210 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960bc936e761b45a server_cookie=dd325610b29c0df in_seq=0 out_seq=0 2024-04-20T11:24:23.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.695+0000 7fa043400640 10 client.25832 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:23.702 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-04-20T11:24:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.696+0000 7fa043400640 1 -- 172.21.15.44:0/1290481460 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f450775b70 con 0x55f45081fe20 2024-04-20T11:24:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.696+0000 7fa043400640 10 client.25832 handle_command_reply: tid=0 2024-04-20T11:24:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.696+0000 7fa06b5cf740 1 client.25832 shutdown 2024-04-20T11:24:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 msgr2=0x55f45080d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080d540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00cba0 tx=0x7fa05c09c4c0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 shutdown_connections 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f45081fe20 0x55f450822210 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080d540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f45080d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1290481460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f45080cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa043400640 0 client.25832 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 >> 172.21.15.44:0/1290481460 conn(0x55f4507fc8d0 msgr2=0x55f4507fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.704 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-04-20T11:24:23.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 shutdown_connections 2024-04-20T11:24:23.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1290481460 wait complete. 2024-04-20T11:24:23.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.697+0000 7fa06b5cf740 20 client.25832 trim_cache size 0 max 16384 2024-04-20T11:24:23.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-04-20T11:24:23.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-04-20T11:24:23.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-04-20T11:24:23.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-04-20T11:24:23.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-04-20T11:24:23.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-04-20T11:24:23.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-04-20T11:24:23.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-04-20T11:24:23.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-04-20T11:24:23.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-04-20T11:24:23.709 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-04-20T11:24:23.710 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-04-20T11:24:23.711 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-04-20T11:24:23.712 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-04-20T11:24:23.713 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-04-20T11:24:23.715 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-04-20T11:24:23.716 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-04-20T11:24:23.717 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 Processor -- start 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 -- start start 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f45080f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45081b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601ab980 con 0x55f450802960 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601ad4c0 con 0x55f45080fab0 2024-04-20T11:24:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.718+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601ac050 con 0x55f450801fb0 2024-04-20T11:24:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53012/0 (socket says 172.21.15.44:53012) 2024-04-20T11:24:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 -- 172.21.15.44:0/1223140090 learned_addr learned my addr 172.21.15.44:0/1223140090 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa065400640 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45081b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.719 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-04-20T11:24:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa065e00640 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f45080f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 -- 172.21.15.44:0/1223140090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 msgr2=0x55f45080f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f45080f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 -- 172.21.15.44:0/1223140090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 msgr2=0x55f45081b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45081b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 -- 172.21.15.44:0/1223140090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f450822840 con 0x55f45080fab0 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.719+0000 7fa066800640 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080ba40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0ce3c0 tx=0x7fa05c09c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecba239fae3f760 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa04e600640 1 -- 172.21.15.44:0/1223140090 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0b6040 con 0x55f45080fab0 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa04e600640 1 -- 172.21.15.44:0/1223140090 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c0a57d0 con 0x55f45080fab0 2024-04-20T11:24:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa04e600640 1 -- 172.21.15.44:0/1223140090 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c09c000 con 0x55f45080fab0 2024-04-20T11:24:23.721 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-04-20T11:24:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1223140090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 msgr2=0x55f45080ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080ba40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0ce3c0 tx=0x7fa05c09c5f0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1223140090 shutdown_connections 2024-04-20T11:24:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f45080ba40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450802960 0x55f45081b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1223140090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450801fb0 0x55f45080f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.720+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1223140090 >> 172.21.15.44:0/1223140090 conn(0x55f4507fc8d0 msgr2=0x55f4507fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.721+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1223140090 shutdown_connections 2024-04-20T11:24:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.721+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1223140090 wait complete. 2024-04-20T11:24:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.721+0000 7fa06b5cf740 1 -- start start 2024-04-20T11:24:23.723 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-04-20T11:24:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.721+0000 7fa06b5cf740 1 Processor -- start 2024-04-20T11:24:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f450817c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f4508181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450814c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601acdf0 con 0x55f450801fb0 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601ac720 con 0x55f45080fab0 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601a9970 con 0x55f450802960 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa06b5cf740 10 client.0 _ll_register_callbacks cb 0x7fa05c010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450814c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa065e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f450817c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa065e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f450817c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40878/0 (socket says 172.21.15.44:40878) 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa065e00640 1 -- 172.21.15.44:0/1036060779 learned_addr learned my addr 172.21.15.44:0/1036060779 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa066800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450814c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53024/0 (socket says 172.21.15.44:53024) 2024-04-20T11:24:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.722+0000 7fa065400640 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f4508181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.726 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa066800640 1 -- 172.21.15.44:0/1036060779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 msgr2=0x55f4508181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa066800640 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f4508181c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa066800640 1 -- 172.21.15.44:0/1036060779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 msgr2=0x55f450817c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa066800640 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f450817c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa066800640 1 -- 172.21.15.44:0/1036060779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f450811ea0 con 0x55f45080fab0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa066800640 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450814c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0ce910 tx=0x7fa05c099400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af7d0af536c34b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa043400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa043400640 1 -- 172.21.15.44:0/1036060779 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0b6040 con 0x55f45080fab0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa043400640 1 -- 172.21.15.44:0/1036060779 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa05c09c1b0 con 0x55f45080fab0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c0a57d0 con 0x55f45080fab0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa043400640 1 -- 172.21.15.44:0/1036060779 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa05c09c930 con 0x55f45080fab0 2024-04-20T11:24:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa06b5cf740 10 client.25838 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.723+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4508018e0 con 0x55f45080fab0 2024-04-20T11:24:23.728 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa043400640 1 -- 172.21.15.44:0/1036060779 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c0a57d0 con 0x55f45080fab0 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa043400640 1 -- 172.21.15.44:0/1036060779 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fa05c082070 con 0x55f45080fab0 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa06b5cf740 10 client.25838 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa06b5cf740 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa06b5cf740 20 client.25838 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f450886e70 0x55f450889260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa06b5cf740 4 client.25838 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f45076f9e0 con 0x55f450886e70 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.724+0000 7fa065e00640 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f450886e70 0x55f450889260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa065e00640 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f450886e70 0x55f450889260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94093473008265a2 server_cookie=cd719eab070cc1a7 in_seq=0 out_seq=0 2024-04-20T11:24:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa043400640 10 client.25838 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:23.730 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-04-20T11:24:23.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:23 smithi165 ceph-mon[21975]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-20T11:24:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa043400640 1 -- 172.21.15.44:0/1036060779 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f45076f9e0 con 0x55f450886e70 2024-04-20T11:24:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa043400640 10 client.25838 handle_command_reply: tid=0 2024-04-20T11:24:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 client.25838 shutdown 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 msgr2=0x55f450814c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450814c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0ce910 tx=0x7fa05c099400 comp rx=0 tx=0).stop 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 shutdown_connections 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f450886e70 0x55f450889260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f45080fab0 0x55f450814c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f450802960 0x55f4508181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 --2- 172.21.15.44:0/1036060779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f450801fb0 0x55f450817c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa043400640 0 client.25838 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 >> 172.21.15.44:0/1036060779 conn(0x55f4507fc8d0 msgr2=0x55f45080ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 shutdown_connections 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 1 -- 172.21.15.44:0/1036060779 wait complete. 2024-04-20T11:24:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.725+0000 7fa06b5cf740 20 client.25838 trim_cache size 0 max 16384 2024-04-20T11:24:23.733 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:24:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:24:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:24:23.733 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:24:23.733 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0480a3da0 msgr2=0x7fa0480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0480a3da0 0x7fa0480a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa050009230 tx=0x7fa050005e80 comp rx=0 tx=0).stop 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 msgr2=0x7fa060198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 0x7fa060198bd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00efd0 tx=0x7fa05c00c6a0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 shutdown_connections 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa060198750 0x7fa060198bd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0480a3da0 0x7fa0480a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa060107290 0x7fa06019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 --2- 172.21.15.44:0/1156437646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06006a7d0 0x7fa06019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 >> 172.21.15.44:0/1156437646 conn(0x7fa060076c60 msgr2=0x7fa0600ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 shutdown_connections 2024-04-20T11:24:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.726+0000 7fa068800640 1 -- 172.21.15.44:0/1156437646 wait complete. 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-04-20T11:24:23.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-04-20T11:24:23.737 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-04-20T11:24:23.738 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-04-20T11:24:23.739 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-04-20T11:24:23.740 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-04-20T11:24:23.742 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-04-20T11:24:23.744 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-04-20T11:24:23.745 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:24:23.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:24:23.746 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-04-20T11:24:23.747 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-04-20T11:24:23.751 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-04-20T11:24:23.751 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-04-20T11:24:23.751 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-04-20T11:24:23.753 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-04-20T11:24:23.754 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-04-20T11:24:23.756 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-04-20T11:24:23.759 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-04-20T11:24:23.761 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-04-20T11:24:23.763 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-04-20T11:24:23.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-04-20T11:24:23.766 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-04-20T11:24:23.768 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-04-20T11:24:23.770 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-04-20T11:24:23.771 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-04-20T11:24:23.774 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-04-20T11:24:23.775 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-04-20T11:24:23.777 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-04-20T11:24:23.778 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-04-20T11:24:23.779 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-04-20T11:24:23.781 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-04-20T11:24:23.782 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-04-20T11:24:23.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-04-20T11:24:23.784 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-04-20T11:24:23.785 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-04-20T11:24:23.786 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-04-20T11:24:23.788 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-04-20T11:24:23.789 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-04-20T11:24:23.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-04-20T11:24:23.791 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-04-20T11:24:23.792 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-04-20T11:24:23.793 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-04-20T11:24:23.794 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-04-20T11:24:23.795 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-04-20T11:24:23.796 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-04-20T11:24:23.798 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-04-20T11:24:23.799 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-04-20T11:24:23.800 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-04-20T11:24:23.801 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-04-20T11:24:23.802 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-04-20T11:24:23.804 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-04-20T11:24:23.805 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-04-20T11:24:23.806 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-04-20T11:24:23.807 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-04-20T11:24:23.808 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-04-20T11:24:23.809 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-04-20T11:24:23.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-04-20T11:24:23.812 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-04-20T11:24:23.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-04-20T11:24:23.814 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-04-20T11:24:23.815 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-04-20T11:24:23.818 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-04-20T11:24:23.820 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-04-20T11:24:23.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-04-20T11:24:23.822 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-04-20T11:24:23.823 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-04-20T11:24:23.824 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-04-20T11:24:23.825 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-04-20T11:24:23.827 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-04-20T11:24:23.828 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-04-20T11:24:23.829 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-04-20T11:24:23.830 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-04-20T11:24:23.832 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-04-20T11:24:23.833 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-04-20T11:24:23.835 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-04-20T11:24:23.836 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-04-20T11:24:23.838 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-04-20T11:24:23.840 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-04-20T11:24:23.841 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-04-20T11:24:23.843 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-04-20T11:24:23.844 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-04-20T11:24:23.846 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-04-20T11:24:23.848 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-04-20T11:24:23.849 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-04-20T11:24:23.851 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-04-20T11:24:23.852 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-04-20T11:24:23.854 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-04-20T11:24:23.856 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-04-20T11:24:23.858 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-04-20T11:24:23.859 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-04-20T11:24:23.861 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-04-20T11:24:23.862 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-04-20T11:24:23.864 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-04-20T11:24:23.866 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-04-20T11:24:23.867 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-04-20T11:24:23.869 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-04-20T11:24:23.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.940+0000 7fbf7f200640 1 -- 172.21.15.44:0/4163266445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78107110 msgr2=0x7fbf78069630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.940+0000 7fbf7f200640 1 --2- 172.21.15.44:0/4163266445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78107110 0x7fbf78069630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009900 tx=0x7fbf6802bc50 comp rx=0 tx=0).stop 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 -- 172.21.15.44:0/4163266445 shutdown_connections 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 --2- 172.21.15.44:0/4163266445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf781047f0 0x7fbf78104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 --2- 172.21.15.44:0/4163266445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78107110 0x7fbf78069630 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 --2- 172.21.15.44:0/4163266445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf78106ca0 0x7fbf780690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 -- 172.21.15.44:0/4163266445 >> 172.21.15.44:0/4163266445 conn(0x7fbf780767a0 msgr2=0x7fbf78076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 -- 172.21.15.44:0/4163266445 shutdown_connections 2024-04-20T11:24:23.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.941+0000 7fbf7f200640 1 -- 172.21.15.44:0/4163266445 wait complete. 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.942+0000 7fbf7f200640 1 Processor -- start 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.942+0000 7fbf7f200640 1 -- start start 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.942+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf781047f0 0x7fbf781a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.942+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 0x7fbf781a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf78107110 0x7fbf7819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf781187e0 con 0x7fbf78106ca0 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78118660 con 0x7fbf78107110 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78118960 con 0x7fbf781047f0 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 0x7fbf781a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 0x7fbf781a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40902/0 (socket says 172.21.15.44:40902) 2024-04-20T11:24:23.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 -- 172.21.15.44:0/1101987692 learned_addr learned my addr 172.21.15.44:0/1101987692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7de00640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf781047f0 0x7fbf781a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7e800640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf78107110 0x7fbf7819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 -- 172.21.15.44:0/1101987692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf781047f0 msgr2=0x7fbf781a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf781047f0 0x7fbf781a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 -- 172.21.15.44:0/1101987692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf78107110 msgr2=0x7fbf7819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf78107110 0x7fbf7819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7d400640 1 -- 172.21.15.44:0/1101987692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7819f220 con 0x7fbf78106ca0 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7de00640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf781047f0 0x7fbf781a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:23.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.943+0000 7fbf7e800640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf78107110 0x7fbf7819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.944+0000 7fbf7d400640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 0x7fbf781a4e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009230 tx=0x7fbf6802dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.944+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68043020 con 0x7fbf78106ca0 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.944+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf68004250 con 0x7fbf78106ca0 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.944+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003b50 con 0x7fbf78106ca0 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.944+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7819f510 con 0x7fbf78106ca0 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.944+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781ab7b0 con 0x7fbf78106ca0 2024-04-20T11:24:23.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.945+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf78068ca0 con 0x7fbf78106ca0 2024-04-20T11:24:23.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.947+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbf68046020 con 0x7fbf78106ca0 2024-04-20T11:24:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.950+0000 7fbf6ea00640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf640a3d20 0x7fbf640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.950+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbf680f26f0 con 0x7fbf78106ca0 2024-04-20T11:24:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.950+0000 7fbf7de00640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf640a3d20 0x7fbf640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.951+0000 7fbf7de00640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf640a3d20 0x7fbf640a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf7819fbe0 tx=0x7fbf70009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:23.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:23.955+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf6804d020 con 0x7fbf78106ca0 2024-04-20T11:24:24.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.202+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf781aba70 con 0x7fbf78106ca0 2024-04-20T11:24:24.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.203+0000 7fbf6ea00640 1 -- 172.21.15.44:0/1101987692 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fbf680beeb0 con 0x7fbf78106ca0 2024-04-20T11:24:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:24:24.204 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:22:35.546711+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:24:24.204 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-04-20T11:24:24.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf640a3d20 msgr2=0x7fbf640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf640a3d20 0x7fbf640a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf7819fbe0 tx=0x7fbf70009500 comp rx=0 tx=0).stop 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 msgr2=0x7fbf781a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 0x7fbf781a4e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009230 tx=0x7fbf6802dd20 comp rx=0 tx=0).stop 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 shutdown_connections 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf78107110 0x7fbf7819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbf640a3d20 0x7fbf640a61e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf78106ca0 0x7fbf781a4e30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 --2- 172.21.15.44:0/1101987692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf781047f0 0x7fbf781a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 >> 172.21.15.44:0/1101987692 conn(0x7fbf780767a0 msgr2=0x7fbf78101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.206+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 shutdown_connections 2024-04-20T11:24:24.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.207+0000 7fbf7f200640 1 -- 172.21.15.44:0/1101987692 wait complete. 2024-04-20T11:24:24.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:24:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.490+0000 7fc5fd600640 1 -- 172.21.15.44:0/2656389727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f8102a30 msgr2=0x7fc5f8106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.490+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2656389727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f8102a30 0x7fc5f8106a70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009900 tx=0x7fc5e002bc30 comp rx=0 tx=0).stop 2024-04-20T11:24:24.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 -- 172.21.15.44:0/2656389727 shutdown_connections 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2656389727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8107150 0x7fc5f810ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2656389727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f8102a30 0x7fc5f8106a70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2656389727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8102080 0x7fc5f8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 -- 172.21.15.44:0/2656389727 >> 172.21.15.44:0/2656389727 conn(0x7fc5f80fd900 msgr2=0x7fc5f80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 -- 172.21.15.44:0/2656389727 shutdown_connections 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.491+0000 7fc5fd600640 1 -- 172.21.15.44:0/2656389727 wait complete. 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 Processor -- start 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 -- start start 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8102080 0x7fc5f81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 0x7fc5f81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f819a810 0x7fc5f819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81122d0 con 0x7fc5f8107150 2024-04-20T11:24:24.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8112150 con 0x7fc5f819a810 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.492+0000 7fc5fd600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8112450 con 0x7fc5f8102080 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 0x7fc5f81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5fcc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f819a810 0x7fc5f819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 0x7fc5f81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40912/0 (socket says 172.21.15.44:40912) 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8102080 0x7fc5f81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 -- 172.21.15.44:0/2124630423 learned_addr learned my addr 172.21.15.44:0/2124630423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 -- 172.21.15.44:0/2124630423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8102080 msgr2=0x7fc5f81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8102080 0x7fc5f81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 -- 172.21.15.44:0/2124630423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f819a810 msgr2=0x7fc5f819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f819a810 0x7fc5f819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.493+0000 7fc5efe00640 1 -- 172.21.15.44:0/2124630423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f819b3a0 con 0x7fc5f8107150 2024-04-20T11:24:24.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.494+0000 7fc5efe00640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 0x7fc5f81a0be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009230 tx=0x7fc5e0004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.494+0000 7fc5f5600640 1 -- 172.21.15.44:0/2124630423 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0043020 con 0x7fc5f8107150 2024-04-20T11:24:24.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.494+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f819b630 con 0x7fc5f8107150 2024-04-20T11:24:24.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.494+0000 7fc5f5600640 1 -- 172.21.15.44:0/2124630423 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5e0004e50 con 0x7fc5f8107150 2024-04-20T11:24:24.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.494+0000 7fc5f5600640 1 -- 172.21.15.44:0/2124630423 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0003a10 con 0x7fc5f8107150 2024-04-20T11:24:24.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.494+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81a7810 con 0x7fc5f8107150 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc5f5600640 1 -- 172.21.15.44:0/2124630423 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc5e0046020 con 0x7fc5f8107150 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 Processor -- start 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 -- start start 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c3a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c43130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555c63b9f050 con 0x555c63c3aa90 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555c63b45490 con 0x555c63c3a170 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc601982740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555c63b9b010 con 0x555c63c43670 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.498+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c43130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c43130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40924/0 (socket says 172.21.15.44:40924) 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 -- 172.21.15.44:0/2460683037 learned_addr learned my addr 172.21.15.44:0/2460683037 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c3a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 -- 172.21.15.44:0/2460683037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 msgr2=0x555c63c45a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c45a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 -- 172.21.15.44:0/2460683037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 msgr2=0x555c63c3a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c3a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 -- 172.21.15.44:0/2460683037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c63b4d070 con 0x555c63c3aa90 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f5600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc5dc0a3d20 0x7fc5dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c43130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009cf0 tx=0x7fc5e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8699f074ae8d74c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5f5600640 1 -- 172.21.15.44:0/2124630423 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc5e00f2570 con 0x7fc5f8107150 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.499+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc5dc0a3d20 0x7fc5dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.500+0000 7fc5ed200640 1 -- 172.21.15.44:0/2460683037 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e801e070 con 0x555c63c3aa90 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.500+0000 7fc5ed200640 1 -- 172.21.15.44:0/2460683037 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5e8019b30 con 0x555c63c3aa90 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.500+0000 7fc5ed200640 1 -- 172.21.15.44:0/2460683037 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800eb20 con 0x555c63c3aa90 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 -- 172.21.15.44:0/2460683037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 msgr2=0x555c63c43130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc5dc0a3d20 0x7fc5dc0a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0005fd0 tx=0x7fc5f0009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c43130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009cf0 tx=0x7fc5e8011810 comp rx=0 tx=0).stop 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 -- 172.21.15.44:0/2460683037 shutdown_connections 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c45a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c43130 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 --2- 172.21.15.44:0/2460683037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c3a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 -- 172.21.15.44:0/2460683037 >> 172.21.15.44:0/2460683037 conn(0x555c63c348a0 msgr2=0x555c63c34c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 -- 172.21.15.44:0/2460683037 shutdown_connections 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 -- 172.21.15.44:0/2460683037 wait complete. 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.501+0000 7fc601982740 1 -- start start 2024-04-20T11:24:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 Processor -- start 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3a170 0x555c63c40680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3aa90 0x555c63c40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 0x555c63c41100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555c63c46460 con 0x555c63c43670 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555c63b58860 con 0x555c63c3aa90 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555c63c465e0 con 0x555c63c3a170 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc601982740 10 client.0 _ll_register_callbacks cb 0x555c63b26fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.502+0000 7fc5fcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 0x555c63c41100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5fcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 0x555c63c41100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40928/0 (socket says 172.21.15.44:40928) 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3a170 0x555c63c40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5fcc00640 1 -- 172.21.15.44:0/1605830912 learned_addr learned my addr 172.21.15.44:0/1605830912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3aa90 0x555c63c40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 -- 172.21.15.44:0/1605830912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3aa90 msgr2=0x555c63c40bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3aa90 0x555c63c40bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 -- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 msgr2=0x555c63c41100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 0x555c63c41100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 -- 172.21.15.44:0/1605830912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c63b5c950 con 0x555c63c3a170 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 0x555c63c41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3a170 0x555c63c40680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8011d10 tx=0x7fc5e8019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f52ef9f7e994afb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.503+0000 7fc5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc5e5600640 1 -- 172.21.15.44:0/1605830912 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e801e050 con 0x555c63c3a170 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e8019b30 con 0x555c63c3a170 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc5e5600640 1 -- 172.21.15.44:0/1605830912 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc5e800eaa0 con 0x555c63c3a170 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc5e5600640 1 -- 172.21.15.44:0/1605830912 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5e8033050 con 0x555c63c3a170 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc601982740 10 client.25861 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:24.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c63b2a820 con 0x555c63c3a170 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.504+0000 7fc5e5600640 1 -- 172.21.15.44:0/1605830912 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5e8019b30 con 0x555c63c3a170 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc5e5600640 1 -- 172.21.15.44:0/1605830912 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fc5e80046c0 con 0x555c63c3a170 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc601982740 10 client.25861 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc601982740 10 client.25861 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc601982740 20 client.25861 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc601982740 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63c53580 0x555c63c55970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc601982740 4 client.25861 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c63baca70 con 0x555c63c53580 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.505+0000 7fc5efe00640 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63c53580 0x555c63c55970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.506+0000 7fc5efe00640 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63c53580 0x555c63c55970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=241bc92627f73f52 server_cookie=c68fd8b4017c1c32 in_seq=0 out_seq=0 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.506+0000 7fc5e5600640 10 client.25861 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.506+0000 7fc5e5600640 1 -- 172.21.15.44:0/1605830912 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c63baca70 con 0x555c63c53580 2024-04-20T11:24:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.506+0000 7fc5e5600640 10 client.25861 handle_command_reply: tid=0 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.506+0000 7fc601982740 1 client.25861 shutdown 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3a170 msgr2=0x555c63c40680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3a170 0x555c63c40680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8011d10 tx=0x7fc5e8019660 comp rx=0 tx=0).stop 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 shutdown_connections 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c43670 0x555c63c41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63c53580 0x555c63c55970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3aa90 0x555c63c40bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 --2- 172.21.15.44:0/1605830912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3a170 0x555c63c40680 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 >> 172.21.15.44:0/1605830912 conn(0x555c63c348a0 msgr2=0x555c63c365f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc5e5600640 0 client.25861 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 shutdown_connections 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 1 -- 172.21.15.44:0/1605830912 wait complete. 2024-04-20T11:24:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.507+0000 7fc601982740 20 client.25861 trim_cache size 0 max 16384 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 Processor -- start 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 -- start start 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 0x555c63c42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3aa90 0x555c63c4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c43670 0x555c63c5b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81afc60 con 0x555c63c3a170 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b17a0 con 0x555c63c43670 2024-04-20T11:24:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.522+0000 7fc601982740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b0330 con 0x555c63c3aa90 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 0x555c63c42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 0x555c63c42db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40944/0 (socket says 172.21.15.44:40944) 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 -- 172.21.15.44:0/207126575 learned_addr learned my addr 172.21.15.44:0/207126575 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5efe00640 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3aa90 0x555c63c4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c43670 0x555c63c5b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 -- 172.21.15.44:0/207126575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3aa90 msgr2=0x555c63c4f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3aa90 0x555c63c4f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 -- 172.21.15.44:0/207126575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c43670 msgr2=0x555c63c5b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c43670 0x555c63c5b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 -- 172.21.15.44:0/207126575 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c63c55ce0 con 0x555c63c3a170 2024-04-20T11:24:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c43670 0x555c63c5b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.523+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 0x555c63c42db0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e8019a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba45e42d9c35ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc5e7e00640 1 -- 172.21.15.44:0/207126575 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8015070 con 0x555c63c3a170 2024-04-20T11:24:24.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc5e7e00640 1 -- 172.21.15.44:0/207126575 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5e802a070 con 0x555c63c3a170 2024-04-20T11:24:24.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc5e7e00640 1 -- 172.21.15.44:0/207126575 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e801e040 con 0x555c63c3a170 2024-04-20T11:24:24.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 -- 172.21.15.44:0/207126575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 msgr2=0x555c63c42db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 0x555c63c42db0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e8019a60 comp rx=0 tx=0).stop 2024-04-20T11:24:24.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 -- 172.21.15.44:0/207126575 shutdown_connections 2024-04-20T11:24:24.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c43670 0x555c63c5b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c3aa90 0x555c63c4f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 --2- 172.21.15.44:0/207126575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3a170 0x555c63c42db0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 -- 172.21.15.44:0/207126575 >> 172.21.15.44:0/207126575 conn(0x555c63c348a0 msgr2=0x555c63c3f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 -- 172.21.15.44:0/207126575 shutdown_connections 2024-04-20T11:24:24.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.524+0000 7fc601982740 1 -- 172.21.15.44:0/207126575 wait complete. 2024-04-20T11:24:24.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 -- start start 2024-04-20T11:24:24.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 Processor -- start 2024-04-20T11:24:24.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c4ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c3cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b10d0 con 0x555c63c3aa90 2024-04-20T11:24:24.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81b0a00 con 0x555c63c3a170 2024-04-20T11:24:24.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.525+0000 7fc601982740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81adc30 con 0x555c63c43670 2024-04-20T11:24:24.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc601982740 10 client.0 _ll_register_callbacks cb 0x7fc5e00f5ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:24.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5fcc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5fcc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c3d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:37070/0 (socket says 172.21.15.44:37070) 2024-04-20T11:24:24.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5fcc00640 1 -- 172.21.15.44:0/3612976725 learned_addr learned my addr 172.21.15.44:0/3612976725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c3cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c4ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 -- 172.21.15.44:0/3612976725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 msgr2=0x555c63c3d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c3d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 -- 172.21.15.44:0/3612976725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 msgr2=0x555c63c4ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c4ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 -- 172.21.15.44:0/3612976725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c638b13f0 con 0x555c63c3aa90 2024-04-20T11:24:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5fcc00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c3d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5efe00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c3cd00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00efde0 tx=0x7fc5e00f7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc0136917781f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.526+0000 7fc5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc5e5600640 1 -- 172.21.15.44:0/3612976725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00be0b0 con 0x555c63c3aa90 2024-04-20T11:24:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc5e5600640 1 -- 172.21.15.44:0/3612976725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc5e00badc0 con 0x555c63c3aa90 2024-04-20T11:24:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e802a070 con 0x555c63c3aa90 2024-04-20T11:24:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc5e5600640 1 -- 172.21.15.44:0/3612976725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5e00bee50 con 0x555c63c3aa90 2024-04-20T11:24:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 10 client.15738 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:24.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c6397b7d0 con 0x555c63c3aa90 2024-04-20T11:24:24.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc5e5600640 1 -- 172.21.15.44:0/3612976725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5e802a070 con 0x555c63c3aa90 2024-04-20T11:24:24.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc5e5600640 1 -- 172.21.15.44:0/3612976725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fc5e00be9b0 con 0x555c63c3aa90 2024-04-20T11:24:24.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:24.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 10 client.15738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:24:24.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 20 client.15738 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:24.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.527+0000 7fc601982740 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63cb9ac0 0x555c63cbbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.528+0000 7fc601982740 4 client.15738 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:24.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.528+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c63ba68e0 con 0x555c63cb9ac0 2024-04-20T11:24:24.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.528+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63cb9ac0 0x555c63cbbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.528+0000 7fc5f7e00640 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63cb9ac0 0x555c63cbbee0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab7fd4107c7a7b60 server_cookie=57d2d123c3f74117 in_seq=0 out_seq=0 2024-04-20T11:24:24.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.528+0000 7fc5e5600640 10 client.15738 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:24.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc5e5600640 1 -- 172.21.15.44:0/3612976725 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c63ba68e0 con 0x555c63cb9ac0 2024-04-20T11:24:24.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc5e5600640 10 client.15738 handle_command_reply: tid=0 2024-04-20T11:24:24.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 client.15738 shutdown 2024-04-20T11:24:24.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 msgr2=0x555c63c3cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c3cd00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00efde0 tx=0x7fc5e00f7040 comp rx=0 tx=0).stop 2024-04-20T11:24:24.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 shutdown_connections 2024-04-20T11:24:24.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555c63c43670 0x555c63c3d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c63c3aa90 0x555c63c3cd00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x555c63cb9ac0 0x555c63cbbee0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 --2- 172.21.15.44:0/3612976725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555c63c3a170 0x555c63c4ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc5e5600640 0 client.15738 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:24.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 >> 172.21.15.44:0/3612976725 conn(0x555c63c348a0 msgr2=0x555c63c40620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 shutdown_connections 2024-04-20T11:24:24.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 1 -- 172.21.15.44:0/3612976725 wait complete. 2024-04-20T11:24:24.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.529+0000 7fc601982740 20 client.15738 trim_cache size 0 max 16384 2024-04-20T11:24:24.544 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:24.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc5dc0a3d20 msgr2=0x7fc5dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc5dc0a3d20 0x7fc5dc0a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0005fd0 tx=0x7fc5f0009500 comp rx=0 tx=0).stop 2024-04-20T11:24:24.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 msgr2=0x7fc5f81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 0x7fc5f81a0be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009230 tx=0x7fc5e0004c30 comp rx=0 tx=0).stop 2024-04-20T11:24:24.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 shutdown_connections 2024-04-20T11:24:24.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f819a810 0x7fc5f819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc5dc0a3d20 0x7fc5dc0a61e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f8107150 0x7fc5f81a0be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 --2- 172.21.15.44:0/2124630423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8102080 0x7fc5f81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 >> 172.21.15.44:0/2124630423 conn(0x7fc5f80fd900 msgr2=0x7fc5f80ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 shutdown_connections 2024-04-20T11:24:24.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.530+0000 7fc5fd600640 1 -- 172.21.15.44:0/2124630423 wait complete. 2024-04-20T11:24:24.548 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:24 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1101987692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:24:24.550 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:24:24.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:24 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1101987692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:24:24.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:24 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1101987692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 -- 172.21.15.44:0/2809081583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108f50 msgr2=0x7fc6201153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 --2- 172.21.15.44:0/2809081583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108f50 0x7fc6201153c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc618009c10 tx=0x7fc61802bef0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 -- 172.21.15.44:0/2809081583 shutdown_connections 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 --2- 172.21.15.44:0/2809081583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108f50 0x7fc6201153c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 --2- 172.21.15.44:0/2809081583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc620108590 0x7fc620108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 --2- 172.21.15.44:0/2809081583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62010e600 0x7fc62010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 -- 172.21.15.44:0/2809081583 >> 172.21.15.44:0/2809081583 conn(0x7fc6200764a0 msgr2=0x7fc6200788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 -- 172.21.15.44:0/2809081583 shutdown_connections 2024-04-20T11:24:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.763+0000 7fc626200640 1 -- 172.21.15.44:0/2809081583 wait complete. 2024-04-20T11:24:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.764+0000 7fc626200640 1 Processor -- start 2024-04-20T11:24:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 -- start start 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108590 0x7fc62019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620108f50 0x7fc62019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 0x7fc6201a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc620118a50 con 0x7fc62010e600 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201188d0 con 0x7fc620108f50 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc626200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc620118bd0 con 0x7fc620108590 2024-04-20T11:24:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.765+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 0x7fc6201a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 0x7fc6201a36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40960/0 (socket says 172.21.15.44:40960) 2024-04-20T11:24:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 -- 172.21.15.44:0/3540780877 learned_addr learned my addr 172.21.15.44:0/3540780877 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61ea00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620108f50 0x7fc62019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61f400640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108590 0x7fc62019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 -- 172.21.15.44:0/3540780877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108590 msgr2=0x7fc62019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108590 0x7fc62019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 -- 172.21.15.44:0/3540780877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620108f50 msgr2=0x7fc62019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620108f50 0x7fc62019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 -- 172.21.15.44:0/3540780877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201a3e20 con 0x7fc62010e600 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.766+0000 7fc61fe00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 0x7fc6201a36f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc61802c430 tx=0x7fc618033720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.767+0000 7fc617e00640 1 -- 172.21.15.44:0/3540780877 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618043020 con 0x7fc62010e600 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.767+0000 7fc617e00640 1 -- 172.21.15.44:0/3540780877 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc618004040 con 0x7fc62010e600 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.767+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201a40b0 con 0x7fc62010e600 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.767+0000 7fc617e00640 1 -- 172.21.15.44:0/3540780877 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61803e6b0 con 0x7fc62010e600 2024-04-20T11:24:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.767+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201ab9a0 con 0x7fc62010e600 2024-04-20T11:24:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.770+0000 7fc617e00640 1 -- 172.21.15.44:0/3540780877 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc618046020 con 0x7fc62010e600 2024-04-20T11:24:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 Processor -- start 2024-04-20T11:24:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 -- start start 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dbfb0 0x55cf602dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 0x55cf602e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cf60241050 con 0x55cf602e54b0 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cf601e7490 con 0x55cf602dc8d0 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc628fe0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cf6023d010 con 0x55cf602dbfb0 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.771+0000 7fc61f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 0x55cf602e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 0x55cf602e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53112/0 (socket says 172.21.15.44:53112) 2024-04-20T11:24:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 -- 172.21.15.44:0/858908303 learned_addr learned my addr 172.21.15.44:0/858908303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61ea00640 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dbfb0 0x55cf602dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 -- 172.21.15.44:0/858908303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dbfb0 msgr2=0x55cf602dc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dbfb0 0x55cf602dc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 -- 172.21.15.44:0/858908303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 msgr2=0x55cf602e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc617e00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6040a3da0 0x7fc6040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 -- 172.21.15.44:0/858908303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf601ef070 con 0x55cf602dc8d0 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61ea00640 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc617e00640 1 -- 172.21.15.44:0/3540780877 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc6180f2b20 con 0x7fc62010e600 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61fe00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6040a3da0 0x7fc6040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.772+0000 7fc61f400640 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 0x55cf602e4f70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc610009cf0 tx=0x7fc610011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147a2911ee546bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.773+0000 7fc60fe00640 1 -- 172.21.15.44:0/858908303 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610004230 con 0x55cf602dc8d0 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.773+0000 7fc60fe00640 1 -- 172.21.15.44:0/858908303 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc6100043d0 con 0x55cf602dc8d0 2024-04-20T11:24:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.773+0000 7fc60fe00640 1 -- 172.21.15.44:0/858908303 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610019900 con 0x55cf602dc8d0 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.773+0000 7fc628fe0740 1 -- 172.21.15.44:0/858908303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 msgr2=0x55cf602e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.773+0000 7fc61fe00640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6040a3da0 0x7fc6040a6260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc61802c7a0 tx=0x7fc61802c710 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.773+0000 7fc628fe0740 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 0x55cf602e4f70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc610009cf0 tx=0x7fc610011810 comp rx=0 tx=0).stop 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 -- 172.21.15.44:0/858908303 shutdown_connections 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dc8d0 0x55cf602e4f70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 --2- 172.21.15.44:0/858908303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dbfb0 0x55cf602dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 -- 172.21.15.44:0/858908303 >> 172.21.15.44:0/858908303 conn(0x55cf602d66e0 msgr2=0x55cf602d6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 -- 172.21.15.44:0/858908303 shutdown_connections 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 -- 172.21.15.44:0/858908303 wait complete. 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 -- start start 2024-04-20T11:24:24.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.774+0000 7fc628fe0740 1 Processor -- start 2024-04-20T11:24:24.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cf602e82a0 con 0x55cf602e54b0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cf601fa860 con 0x55cf602dbfb0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cf602e8420 con 0x55cf602dc8d0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc628fe0740 10 client.0 _ll_register_callbacks cb 0x55cf601c8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc61f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53126/0 (socket says 172.21.15.44:53126) 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc61f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602e2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:37090/0 (socket says 172.21.15.44:37090) 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc61ea00640 1 -- 172.21.15.44:0/1005561379 learned_addr learned my addr 172.21.15.44:0/1005561379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.775+0000 7fc61ea00640 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc61f400640 1 -- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 msgr2=0x55cf602e2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc61f400640 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc61f400640 1 -- 172.21.15.44:0/1005561379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 msgr2=0x55cf602e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc61f400640 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc61f400640 1 -- 172.21.15.44:0/1005561379 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf601fe950 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc61f400640 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602e2a40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc610009cf0 tx=0x7fc610005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f199e991990c66d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.776+0000 7fc60cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc60cc00640 1 -- 172.21.15.44:0/1005561379 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610027070 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6100043d0 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc60cc00640 1 -- 172.21.15.44:0/1005561379 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc6100037f0 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc60cc00640 1 -- 172.21.15.44:0/1005561379 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc610033050 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc628fe0740 10 client.25879 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf601cc820 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.777+0000 7fc60cc00640 1 -- 172.21.15.44:0/1005561379 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc6100043d0 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc60cc00640 1 -- 172.21.15.44:0/1005561379 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fc6100039f0 con 0x55cf602dc8d0 2024-04-20T11:24:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc628fe0740 10 client.25879 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc628fe0740 10 client.25879 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc628fe0740 20 client.25879 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc628fe0740 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf602f5490 0x55cf602f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc628fe0740 4 client.25879 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6024ea70 con 0x55cf602f5490 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.778+0000 7fc61ea00640 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf602f5490 0x55cf602f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.779+0000 7fc61ea00640 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf602f5490 0x55cf602f7880 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e694da22d2bffea server_cookie=77c0096d6257853b in_seq=0 out_seq=0 2024-04-20T11:24:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.779+0000 7fc60cc00640 10 client.25879 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc60cc00640 1 -- 172.21.15.44:0/1005561379 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf6024ea70 con 0x55cf602f5490 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc60cc00640 10 client.25879 handle_command_reply: tid=0 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 client.25879 shutdown 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 msgr2=0x55cf602e2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602e2a40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc610009cf0 tx=0x7fc610005270 comp rx=0 tx=0).stop 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 shutdown_connections 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf602f5490 0x55cf602f7880 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602e2a40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 --2- 172.21.15.44:0/1005561379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 >> 172.21.15.44:0/1005561379 conn(0x55cf602d66e0 msgr2=0x55cf602d8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc60cc00640 0 client.25879 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 shutdown_connections 2024-04-20T11:24:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 1 -- 172.21.15.44:0/1005561379 wait complete. 2024-04-20T11:24:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.780+0000 7fc628fe0740 20 client.25879 trim_cache size 0 max 16384 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.795+0000 7fc628fe0740 1 Processor -- start 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.795+0000 7fc628fe0740 1 -- start start 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.795+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.795+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.795+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc628fe0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b4150 con 0x55cf602e54b0 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc628fe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b5c90 con 0x55cf602dbfb0 2024-04-20T11:24:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc628fe0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b4820 con 0x55cf602dc8d0 2024-04-20T11:24:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53132/0 (socket says 172.21.15.44:53132) 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 -- 172.21.15.44:0/4221370934 learned_addr learned my addr 172.21.15.44:0/4221370934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61f400640 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 -- 172.21.15.44:0/4221370934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 msgr2=0x55cf602f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 -- 172.21.15.44:0/4221370934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 msgr2=0x55cf602fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.796+0000 7fc61ea00640 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.797+0000 7fc61ea00640 1 -- 172.21.15.44:0/4221370934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5ff533f0 con 0x55cf602dbfb0 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.797+0000 7fc61f400640 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.797+0000 7fc61ea00640 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e4c30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc608009ec0 tx=0x7fc608009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21256bc1bb22b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.797+0000 7fc60f400640 1 -- 172.21.15.44:0/4221370934 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080264a0 con 0x55cf602dbfb0 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.797+0000 7fc60f400640 1 -- 172.21.15.44:0/4221370934 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc608005520 con 0x55cf602dbfb0 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.797+0000 7fc60f400640 1 -- 172.21.15.44:0/4221370934 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004a00 con 0x55cf602dbfb0 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 -- 172.21.15.44:0/4221370934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 msgr2=0x55cf602e4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e4c30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc608009ec0 tx=0x7fc608009620 comp rx=0 tx=0).stop 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 -- 172.21.15.44:0/4221370934 shutdown_connections 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602e54b0 0x55cf602fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 --2- 172.21.15.44:0/4221370934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602dbfb0 0x55cf602e4c30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 -- 172.21.15.44:0/4221370934 >> 172.21.15.44:0/4221370934 conn(0x55cf602d66e0 msgr2=0x55cf602dd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 -- 172.21.15.44:0/4221370934 shutdown_connections 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 -- 172.21.15.44:0/4221370934 wait complete. 2024-04-20T11:24:24.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.798+0000 7fc628fe0740 1 -- start start 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 Processor -- start 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602dbfb0 0x55cf602ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602e54b0 0x55cf602debe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b55c0 con 0x55cf602dbfb0 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b4ef0 con 0x55cf602e54b0 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201b2180 con 0x55cf602dc8d0 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc628fe0740 10 client.0 _ll_register_callbacks cb 0x7fc610009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.799+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602dbfb0 0x55cf602ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602dbfb0 0x55cf602ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41004/0 (socket says 172.21.15.44:41004) 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61ea00640 1 -- 172.21.15.44:0/812092001 learned_addr learned my addr 172.21.15.44:0/812092001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61ea00640 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602e54b0 0x55cf602debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 -- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602e54b0 msgr2=0x55cf602debe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602e54b0 0x55cf602debe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 -- 172.21.15.44:0/812092001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602dbfb0 msgr2=0x55cf602ed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602dbfb0 0x55cf602ed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 -- 172.21.15.44:0/812092001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6001d7d0 con 0x55cf602dc8d0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc61f400640 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602edbe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc610009250 tx=0x7fc610019890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52114f4c94d3b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.800+0000 7fc60cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc60cc00640 1 -- 172.21.15.44:0/812092001 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61003e060 con 0x55cf602dc8d0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc608005520 con 0x55cf602dc8d0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc60cc00640 1 -- 172.21.15.44:0/812092001 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc610019d90 con 0x55cf602dc8d0 2024-04-20T11:24:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc60cc00640 1 -- 172.21.15.44:0/812092001 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc610027040 con 0x55cf602dc8d0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc628fe0740 10 client.25885 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf6035b2a0 con 0x55cf602dc8d0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc60cc00640 1 -- 172.21.15.44:0/812092001 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc608005520 con 0x55cf602dc8d0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.801+0000 7fc60cc00640 1 -- 172.21.15.44:0/812092001 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fc610041020 con 0x55cf602dc8d0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc628fe0740 10 client.25885 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc628fe0740 10 client.25885 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc628fe0740 20 client.25885 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc628fe0740 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6035b9c0 0x55cf6035dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc628fe0740 4 client.25885 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf602488e0 con 0x55cf6035b9c0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc61ea00640 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6035b9c0 0x55cf6035dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.802+0000 7fc61ea00640 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6035b9c0 0x55cf6035dde0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b103952a00605ee7 server_cookie=e1ab1d404b71ee1d in_seq=0 out_seq=0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc60cc00640 10 client.25885 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc60cc00640 1 -- 172.21.15.44:0/812092001 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf602488e0 con 0x55cf6035b9c0 2024-04-20T11:24:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc60cc00640 10 client.25885 handle_command_reply: tid=0 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 client.25885 shutdown 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 msgr2=0x55cf602edbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602edbe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc610009250 tx=0x7fc610019890 comp rx=0 tx=0).stop 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 shutdown_connections 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf602e54b0 0x55cf602debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf602dc8d0 0x55cf602edbe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6035b9c0 0x55cf6035dde0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 --2- 172.21.15.44:0/812092001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf602dbfb0 0x55cf602ed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 >> 172.21.15.44:0/812092001 conn(0x55cf602d66e0 msgr2=0x55cf602d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc60cc00640 0 client.25885 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 shutdown_connections 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 1 -- 172.21.15.44:0/812092001 wait complete. 2024-04-20T11:24:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.803+0000 7fc628fe0740 20 client.25885 trim_cache size 0 max 16384 2024-04-20T11:24:24.807 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6040a3da0 msgr2=0x7fc6040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6040a3da0 0x7fc6040a6260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc61802c7a0 tx=0x7fc61802c710 comp rx=0 tx=0).stop 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 msgr2=0x7fc6201a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 0x7fc6201a36f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc61802c430 tx=0x7fc618033720 comp rx=0 tx=0).stop 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 shutdown_connections 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6040a3da0 0x7fc6040a6260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc62010e600 0x7fc6201a36f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc620108f50 0x7fc62019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 --2- 172.21.15.44:0/3540780877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc620108590 0x7fc62019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 >> 172.21.15.44:0/3540780877 conn(0x7fc6200764a0 msgr2=0x7fc620077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 shutdown_connections 2024-04-20T11:24:24.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:24.804+0000 7fc626200640 1 -- 172.21.15.44:0/3540780877 wait complete. 2024-04-20T11:24:24.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 -- 172.21.15.44:0/2838943050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 msgr2=0x7f1d40115090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 --2- 172.21.15.44:0/2838943050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 0x7f1d40115090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009ca0 tx=0x7f1d3802c100 comp rx=0 tx=0).stop 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 -- 172.21.15.44:0/2838943050 shutdown_connections 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 --2- 172.21.15.44:0/2838943050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 0x7f1d40115090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 --2- 172.21.15.44:0/2838943050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d40076db0 0x7f1d40112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 --2- 172.21.15.44:0/2838943050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d40076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 -- 172.21.15.44:0/2838943050 >> 172.21.15.44:0/2838943050 conn(0x7f1d400ffad0 msgr2=0x7f1d40101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 -- 172.21.15.44:0/2838943050 shutdown_connections 2024-04-20T11:24:25.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.040+0000 7f1d44c00640 1 -- 172.21.15.44:0/2838943050 wait complete. 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.041+0000 7f1d44c00640 1 Processor -- start 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.041+0000 7f1d44c00640 1 -- start start 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d44c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d4019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d44c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d40076db0 0x7f1d4019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d44c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 0x7f1d401a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d44c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40118920 con 0x7f1d40112ca0 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d44c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401187a0 con 0x7f1d40076490 2024-04-20T11:24:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d44c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40118aa0 con 0x7f1d40076db0 2024-04-20T11:24:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d4019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d4019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53154/0 (socket says 172.21.15.44:53154) 2024-04-20T11:24:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d3f400640 1 -- 172.21.15.44:0/3076371492 learned_addr learned my addr 172.21.15.44:0/3076371492 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.042+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 0x7f1d401a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d40076db0 0x7f1d4019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3f400640 1 -- 172.21.15.44:0/3076371492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d40076db0 msgr2=0x7f1d4019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3f400640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d40076db0 0x7f1d4019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3f400640 1 -- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 msgr2=0x7f1d401a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3f400640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 0x7f1d401a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3f400640 1 -- 172.21.15.44:0/3076371492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401a3d60 con 0x7f1d40076490 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.043+0000 7f1d3f400640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d4019edc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000e9f0 tx=0x7f1d3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.044+0000 7f1d37e00640 1 -- 172.21.15.44:0/3076371492 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000cd80 con 0x7f1d40076490 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.044+0000 7f1d37e00640 1 -- 172.21.15.44:0/3076371492 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d30010930 con 0x7f1d40076490 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.044+0000 7f1d37e00640 1 -- 172.21.15.44:0/3076371492 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30018b00 con 0x7f1d40076490 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.044+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401a4050 con 0x7f1d40076490 2024-04-20T11:24:25.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.044+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401ab8e0 con 0x7f1d40076490 2024-04-20T11:24:25.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.047+0000 7f1d48fe5740 1 Processor -- start 2024-04-20T11:24:25.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.047+0000 7f1d48fe5740 1 -- start start 2024-04-20T11:24:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.047+0000 7f1d37e00640 1 -- 172.21.15.44:0/3076371492 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1d30018ca0 con 0x7f1d40076490 2024-04-20T11:24:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902170 0x55d82b902550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902a90 0x55d82b90b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b90da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d82b867050 con 0x55d82b902170 2024-04-20T11:24:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d82b80d490 con 0x55d82b902a90 2024-04-20T11:24:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d82b863010 con 0x55d82b90b670 2024-04-20T11:24:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902170 0x55d82b902550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902a90 0x55d82b90b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b90da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902170 0x55d82b902550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41018/0 (socket says 172.21.15.44:41018) 2024-04-20T11:24:25.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3fe00640 1 -- 172.21.15.44:0/2671027347 learned_addr learned my addr 172.21.15.44:0/2671027347 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2671027347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902a90 msgr2=0x55d82b90b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902a90 0x55d82b90b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2671027347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902170 msgr2=0x55d82b902550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902170 0x55d82b902550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.048+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2671027347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82b815070 con 0x55d82b90b670 2024-04-20T11:24:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b90da60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009bd0 tx=0x7f1d2c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83181820b951ece8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d37e00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d240a3d20 0x7f1d240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d2be00640 1 -- 172.21.15.44:0/2671027347 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004050 con 0x55d82b90b670 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d2be00640 1 -- 172.21.15.44:0/2671027347 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d2c005520 con 0x55d82b90b670 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d2be00640 1 -- 172.21.15.44:0/2671027347 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004d90 con 0x55d82b90b670 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d37e00640 1 -- 172.21.15.44:0/3076371492 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1d300d0ae0 con 0x7f1d40076490 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.049+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d240a3d20 0x7f1d240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2671027347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 msgr2=0x55d82b90da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d240a3d20 0x7f1d240a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d38005e30 tx=0x7f1d38005d80 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b90da60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009bd0 tx=0x7f1d2c011810 comp rx=0 tx=0).stop 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2671027347 shutdown_connections 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b90da60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902a90 0x55d82b90b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2671027347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902170 0x55d82b902550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2671027347 >> 172.21.15.44:0/2671027347 conn(0x55d82b8fc8a0 msgr2=0x55d82b8fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2671027347 shutdown_connections 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.050+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2671027347 wait complete. 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.051+0000 7f1d48fe5740 1 -- start start 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.051+0000 7f1d48fe5740 1 Processor -- start 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902170 0x55d82b908680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b908bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b90b670 0x55d82b909100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d82b90e460 con 0x55d82b902a90 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d82b820860 con 0x55d82b90b670 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d82b90e5e0 con 0x55d82b902170 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d48fe5740 10 client.0 _ll_register_callbacks cb 0x55d82b7eefe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:25.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b908bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b908bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41028/0 (socket says 172.21.15.44:41028) 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 -- 172.21.15.44:0/1262823225 learned_addr learned my addr 172.21.15.44:0/1262823225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3f400640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902170 0x55d82b908680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b90b670 0x55d82b909100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 -- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902170 msgr2=0x55d82b908680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902170 0x55d82b908680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 -- 172.21.15.44:0/1262823225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b90b670 msgr2=0x55d82b909100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b90b670 0x55d82b909100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3ea00640 1 -- 172.21.15.44:0/1262823225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82b824950 con 0x55d82b902a90 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.052+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b90b670 0x55d82b909100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b908bc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009710 tx=0x7f1d2c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedce9a6d771713 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d28c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d28c00640 1 -- 172.21.15.44:0/1262823225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c029070 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d28c00640 1 -- 172.21.15.44:0/1262823225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1d2c0051c0 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2c005520 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d28c00640 1 -- 172.21.15.44:0/1262823225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d2c035050 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d48fe5740 10 client.15768 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.053+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d82b7f2820 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d28c00640 1 -- 172.21.15.44:0/1262823225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d2c005520 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d28c00640 1 -- 172.21.15.44:0/1262823225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f1d2c023430 con 0x55d82b902a90 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d48fe5740 10 client.15768 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d48fe5740 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d48fe5740 20 client.15768 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b91b530 0x55d82b91d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d48fe5740 4 client.15768 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.054+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82b874a70 con 0x55d82b91b530 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.055+0000 7f1d3f400640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b91b530 0x55d82b91d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.055+0000 7f1d3f400640 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b91b530 0x55d82b91d920 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43dc9b89815d9e74 server_cookie=8e0d0823baf0e203 in_seq=0 out_seq=0 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.055+0000 7f1d28c00640 10 client.15768 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d28c00640 1 -- 172.21.15.44:0/1262823225 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d82b874a70 con 0x55d82b91b530 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d28c00640 10 client.15768 handle_command_reply: tid=0 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 client.15768 shutdown 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 msgr2=0x55d82b908bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b908bc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009710 tx=0x7f1d2c004dc0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 shutdown_connections 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b91b530 0x55d82b91d920 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b90b670 0x55d82b909100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b908bc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/1262823225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902170 0x55d82b908680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d28c00640 0 client.15768 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 >> 172.21.15.44:0/1262823225 conn(0x55d82b8fc8a0 msgr2=0x55d82b8fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.056+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 shutdown_connections 2024-04-20T11:24:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.057+0000 7f1d48fe5740 1 -- 172.21.15.44:0/1262823225 wait complete. 2024-04-20T11:24:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.057+0000 7f1d48fe5740 20 client.15768 trim_cache size 0 max 16384 2024-04-20T11:24:25.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.071+0000 7f1d48fe5740 1 Processor -- start 2024-04-20T11:24:25.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.071+0000 7f1d48fe5740 1 -- start start 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.071+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b90adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b917320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b9077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b4020 con 0x55d82b902a90 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b5b60 con 0x55d82b902170 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b46f0 con 0x55d82b90b670 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b917320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b917320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41040/0 (socket says 172.21.15.44:41040) 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2674567878 learned_addr learned my addr 172.21.15.44:0/2674567878 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3f400640 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b90adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b9077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2674567878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 msgr2=0x55d82b9077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b9077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2674567878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 msgr2=0x55d82b90adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b90adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.072+0000 7f1d3ea00640 1 -- 172.21.15.44:0/2674567878 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82b9028c0 con 0x55d82b902a90 2024-04-20T11:24:25.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.073+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b917320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009b20 tx=0x7f1d2c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2337d265bf2b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.073+0000 7f1d2b400640 1 -- 172.21.15.44:0/2674567878 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c032030 con 0x55d82b902a90 2024-04-20T11:24:25.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.073+0000 7f1d2b400640 1 -- 172.21.15.44:0/2674567878 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d2c009800 con 0x55d82b902a90 2024-04-20T11:24:25.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.073+0000 7f1d2b400640 1 -- 172.21.15.44:0/2674567878 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c029050 con 0x55d82b902a90 2024-04-20T11:24:25.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.073+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2674567878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 msgr2=0x55d82b917320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.073+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b917320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009b20 tx=0x7f1d2c004970 comp rx=0 tx=0).stop 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2674567878 shutdown_connections 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b90b670 0x55d82b9077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b902a90 0x55d82b917320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/2674567878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b90adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2674567878 >> 172.21.15.44:0/2674567878 conn(0x55d82b8fc8a0 msgr2=0x55d82b902e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2674567878 shutdown_connections 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 -- 172.21.15.44:0/2674567878 wait complete. 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 -- start start 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.074+0000 7f1d48fe5740 1 Processor -- start 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b913830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902a90 0x55d82b913d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 0x55d82b904ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b5490 con 0x55d82b90b670 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b4dc0 con 0x55d82b902170 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401b1ff0 con 0x55d82b902a90 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d48fe5740 10 client.0 _ll_register_callbacks cb 0x7f1d2c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 0x55d82b904ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 0x55d82b904ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41054/0 (socket says 172.21.15.44:41054) 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3fe00640 1 -- 172.21.15.44:0/656518552 learned_addr learned my addr 172.21.15.44:0/656518552 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b913830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902a90 0x55d82b913d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3f400640 1 -- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902a90 msgr2=0x55d82b913d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3f400640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902a90 0x55d82b913d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3f400640 1 -- 172.21.15.44:0/656518552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 msgr2=0x55d82b904ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.075+0000 7f1d3f400640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 0x55d82b904ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d3f400640 1 -- 172.21.15.44:0/656518552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82b91dc90 con 0x55d82b902170 2024-04-20T11:24:25.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902a90 0x55d82b913d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 0x55d82b904ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d3f400640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b913830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d300ce2a0 tx=0x7f1d300d2ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cef860e2ee277b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d28c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d28c00640 1 -- 172.21.15.44:0/656518552 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300d4480 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d82b86e8e0 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d28c00640 1 -- 172.21.15.44:0/656518552 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1d3009aea0 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d28c00640 1 -- 172.21.15.44:0/656518552 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d3009cba0 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d48fe5740 10 client.25898 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.076+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d82b5793f0 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d28c00640 1 -- 172.21.15.44:0/656518552 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55d82b86e8e0 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d28c00640 1 -- 172.21.15.44:0/656518552 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f1d30099210 con 0x55d82b902170 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d48fe5740 10 client.25898 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d48fe5740 10 client.25898 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:24:25.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d48fe5740 20 client.25898 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b9818e0 0x55d82b983d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d48fe5740 4 client.25898 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d82b7ea5a0 con 0x55d82b9818e0 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.077+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b9818e0 0x55d82b983d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.078+0000 7f1d3ea00640 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b9818e0 0x55d82b983d20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51a138819885f443 server_cookie=9bcb9a0638b1589 in_seq=0 out_seq=0 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.078+0000 7f1d28c00640 10 client.25898 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.078+0000 7f1d28c00640 1 -- 172.21.15.44:0/656518552 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d82b7ea5a0 con 0x55d82b9818e0 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.078+0000 7f1d28c00640 10 client.25898 handle_command_reply: tid=0 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.078+0000 7f1d48fe5740 1 client.25898 shutdown 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 msgr2=0x55d82b913830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b913830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d300ce2a0 tx=0x7f1d300d2ff0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 shutdown_connections 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55d82b9818e0 0x55d82b983d20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d82b90b670 0x55d82b904ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d82b902a90 0x55d82b913d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 --2- 172.21.15.44:0/656518552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d82b902170 0x55d82b913830 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 >> 172.21.15.44:0/656518552 conn(0x55d82b8fc8a0 msgr2=0x55d82b9077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d28c00640 0 client.25898 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 shutdown_connections 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 1 -- 172.21.15.44:0/656518552 wait complete. 2024-04-20T11:24:25.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.079+0000 7f1d48fe5740 20 client.25898 trim_cache size 0 max 16384 2024-04-20T11:24:25.084 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:25.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d240a3d20 msgr2=0x7f1d240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d240a3d20 0x7f1d240a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d38005e30 tx=0x7f1d38005d80 comp rx=0 tx=0).stop 2024-04-20T11:24:25.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 msgr2=0x7f1d4019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d4019edc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000e9f0 tx=0x7f1d3000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 shutdown_connections 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d240a3d20 0x7f1d240a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d40112ca0 0x7f1d401a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d40076db0 0x7f1d4019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 --2- 172.21.15.44:0/3076371492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d40076490 0x7f1d4019edc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 >> 172.21.15.44:0/3076371492 conn(0x7f1d400ffad0 msgr2=0x7f1d40101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 shutdown_connections 2024-04-20T11:24:25.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.080+0000 7f1d44c00640 1 -- 172.21.15.44:0/3076371492 wait complete. 2024-04-20T11:24:25.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:24:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 -- 172.21.15.44:0/3074161343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e54102bd0 msgr2=0x7f3e54106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 --2- 172.21.15.44:0/3074161343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e54102bd0 0x7f3e54106c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:24:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 -- 172.21.15.44:0/3074161343 shutdown_connections 2024-04-20T11:24:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 --2- 172.21.15.44:0/3074161343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e541072f0 0x7f3e5410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 --2- 172.21.15.44:0/3074161343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e54102bd0 0x7f3e54106c10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 --2- 172.21.15.44:0/3074161343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e54102220 0x7f3e54102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 -- 172.21.15.44:0/3074161343 >> 172.21.15.44:0/3074161343 conn(0x7f3e540fdac0 msgr2=0x7f3e540ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.300+0000 7f3e5a800640 1 -- 172.21.15.44:0/3074161343 shutdown_connections 2024-04-20T11:24:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.301+0000 7f3e5a800640 1 -- 172.21.15.44:0/3074161343 wait complete. 2024-04-20T11:24:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.302+0000 7f3e5a800640 1 Processor -- start 2024-04-20T11:24:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.302+0000 7f3e5a800640 1 -- start start 2024-04-20T11:24:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.302+0000 7f3e5a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e54102220 0x7f3e541a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.302+0000 7f3e5a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e541072f0 0x7f3e541a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.302+0000 7f3e5a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 0x7f3e5419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.302+0000 7f3e5a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541124c0 con 0x7f3e5419aa80 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e5a800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54112340 con 0x7f3e541072f0 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e5a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54112640 con 0x7f3e54102220 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 0x7f3e5419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e54102220 0x7f3e541a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 0x7f3e5419af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41084/0 (socket says 172.21.15.44:41084) 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53e00640 1 -- 172.21.15.44:0/1552525934 learned_addr learned my addr 172.21.15.44:0/1552525934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e52a00640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e541072f0 0x7f3e541a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53400640 1 -- 172.21.15.44:0/1552525934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e541072f0 msgr2=0x7f3e541a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53400640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e541072f0 0x7f3e541a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53400640 1 -- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 msgr2=0x7f3e5419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53400640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 0x7f3e5419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53400640 1 -- 172.21.15.44:0/1552525934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5419b5a0 con 0x7f3e54102220 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.303+0000 7f3e53e00640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 0x7f3e5419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.304+0000 7f3e53400640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e54102220 0x7f3e541a0880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400e9f0 tx=0x7f3e4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.304+0000 7f3e4be00640 1 -- 172.21.15.44:0/1552525934 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400cd80 con 0x7f3e54102220 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.304+0000 7f3e4be00640 1 -- 172.21.15.44:0/1552525934 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e440035f0 con 0x7f3e54102220 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.304+0000 7f3e4be00640 1 -- 172.21.15.44:0/1552525934 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44010690 con 0x7f3e54102220 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.304+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5419b890 con 0x7f3e54102220 2024-04-20T11:24:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.304+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e541a7ae0 con 0x7f3e54102220 2024-04-20T11:24:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.307+0000 7f3e5d5df740 1 Processor -- start 2024-04-20T11:24:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.307+0000 7f3e5d5df740 1 -- start start 2024-04-20T11:24:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.307+0000 7f3e4be00640 1 -- 172.21.15.44:0/1552525934 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3e440037a0 con 0x7f3e54102220 2024-04-20T11:24:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 0x55582ad4b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4b8d0 0x55582ad53f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55582acb0050 con 0x55582ad4afb0 2024-04-20T11:24:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55582ac56490 con 0x55582ad4b8d0 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55582acac010 con 0x55582ad544b0 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 0x55582ad4b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 0x55582ad4b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41100/0 (socket says 172.21.15.44:41100) 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e53e00640 1 -- 172.21.15.44:0/2911495859 learned_addr learned my addr 172.21.15.44:0/2911495859 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e52a00640 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.308+0000 7f3e53400640 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4b8d0 0x55582ad53f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53e00640 1 -- 172.21.15.44:0/2911495859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 msgr2=0x55582ad568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53e00640 1 -- 172.21.15.44:0/2911495859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4b8d0 msgr2=0x55582ad53f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4b8d0 0x55582ad53f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53e00640 1 -- 172.21.15.44:0/2911495859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55582ac5e070 con 0x55582ad4afb0 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53400640 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4b8d0 0x55582ad53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 0x55582ad4b390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c011850 tx=0x7f3e4c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9622664d0089a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e48c00640 1 -- 172.21.15.44:0/2911495859 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c011330 con 0x55582ad4afb0 2024-04-20T11:24:25.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e48c00640 1 -- 172.21.15.44:0/2911495859 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e4c010430 con 0x55582ad4afb0 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e48c00640 1 -- 172.21.15.44:0/2911495859 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00fea0 con 0x55582ad4afb0 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.309+0000 7f3e4be00640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e380a3da0 0x7f3e380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e4be00640 1 -- 172.21.15.44:0/1552525934 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3e440d0a00 con 0x7f3e54102220 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e53400640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e380a3da0 0x7f3e380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2911495859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 msgr2=0x55582ad4b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 0x55582ad4b390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c011850 tx=0x7f3e4c011d10 comp rx=0 tx=0).stop 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2911495859 shutdown_connections 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4b8d0 0x55582ad53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2911495859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4afb0 0x55582ad4b390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2911495859 >> 172.21.15.44:0/2911495859 conn(0x55582ad456e0 msgr2=0x55582ad45ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.310+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2911495859 shutdown_connections 2024-04-20T11:24:25.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.311+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2911495859 wait complete. 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.311+0000 7f3e53400640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e380a3da0 0x7f3e380a6260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55582ad4aa80 tx=0x7f3e44098640 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.311+0000 7f3e5d5df740 1 -- start start 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.311+0000 7f3e5d5df740 1 Processor -- start 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad4afb0 0x55582ad51450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad544b0 0x55582ad51ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55582ad572a0 con 0x55582ad4b8d0 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55582ac69860 con 0x55582ad544b0 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55582ad57420 con 0x55582ad4afb0 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e5d5df740 10 client.0 _ll_register_callbacks cb 0x55582ac37fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41108/0 (socket says 172.21.15.44:41108) 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 -- 172.21.15.44:0/2834598490 learned_addr learned my addr 172.21.15.44:0/2834598490 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad4afb0 0x55582ad51450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad544b0 0x55582ad51ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 -- 172.21.15.44:0/2834598490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad4afb0 msgr2=0x55582ad51450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad4afb0 0x55582ad51450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 -- 172.21.15.44:0/2834598490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad544b0 msgr2=0x55582ad51ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad544b0 0x55582ad51ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e52a00640 1 -- 172.21.15.44:0/2834598490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55582ac6d950 con 0x55582ad4b8d0 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.312+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad544b0 0x55582ad51ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e52a00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdb5e9278d0a9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e41600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e41600640 1 -- 172.21.15.44:0/2834598490 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c044070 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4c010430 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e41600640 1 -- 172.21.15.44:0/2834598490 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3e3c004310 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e41600640 1 -- 172.21.15.44:0/2834598490 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e3c050050 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e5d5df740 10 client.15786 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.313+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55582ac3b820 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e41600640 1 -- 172.21.15.44:0/2834598490 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e4c010430 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e41600640 1 -- 172.21.15.44:0/2834598490 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f3e3c004cc0 con 0x55582ad4b8d0 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e5d5df740 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e5d5df740 10 client.15786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e5d5df740 20 client.15786 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582ad643a0 0x55582ad667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e5d5df740 4 client.15786 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:25.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55582acbda70 con 0x55582ad643a0 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.314+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582ad643a0 0x55582ad667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.315+0000 7f3e53e00640 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582ad643a0 0x55582ad667e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f89316259eddaa19 server_cookie=fcb35b9453b94bc5 in_seq=0 out_seq=0 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.315+0000 7f3e41600640 10 client.15786 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e41600640 1 -- 172.21.15.44:0/2834598490 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55582acbda70 con 0x55582ad643a0 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e41600640 10 client.15786 handle_command_reply: tid=0 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 client.15786 shutdown 2024-04-20T11:24:25.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 msgr2=0x55582ad51990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009900 tx=0x7f3e3c02dd60 comp rx=0 tx=0).stop 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 shutdown_connections 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582ad643a0 0x55582ad667e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad544b0 0x55582ad51ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e41600640 0 client.15786 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/2834598490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad4afb0 0x55582ad51450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.316+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 >> 172.21.15.44:0/2834598490 conn(0x55582ad456e0 msgr2=0x55582ad47430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.317+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 shutdown_connections 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.317+0000 7f3e5d5df740 1 -- 172.21.15.44:0/2834598490 wait complete. 2024-04-20T11:24:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.317+0000 7f3e5d5df740 20 client.15786 trim_cache size 0 max 16384 2024-04-20T11:24:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 Processor -- start 2024-04-20T11:24:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 -- start start 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541afe60 con 0x55582ad4b8d0 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541b19a0 con 0x55582ad4afb0 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.333+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541b0530 con 0x55582ad544b0 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53228/0 (socket says 172.21.15.44:53228) 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 -- 172.21.15.44:0/3761382392 learned_addr learned my addr 172.21.15.44:0/3761382392 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e52a00640 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 -- 172.21.15.44:0/3761382392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 msgr2=0x55582ad445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 -- 172.21.15.44:0/3761382392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 msgr2=0x55582ad51cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 -- 172.21.15.44:0/3761382392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55582a9c23f0 con 0x55582ad4afb0 2024-04-20T11:24:25.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.334+0000 7f3e53e00640 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad53b80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c011850 tx=0x7f3e4c010e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6860a4199fea2f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.335+0000 7f3e43e00640 1 -- 172.21.15.44:0/3761382392 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c004a00 con 0x55582ad4afb0 2024-04-20T11:24:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.335+0000 7f3e43e00640 1 -- 172.21.15.44:0/3761382392 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e4c00fe40 con 0x55582ad4afb0 2024-04-20T11:24:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.335+0000 7f3e43e00640 1 -- 172.21.15.44:0/3761382392 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c010930 con 0x55582ad4afb0 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 -- 172.21.15.44:0/3761382392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 msgr2=0x55582ad53b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad53b80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c011850 tx=0x7f3e4c010e50 comp rx=0 tx=0).stop 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 -- 172.21.15.44:0/3761382392 shutdown_connections 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad51cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/3761382392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad53b80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 -- 172.21.15.44:0/3761382392 >> 172.21.15.44:0/3761382392 conn(0x55582ad456e0 msgr2=0x55582ad43eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 -- 172.21.15.44:0/3761382392 shutdown_connections 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 -- 172.21.15.44:0/3761382392 wait complete. 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 -- start start 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.336+0000 7f3e5d5df740 1 Processor -- start 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad5c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad4dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad4e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541b12d0 con 0x55582ad4b8d0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541b0c00 con 0x55582ad4afb0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541ade30 con 0x55582ad544b0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e5d5df740 10 client.0 _ll_register_callbacks cb 0x7f3e3c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad4dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad4dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41126/0 (socket says 172.21.15.44:41126) 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e52a00640 1 -- 172.21.15.44:0/1872186857 learned_addr learned my addr 172.21.15.44:0/1872186857 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e53e00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad5c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.337+0000 7f3e53e00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad4e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e52a00640 1 -- 172.21.15.44:0/1872186857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 msgr2=0x55582ad4e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e52a00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad4e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e52a00640 1 -- 172.21.15.44:0/1872186857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 msgr2=0x55582ad5c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e52a00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad5c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e52a00640 1 -- 172.21.15.44:0/1872186857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55582aa8c7d0 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e53e00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad4e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e52a00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad4dd00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c003640 tx=0x7f3e3c004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd84b44030b48a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e41600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e41600640 1 -- 172.21.15.44:0/1872186857 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c044040 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e41600640 1 -- 172.21.15.44:0/1872186857 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3e3c003a30 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4c00fe40 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e41600640 1 -- 172.21.15.44:0/1872186857 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e3c062050 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e5d5df740 10 client.15792 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.338+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55582adca660 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e41600640 1 -- 172.21.15.44:0/1872186857 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e4c00fe40 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e41600640 1 -- 172.21.15.44:0/1872186857 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f3e3c004bc0 con 0x55582ad4b8d0 2024-04-20T11:24:25.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e5d5df740 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:25.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e5d5df740 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:24:25.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e5d5df740 20 client.15792 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:25.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582adcae00 0x55582adcd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e5d5df740 4 client.15792 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:25.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55582acb78e0 con 0x55582adcae00 2024-04-20T11:24:25.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.339+0000 7f3e53e00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582adcae00 0x55582adcd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e53e00640 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582adcae00 0x55582adcd220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87102dead22e9b5e server_cookie=af9b3fe43494e539 in_seq=0 out_seq=0 2024-04-20T11:24:25.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e41600640 10 client.15792 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:25.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e41600640 1 -- 172.21.15.44:0/1872186857 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55582acb78e0 con 0x55582adcae00 2024-04-20T11:24:25.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e41600640 10 client.15792 handle_command_reply: tid=0 2024-04-20T11:24:25.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e5d5df740 1 client.15792 shutdown 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 msgr2=0x55582ad4dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad4dd00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c003640 tx=0x7f3e3c004540 comp rx=0 tx=0).stop 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.340+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 shutdown_connections 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55582adcae00 0x55582adcd220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55582ad544b0 0x55582ad4e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55582ad4b8d0 0x55582ad4dd00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e41600640 0 client.15792 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 --2- 172.21.15.44:0/1872186857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55582ad4afb0 0x55582ad5c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 >> 172.21.15.44:0/1872186857 conn(0x55582ad456e0 msgr2=0x55582ad44740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 shutdown_connections 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 1 -- 172.21.15.44:0/1872186857 wait complete. 2024-04-20T11:24:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.341+0000 7f3e5d5df740 20 client.15792 trim_cache size 0 max 16384 2024-04-20T11:24:25.345 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:25.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e380a3da0 msgr2=0x7f3e380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e380a3da0 0x7f3e380a6260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55582ad4aa80 tx=0x7f3e44098640 comp rx=0 tx=0).stop 2024-04-20T11:24:25.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e54102220 msgr2=0x7f3e541a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e54102220 0x7f3e541a0880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400e9f0 tx=0x7f3e4400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 shutdown_connections 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3e380a3da0 0x7f3e380a6260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e5419aa80 0x7f3e5419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e541072f0 0x7f3e541a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 --2- 172.21.15.44:0/1552525934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e54102220 0x7f3e541a0880 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 >> 172.21.15.44:0/1552525934 conn(0x7f3e540fdac0 msgr2=0x7f3e540ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 shutdown_connections 2024-04-20T11:24:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.342+0000 7f3e5a800640 1 -- 172.21.15.44:0/1552525934 wait complete. 2024-04-20T11:24:25.356 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:25 smithi044 ceph-mon[16855]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-20T11:24:25.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 -- 172.21.15.44:0/1021736096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984106f60 msgr2=0x7f9984107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 --2- 172.21.15.44:0/1021736096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984106f60 0x7f9984107340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99740098c0 tx=0x7f997402bb70 comp rx=0 tx=0).stop 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 -- 172.21.15.44:0/1021736096 shutdown_connections 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 --2- 172.21.15.44:0/1021736096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984100e90 0x7f998410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 --2- 172.21.15.44:0/1021736096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99841004d0 0x7f9984100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 --2- 172.21.15.44:0/1021736096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984106f60 0x7f9984107340 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 -- 172.21.15.44:0/1021736096 >> 172.21.15.44:0/1021736096 conn(0x7f9984077460 msgr2=0x7f99840fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 -- 172.21.15.44:0/1021736096 shutdown_connections 2024-04-20T11:24:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.576+0000 7f998ae00640 1 -- 172.21.15.44:0/1021736096 wait complete. 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.577+0000 7f998ae00640 1 Processor -- start 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.577+0000 7f998ae00640 1 -- start start 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f998ae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99841004d0 0x7f99841a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f998ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984100e90 0x7f99841a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f998ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 0x7f998419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f998ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9984112500 con 0x7f9984106f60 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f998ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9984112380 con 0x7f9984100e90 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f998ae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9984112680 con 0x7f99841004d0 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 0x7f998419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 0x7f998419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41144/0 (socket says 172.21.15.44:41144) 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 -- 172.21.15.44:0/3082304300 learned_addr learned my addr 172.21.15.44:0/3082304300 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9983e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99841004d0 0x7f99841a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9983400640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984100e90 0x7f99841a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 -- 172.21.15.44:0/3082304300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99841004d0 msgr2=0x7f99841a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99841004d0 0x7f99841a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 -- 172.21.15.44:0/3082304300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984100e90 msgr2=0x7f99841a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984100e90 0x7f99841a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.578+0000 7f9988e00640 1 -- 172.21.15.44:0/3082304300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998419b170 con 0x7f9984106f60 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.579+0000 7f9988e00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 0x7f998419a9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f997c00efc0 tx=0x7f997c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.579+0000 7f9980c00640 1 -- 172.21.15.44:0/3082304300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c003970 con 0x7f9984106f60 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.579+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998419b460 con 0x7f9984106f60 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.579+0000 7f9980c00640 1 -- 172.21.15.44:0/3082304300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f997c004520 con 0x7f9984106f60 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.579+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99841a7700 con 0x7f9984106f60 2024-04-20T11:24:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.579+0000 7f9980c00640 1 -- 172.21.15.44:0/3082304300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c004d40 con 0x7f9984106f60 2024-04-20T11:24:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 Processor -- start 2024-04-20T11:24:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f9980c00640 1 -- 172.21.15.44:0/3082304300 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f997c01d020 con 0x7f9984106f60 2024-04-20T11:24:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 -- start start 2024-04-20T11:24:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c2ac641050 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c2ac5e7490 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.583+0000 7f998da45740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c2ac63d010 con 0x55c2ac6e54b0 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41156/0 (socket says 172.21.15.44:41156) 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 -- 172.21.15.44:0/427712963 learned_addr learned my addr 172.21.15.44:0/427712963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983400640 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983400640 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 -- 172.21.15.44:0/427712963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 msgr2=0x55c2ac6e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 -- 172.21.15.44:0/427712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dbfb0 msgr2=0x55c2ac6dc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6dc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9983e00640 1 -- 172.21.15.44:0/427712963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ac5ef070 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.584+0000 7f9980c00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f99680a3d20 0x7f99680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.585+0000 7f9983e00640 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e4f70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f997402c0b0 tx=0x7f9974009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98dddcd9b850103 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.585+0000 7f9980c00640 1 -- 172.21.15.44:0/3082304300 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f997c0d0770 con 0x7f9984106f60 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.585+0000 7f9983400640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f99680a3d20 0x7f99680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.585+0000 7f9979600640 1 -- 172.21.15.44:0/427712963 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974043020 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.585+0000 7f9979600640 1 -- 172.21.15.44:0/427712963 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f99740323c0 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.585+0000 7f9979600640 1 -- 172.21.15.44:0/427712963 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974039040 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 -- 172.21.15.44:0/427712963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 msgr2=0x55c2ac6e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f9983400640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f99680a3d20 0x7f99680a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c2ac6d7870 tx=0x7f99700096c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e4f70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f997402c0b0 tx=0x7f9974009500 comp rx=0 tx=0).stop 2024-04-20T11:24:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 -- 172.21.15.44:0/427712963 shutdown_connections 2024-04-20T11:24:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e4f70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 --2- 172.21.15.44:0/427712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 -- 172.21.15.44:0/427712963 >> 172.21.15.44:0/427712963 conn(0x55c2ac6d66e0 msgr2=0x55c2ac6d6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 -- 172.21.15.44:0/427712963 shutdown_connections 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.586+0000 7f998da45740 1 -- 172.21.15.44:0/427712963 wait complete. 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 -- start start 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 Processor -- start 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c2ac6e8240 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c2ac5fa860 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c2ac6e83c0 con 0x55c2ac6e54b0 2024-04-20T11:24:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.587+0000 7f998da45740 10 client.0 _ll_register_callbacks cb 0x55c2ac5c8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9988e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41172/0 (socket says 172.21.15.44:41172) 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53266/0 (socket says 172.21.15.44:53266) 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 -- 172.21.15.44:0/1462383533 learned_addr learned my addr 172.21.15.44:0/1462383533 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 -- 172.21.15.44:0/1462383533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 msgr2=0x55c2ac6e2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 -- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 msgr2=0x55c2ac6e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 -- 172.21.15.44:0/1462383533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ac5fe950 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983400640 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f9983e00640 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e2470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9974005b60 tx=0x7f9974034820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3810a9703b8648d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.588+0000 7f996e000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f996e000640 1 -- 172.21.15.44:0/1462383533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974043050 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f996e000640 1 -- 172.21.15.44:0/1462383533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9974004230 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99740323c0 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f996e000640 1 -- 172.21.15.44:0/1462383533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9974059050 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f998da45740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2ac5cc820 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f996e000640 1 -- 172.21.15.44:0/1462383533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f99740323c0 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.589+0000 7f996e000640 1 -- 172.21.15.44:0/1462383533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f99740328e0 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f998da45740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f998da45740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f998da45740 20 client.15804 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f998da45740 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac6f52d0 0x55c2ac6f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f998da45740 4 client.15804 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2ac64ea70 con 0x55c2ac6f52d0 2024-04-20T11:24:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.590+0000 7f9983400640 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac6f52d0 0x55c2ac6f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.591+0000 7f9983400640 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac6f52d0 0x55c2ac6f76c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340618d6a879aa2a server_cookie=c6f9c80cd59595b3 in_seq=0 out_seq=0 2024-04-20T11:24:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.591+0000 7f996e000640 10 client.15804 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.591+0000 7f996e000640 1 -- 172.21.15.44:0/1462383533 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2ac64ea70 con 0x55c2ac6f52d0 2024-04-20T11:24:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.591+0000 7f996e000640 10 client.15804 handle_command_reply: tid=0 2024-04-20T11:24:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.591+0000 7f998da45740 1 client.15804 shutdown 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 msgr2=0x55c2ac6e2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e2470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9974005b60 tx=0x7f9974034820 comp rx=0 tx=0).stop 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 shutdown_connections 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac6f52d0 0x55c2ac6f76c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 --2- 172.21.15.44:0/1462383533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e2470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f996e000640 0 client.15804 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 >> 172.21.15.44:0/1462383533 conn(0x55c2ac6d66e0 msgr2=0x55c2ac6d8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 shutdown_connections 2024-04-20T11:24:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 1 -- 172.21.15.44:0/1462383533 wait complete. 2024-04-20T11:24:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.592+0000 7f998da45740 20 client.15804 trim_cache size 0 max 16384 2024-04-20T11:24:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 Processor -- start 2024-04-20T11:24:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 -- start start 2024-04-20T11:24:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99841afd20 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b1860 con 0x55c2ac6e54b0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f998da45740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b03f0 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.607+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41184/0 (socket says 172.21.15.44:41184) 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 -- 172.21.15.44:0/3486292314 learned_addr learned my addr 172.21.15.44:0/3486292314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983400640 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9988e00640 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 -- 172.21.15.44:0/3486292314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6dc8d0 msgr2=0x55c2ac6f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 -- 172.21.15.44:0/3486292314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6e54b0 msgr2=0x55c2ac6e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 -- 172.21.15.44:0/3486292314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ac3533f0 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9983e00640 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e4ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9974032d50 tx=0x7f9974009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3ead555cd2927e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.608+0000 7f9978c00640 1 -- 172.21.15.44:0/3486292314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974043070 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f9978c00640 1 -- 172.21.15.44:0/3486292314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9974009450 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f9978c00640 1 -- 172.21.15.44:0/3486292314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974039040 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 -- 172.21.15.44:0/3486292314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 msgr2=0x55c2ac6e4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e4ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9974032d50 tx=0x7f9974009860 comp rx=0 tx=0).stop 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 -- 172.21.15.44:0/3486292314 shutdown_connections 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6e54b0 0x55c2ac6e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 --2- 172.21.15.44:0/3486292314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e4ba0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 -- 172.21.15.44:0/3486292314 >> 172.21.15.44:0/3486292314 conn(0x55c2ac6d66e0 msgr2=0x55c2ac6e0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 -- 172.21.15.44:0/3486292314 shutdown_connections 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.609+0000 7f998da45740 1 -- 172.21.15.44:0/3486292314 wait complete. 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 -- start start 2024-04-20T11:24:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 Processor -- start 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac759640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b1190 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99841b0ac0 con 0x55c2ac6dc8d0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99841adcf0 con 0x55c2ac6e54b0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.610+0000 7f998da45740 10 client.0 _ll_register_callbacks cb 0x7f99740096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41194/0 (socket says 172.21.15.44:41194) 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 -- 172.21.15.44:0/1401031264 learned_addr learned my addr 172.21.15.44:0/1401031264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983400640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9988e00640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac759640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 -- 172.21.15.44:0/1401031264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 msgr2=0x55c2ac759640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac759640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 -- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 msgr2=0x55c2ac6e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 -- 172.21.15.44:0/1401031264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ac41d7d0 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983400640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f9983e00640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e8630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9974032dd0 tx=0x7f9974032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7857de9bc051f0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.611+0000 7f996e000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f996e000640 1 -- 172.21.15.44:0/1401031264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974062070 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f996e000640 1 -- 172.21.15.44:0/1401031264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9974004240 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9974009450 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f996e000640 1 -- 172.21.15.44:0/1401031264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9974039070 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f998da45740 10 client.15810 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2ac75ab60 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f996e000640 1 -- 172.21.15.44:0/1401031264 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9974009450 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.612+0000 7f996e000640 1 -- 172.21.15.44:0/1401031264 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7f9974065030 con 0x55c2ac6dbfb0 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f998da45740 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f998da45740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f998da45740 20 client.15810 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f998da45740 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac75aef0 0x55c2ac75d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f998da45740 4 client.15810 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2ac6488e0 con 0x55c2ac75aef0 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f9983400640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac75aef0 0x55c2ac75d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f9983400640 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac75aef0 0x55c2ac75d330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590d805d26985579 server_cookie=6701b4ada5f4b48e in_seq=0 out_seq=0 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.613+0000 7f996e000640 10 client.15810 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f996e000640 1 -- 172.21.15.44:0/1401031264 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2ac6488e0 con 0x55c2ac75aef0 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f996e000640 10 client.15810 handle_command_reply: tid=0 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 client.15810 shutdown 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 msgr2=0x55c2ac6e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e8630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9974032dd0 tx=0x7f9974032fc0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 shutdown_connections 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c2ac75aef0 0x55c2ac75d330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c2ac6e54b0 0x55c2ac759640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2ac6dc8d0 0x55c2ac6e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f996e000640 0 client.15810 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 --2- 172.21.15.44:0/1401031264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2ac6dbfb0 0x55c2ac6e8630 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 >> 172.21.15.44:0/1401031264 conn(0x55c2ac6d66e0 msgr2=0x55c2ac6d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 shutdown_connections 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 1 -- 172.21.15.44:0/1401031264 wait complete. 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.614+0000 7f998da45740 20 client.15810 trim_cache size 0 max 16384 2024-04-20T11:24:25.618 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f99680a3d20 msgr2=0x7f99680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f99680a3d20 0x7f99680a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c2ac6d7870 tx=0x7f99700096c0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 msgr2=0x7f998419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 0x7f998419a9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f997c00efc0 tx=0x7f997c00c800 comp rx=0 tx=0).stop 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 shutdown_connections 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f99680a3d20 0x7f99680a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9984106f60 0x7f998419a9f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9984100e90 0x7f99841a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 --2- 172.21.15.44:0/3082304300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99841004d0 0x7f99841a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 >> 172.21.15.44:0/3082304300 conn(0x7f9984077460 msgr2=0x7f9984105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 shutdown_connections 2024-04-20T11:24:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:25.615+0000 7f998ae00640 1 -- 172.21.15.44:0/3082304300 wait complete. 2024-04-20T11:24:25.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:25 smithi078 ceph-mon[21464]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-20T11:24:25.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:25 smithi165 ceph-mon[21975]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-20T11:24:26.632 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:24:26.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:24:26.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 -- 172.21.15.44:0/837507574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c104410 msgr2=0x7fb06c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 --2- 172.21.15.44:0/837507574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c104410 0x7fb06c1047f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402bc50 comp rx=0 tx=0).stop 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 -- 172.21.15.44:0/837507574 shutdown_connections 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 --2- 172.21.15.44:0/837507574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 --2- 172.21.15.44:0/837507574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104dc0 0x7fb06c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 --2- 172.21.15.44:0/837507574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c104410 0x7fb06c1047f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 -- 172.21.15.44:0/837507574 >> 172.21.15.44:0/837507574 conn(0x7fb06c0ffcf0 msgr2=0x7fb06c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 -- 172.21.15.44:0/837507574 shutdown_connections 2024-04-20T11:24:26.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.871+0000 7fb071c00640 1 -- 172.21.15.44:0/837507574 wait complete. 2024-04-20T11:24:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.872+0000 7fb071c00640 1 Processor -- start 2024-04-20T11:24:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.872+0000 7fb071c00640 1 -- start start 2024-04-20T11:24:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb071c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104410 0x7fb06c1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb071c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb071c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c19aa10 0x7fb06c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb071c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1146b0 con 0x7fb06c1094e0 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb071c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c114530 con 0x7fb06c19aa10 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb071c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c114830 con 0x7fb06c104410 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41202/0 (socket says 172.21.15.44:41202) 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104410 0x7fb06c1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06a000640 1 -- 172.21.15.44:0/8189179 learned_addr learned my addr 172.21.15.44:0/8189179 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06b400640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c19aa10 0x7fb06c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06a000640 1 -- 172.21.15.44:0/8189179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104410 msgr2=0x7fb06c1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06a000640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104410 0x7fb06c1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.873+0000 7fb06a000640 1 -- 172.21.15.44:0/8189179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c19aa10 msgr2=0x7fb06c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb06a000640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c19aa10 0x7fb06c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb06a000640 1 -- 172.21.15.44:0/8189179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c19b530 con 0x7fb06c1094e0 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb06b400640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c19aa10 0x7fb06c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb06a000640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1a0da0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00dbf0 tx=0x7fb05c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb063400640 1 -- 172.21.15.44:0/8189179 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c017070 con 0x7fb06c1094e0 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb063400640 1 -- 172.21.15.44:0/8189179 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb05c004020 con 0x7fb06c1094e0 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c19b820 con 0x7fb06c1094e0 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb063400640 1 -- 172.21.15.44:0/8189179 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0106c0 con 0x7fb06c1094e0 2024-04-20T11:24:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.874+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a7a70 con 0x7fb06c1094e0 2024-04-20T11:24:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.877+0000 7fb074881740 1 Processor -- start 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 -- start start 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb063400640 1 -- 172.21.15.44:0/8189179 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb05c0041d0 con 0x7fb06c1094e0 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c9410 0x55c4dc8cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc824d20 con 0x55c4dc8c0830 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc7cb3e0 con 0x55c4dc8bff10 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb074881740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc820ce0 con 0x55c4dc8c9410 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41218/0 (socket says 172.21.15.44:41218) 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb06aa00640 1 -- 172.21.15.44:0/2576465622 learned_addr learned my addr 172.21.15.44:0/2576465622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.878+0000 7fb06b400640 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06a000640 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c9410 0x55c4dc8cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06aa00640 1 -- 172.21.15.44:0/2576465622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c9410 msgr2=0x55c4dc8cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06aa00640 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c9410 0x55c4dc8cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06aa00640 1 -- 172.21.15.44:0/2576465622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 msgr2=0x55c4dc8c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06aa00640 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06aa00640 1 -- 172.21.15.44:0/2576465622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dc7ea980 con 0x55c4dc8c0830 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb06aa00640 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c8ed0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b858e3b6463b008a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.879+0000 7fb05be00640 1 -- 172.21.15.44:0/2576465622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05402c820 con 0x55c4dc8c0830 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb05be00640 1 -- 172.21.15.44:0/2576465622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb054004040 con 0x55c4dc8c0830 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb05be00640 1 -- 172.21.15.44:0/2576465622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540046c0 con 0x55c4dc8c0830 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb063400640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb0500a3d20 0x7fb0500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb063400640 1 -- 172.21.15.44:0/8189179 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fb05c0d04e0 con 0x7fb06c1094e0 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb06b400640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb0500a3d20 0x7fb0500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 -- 172.21.15.44:0/2576465622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 msgr2=0x55c4dc8c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c8ed0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402c630 comp rx=0 tx=0).stop 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 -- 172.21.15.44:0/2576465622 shutdown_connections 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c9410 0x55c4dc8cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c8ed0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 --2- 172.21.15.44:0/2576465622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.880+0000 7fb074881740 1 -- 172.21.15.44:0/2576465622 >> 172.21.15.44:0/2576465622 conn(0x55c4dc8ba640 msgr2=0x55c4dc8baa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.881+0000 7fb074881740 1 -- 172.21.15.44:0/2576465622 shutdown_connections 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.881+0000 7fb06b400640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb0500a3d20 0x7fb0500a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb064012560 tx=0x7fb064005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.881+0000 7fb074881740 1 -- 172.21.15.44:0/2576465622 wait complete. 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.881+0000 7fb074881740 1 -- start start 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.881+0000 7fb074881740 1 Processor -- start 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc8cc1a0 con 0x55c4dc8c0830 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc7de7a0 con 0x55c4dc8c9410 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c4dc8cc320 con 0x55c4dc8bff10 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb074881740 10 client.0 _ll_register_callbacks cb 0x55c4dc7e0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb06a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb06a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41232/0 (socket says 172.21.15.44:41232) 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb06a000640 1 -- 172.21.15.44:0/218306445 learned_addr learned my addr 172.21.15.44:0/218306445 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb06b400640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.882+0000 7fb06aa00640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb06a000640 1 -- 172.21.15.44:0/218306445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 msgr2=0x55c4dc8c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb06a000640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb06a000640 1 -- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 msgr2=0x55c4dc8c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb06a000640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb06a000640 1 -- 172.21.15.44:0/218306445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4dc5ff7e0 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb06a000640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c6930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0cdca0 tx=0x7fb05c0981e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26d837d1df3bd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb058c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb058c00640 1 -- 172.21.15.44:0/218306445 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c020070 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb058c00640 1 -- 172.21.15.44:0/218306445 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fb05c098bb0 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.883+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb054004040 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb058c00640 1 -- 172.21.15.44:0/218306445 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb05c09c540 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 10 client.15822 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4dc65ff70 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb058c00640 1 -- 172.21.15.44:0/218306445 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb054004040 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb058c00640 1 -- 172.21.15.44:0/218306445 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fb05c0d00b0 con 0x55c4dc8c0830 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 20 client.15822 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc8d96b0 0x55c4dc8dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 4 client.15822 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.884+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4dc832740 con 0x55c4dc8d96b0 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.885+0000 7fb06aa00640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc8d96b0 0x55c4dc8dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.885+0000 7fb06aa00640 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc8d96b0 0x55c4dc8dbae0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1b0bc076d85968 server_cookie=13987bb1b15feb8f in_seq=0 out_seq=0 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.885+0000 7fb058c00640 10 client.15822 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb058c00640 1 -- 172.21.15.44:0/218306445 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4dc832740 con 0x55c4dc8d96b0 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb058c00640 10 client.15822 handle_command_reply: tid=0 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 client.15822 shutdown 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 msgr2=0x55c4dc8c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c6930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0cdca0 tx=0x7fb05c0981e0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 shutdown_connections 2024-04-20T11:24:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc8d96b0 0x55c4dc8dbae0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8c6930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 --2- 172.21.15.44:0/218306445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb058c00640 0 client.15822 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.886+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 >> 172.21.15.44:0/218306445 conn(0x55c4dc8ba640 msgr2=0x55c4dc8bc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.887+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 shutdown_connections 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.887+0000 7fb074881740 1 -- 172.21.15.44:0/218306445 wait complete. 2024-04-20T11:24:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.887+0000 7fb074881740 20 client.15822 trim_cache size 0 max 16384 2024-04-20T11:24:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 Processor -- start 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 -- start start 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c9410 0x55c4dc8e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1afec0 con 0x55c4dc8c9410 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b1a00 con 0x55c4dc8bff10 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.906+0000 7fb074881740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b0590 con 0x55c4dc8c0830 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c9410 0x55c4dc8e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c9410 0x55c4dc8e13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41242/0 (socket says 172.21.15.44:41242) 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06b400640 1 -- 172.21.15.44:0/1754529634 learned_addr learned my addr 172.21.15.44:0/1754529634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06a000640 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06aa00640 1 -- 172.21.15.44:0/1754529634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c0830 msgr2=0x55c4dc8d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06aa00640 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06aa00640 1 -- 172.21.15.44:0/1754529634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c9410 msgr2=0x55c4dc8e13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06aa00640 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c9410 0x55c4dc8e13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.907+0000 7fb06aa00640 1 -- 172.21.15.44:0/1754529634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dc4f7170 con 0x55c4dc8bff10 2024-04-20T11:24:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.908+0000 7fb06aa00640 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c8b20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb054036040 tx=0x7fb054005360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa15598337485c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.908+0000 7fb05b400640 1 -- 172.21.15.44:0/1754529634 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05402c9f0 con 0x55c4dc8bff10 2024-04-20T11:24:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.908+0000 7fb05b400640 1 -- 172.21.15.44:0/1754529634 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb05402cb90 con 0x55c4dc8bff10 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.908+0000 7fb05b400640 1 -- 172.21.15.44:0/1754529634 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054034740 con 0x55c4dc8bff10 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.908+0000 7fb074881740 1 -- 172.21.15.44:0/1754529634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 msgr2=0x55c4dc8c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.908+0000 7fb074881740 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c8b20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb054036040 tx=0x7fb054005360 comp rx=0 tx=0).stop 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 -- 172.21.15.44:0/1754529634 shutdown_connections 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c9410 0x55c4dc8e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 --2- 172.21.15.44:0/1754529634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8bff10 0x55c4dc8c8b20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 -- 172.21.15.44:0/1754529634 >> 172.21.15.44:0/1754529634 conn(0x55c4dc8ba640 msgr2=0x55c4dc8bc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 -- 172.21.15.44:0/1754529634 shutdown_connections 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 -- 172.21.15.44:0/1754529634 wait complete. 2024-04-20T11:24:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 -- start start 2024-04-20T11:24:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.909+0000 7fb074881740 1 Processor -- start 2024-04-20T11:24:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b1330 con 0x55c4dc8c0830 2024-04-20T11:24:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b0c60 con 0x55c4dc8c9410 2024-04-20T11:24:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1adef0 con 0x55c4dc8bff10 2024-04-20T11:24:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb074881740 10 client.0 _ll_register_callbacks cb 0x7fb05c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41246/0 (socket says 172.21.15.44:41246) 2024-04-20T11:24:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 -- 172.21.15.44:0/1862643453 learned_addr learned my addr 172.21.15.44:0/1862643453 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06aa00640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06b400640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 -- 172.21.15.44:0/1862643453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 msgr2=0x55c4dc8d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 -- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 msgr2=0x55c4dc8ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.910+0000 7fb06a000640 1 -- 172.21.15.44:0/1862643453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4dc8cee90 con 0x55c4dc8c0830 2024-04-20T11:24:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb06a000640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d1ae0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009af0 tx=0x7fb05c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cddd2e11b3e30b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb058c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb058c00640 1 -- 172.21.15.44:0/1862643453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0b4030 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb058c00640 1 -- 172.21.15.44:0/1862643453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fb05c09b9e0 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05402cb90 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb058c00640 1 -- 172.21.15.44:0/1862643453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb05c020040 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb074881740 10 client.15828 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.911+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4dc940890 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb058c00640 1 -- 172.21.15.44:0/1862643453 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb05402cb90 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb058c00640 1 -- 172.21.15.44:0/1862643453 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fb05c0bb2a0 con 0x55c4dc8c0830 2024-04-20T11:24:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb074881740 10 client.15828 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb074881740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb074881740 20 client.15828 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb074881740 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc940df0 0x55c4dc943210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb074881740 4 client.15828 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4dc82c5b0 con 0x55c4dc940df0 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.912+0000 7fb06aa00640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc940df0 0x55c4dc943210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.913+0000 7fb06aa00640 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc940df0 0x55c4dc943210 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbac692a19d6e46 server_cookie=a68ed1e33ab3100 in_seq=0 out_seq=0 2024-04-20T11:24:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.913+0000 7fb058c00640 10 client.15828 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb058c00640 1 -- 172.21.15.44:0/1862643453 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4dc82c5b0 con 0x55c4dc940df0 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb058c00640 10 client.15828 handle_command_reply: tid=0 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 client.15828 shutdown 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 msgr2=0x55c4dc8d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d1ae0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009af0 tx=0x7fb05c003f70 comp rx=0 tx=0).stop 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 shutdown_connections 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55c4dc940df0 0x55c4dc943210 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4dc8c9410 0x55c4dc8ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4dc8c0830 0x55c4dc8d1ae0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 --2- 172.21.15.44:0/1862643453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c4dc8bff10 0x55c4dc8d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 >> 172.21.15.44:0/1862643453 conn(0x55c4dc8ba640 msgr2=0x55c4dc8bba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.914+0000 7fb058c00640 0 client.15828 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 shutdown_connections 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb074881740 1 -- 172.21.15.44:0/1862643453 wait complete. 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb074881740 20 client.15828 trim_cache size 0 max 16384 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "2f41d9d6-d02d-4029-9915-f9e14bf9eba3", 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:24:26.922 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:24:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb0500a3d20 msgr2=0x7fb0500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb071c00640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb0500a3d20 0x7fb0500a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb064012560 tx=0x7fb064005f90 comp rx=0 tx=0).stop 2024-04-20T11:24:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 msgr2=0x7fb06c1a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.915+0000 7fb071c00640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1a0da0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00dbf0 tx=0x7fb05c00b630 comp rx=0 tx=0).stop 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 shutdown_connections 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06c19aa10 0x7fb06c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb0500a3d20 0x7fb0500a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1094e0 0x7fb06c1a0da0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 --2- 172.21.15.44:0/8189179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104410 0x7fb06c1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 >> 172.21.15.44:0/8189179 conn(0x7fb06c0ffcf0 msgr2=0x7fb06c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 shutdown_connections 2024-04-20T11:24:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:26.916+0000 7fb071c00640 1 -- 172.21.15.44:0/8189179 wait complete. 2024-04-20T11:24:26.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:24:26.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:24:27.046 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-04-20T11:24:27.047 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-04-20T11:24:27.049 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-04-20T11:24:27.051 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-04-20T11:24:27.052 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-04-20T11:24:27.054 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-04-20T11:24:27.056 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-04-20T11:24:27.057 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-04-20T11:24:27.059 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-04-20T11:24:27.060 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-04-20T11:24:27.062 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-04-20T11:24:27.063 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-04-20T11:24:27.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-04-20T11:24:27.067 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-04-20T11:24:27.068 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-04-20T11:24:27.070 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-04-20T11:24:27.072 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-04-20T11:24:27.074 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-04-20T11:24:27.076 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-04-20T11:24:27.078 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-04-20T11:24:27.080 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-04-20T11:24:27.083 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-04-20T11:24:27.084 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-04-20T11:24:27.086 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-04-20T11:24:27.088 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-04-20T11:24:27.090 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-04-20T11:24:27.092 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-04-20T11:24:27.093 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-04-20T11:24:27.095 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-04-20T11:24:27.097 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-04-20T11:24:27.099 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-04-20T11:24:27.100 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-04-20T11:24:27.102 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-04-20T11:24:27.104 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-04-20T11:24:27.106 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-04-20T11:24:27.107 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-04-20T11:24:27.109 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-04-20T11:24:27.111 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-04-20T11:24:27.113 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-04-20T11:24:27.114 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-04-20T11:24:27.116 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-04-20T11:24:27.118 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-04-20T11:24:27.121 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-04-20T11:24:27.121 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-04-20T11:24:27.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-04-20T11:24:27.125 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-04-20T11:24:27.127 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-04-20T11:24:27.128 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-04-20T11:24:27.133 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-04-20T11:24:27.135 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-04-20T11:24:27.136 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-04-20T11:24:27.138 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-04-20T11:24:27.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-04-20T11:24:27.142 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-04-20T11:24:27.144 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-04-20T11:24:27.145 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-04-20T11:24:27.146 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-04-20T11:24:27.147 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 -- 172.21.15.44:0/1219452594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 msgr2=0x7fcc9c108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 --2- 172.21.15.44:0/1219452594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c108420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009900 tx=0x7fcc9002bcc0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 -- 172.21.15.44:0/1219452594 shutdown_connections 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 --2- 172.21.15.44:0/1219452594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 --2- 172.21.15.44:0/1219452594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c102000 0x7fcc9c102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 --2- 172.21.15.44:0/1219452594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c108420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 -- 172.21.15.44:0/1219452594 >> 172.21.15.44:0/1219452594 conn(0x7fcc9c076d00 msgr2=0x7fcc9c100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 -- 172.21.15.44:0/1219452594 shutdown_connections 2024-04-20T11:24:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.149+0000 7fcca3400640 1 -- 172.21.15.44:0/1219452594 wait complete. 2024-04-20T11:24:27.150 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-04-20T11:24:27.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.151+0000 7fcca3400640 1 Processor -- start 2024-04-20T11:24:27.151 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-04-20T11:24:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.151+0000 7fcca3400640 1 -- start start 2024-04-20T11:24:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 0x7fcc9c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c112430 con 0x7fcc9c1029c0 2024-04-20T11:24:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca3400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1122b0 con 0x7fcc9c108040 2024-04-20T11:24:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca3400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1125b0 con 0x7fcc9c102000 2024-04-20T11:24:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 0x7fcc9c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.153 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-04-20T11:24:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41264/0 (socket says 172.21.15.44:41264) 2024-04-20T11:24:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcc9be00640 1 -- 172.21.15.44:0/1984539200 learned_addr learned my addr 172.21.15.44:0/1984539200 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 0x7fcc9c19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:37352/0 (socket says 172.21.15.44:37352) 2024-04-20T11:24:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.152+0000 7fcca1400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc9be00640 1 -- 172.21.15.44:0/1984539200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 msgr2=0x7fcc9c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc9be00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 0x7fcc9c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc9be00640 1 -- 172.21.15.44:0/1984539200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 msgr2=0x7fcc9c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc9be00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc9be00640 1 -- 172.21.15.44:0/1984539200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c198f90 con 0x7fcc9c1029c0 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcca1400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 0x7fcc9c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc9be00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c19eb00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800e9f0 tx=0x7fcc8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.155 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-04-20T11:24:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc99600640 1 -- 172.21.15.44:0/1984539200 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800cd80 con 0x7fcc9c1029c0 2024-04-20T11:24:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc99600640 1 -- 172.21.15.44:0/1984539200 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc88010930 con 0x7fcc9c1029c0 2024-04-20T11:24:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c199280 con 0x7fcc9c1029c0 2024-04-20T11:24:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.153+0000 7fcc99600640 1 -- 172.21.15.44:0/1984539200 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88018b00 con 0x7fcc9c1029c0 2024-04-20T11:24:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.154+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9c1a5540 con 0x7fcc9c1029c0 2024-04-20T11:24:27.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-04-20T11:24:27.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/Changes 2024-04-20T11:24:27.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-04-20T11:24:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.157+0000 7fcca619b740 1 Processor -- start 2024-04-20T11:24:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 -- start start 2024-04-20T11:24:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238614580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923861d6a0 0x55923861fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55923857a150 con 0x559238614ac0 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559238520370 con 0x55923861d6a0 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca619b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559238576110 con 0x5592386141a0 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238614580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41270/0 (socket says 172.21.15.44:41270) 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcca0a00640 1 -- 172.21.15.44:0/130806255 learned_addr learned my addr 172.21.15.44:0/130806255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.158+0000 7fcc99600640 1 -- 172.21.15.44:0/1984539200 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcc88018ca0 con 0x7fcc9c1029c0 2024-04-20T11:24:27.159 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/CodingStyle 2024-04-20T11:24:27.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcc9be00640 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923861d6a0 0x55923861fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca1400640 1 -- 172.21.15.44:0/130806255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923861d6a0 msgr2=0x55923861fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca1400640 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923861d6a0 0x55923861fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca1400640 1 -- 172.21.15.44:0/130806255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 msgr2=0x55923861d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca1400640 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca1400640 1 -- 172.21.15.44:0/130806255 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592383567d0 con 0x5592386141a0 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcc9be00640 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923861d6a0 0x55923861fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca0a00640 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.159+0000 7fcca1400640 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238614580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc94005fd0 tx=0x7fcc94011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96b6a20b05eadfd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.160+0000 7fcc8dc00640 1 -- 172.21.15.44:0/130806255 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94004d20 con 0x5592386141a0 2024-04-20T11:24:27.161 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-04-20T11:24:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.160+0000 7fcc8dc00640 1 -- 172.21.15.44:0/130806255 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc9400f9a0 con 0x5592386141a0 2024-04-20T11:24:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.160+0000 7fcc8dc00640 1 -- 172.21.15.44:0/130806255 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94004580 con 0x5592386141a0 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.160+0000 7fcc99600640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc840a3d20 0x7fcc840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcc99600640 1 -- 172.21.15.44:0/1984539200 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc880d0b50 con 0x7fcc9c1029c0 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc840a3d20 0x7fcc840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 -- 172.21.15.44:0/130806255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 msgr2=0x559238614580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238614580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc94005fd0 tx=0x7fcc94011bf0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 -- 172.21.15.44:0/130806255 shutdown_connections 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923861d6a0 0x55923861fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 --2- 172.21.15.44:0/130806255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238614580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 -- 172.21.15.44:0/130806255 >> 172.21.15.44:0/130806255 conn(0x55923860e8d0 msgr2=0x55923860ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.163 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 -- 172.21.15.44:0/130806255 shutdown_connections 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.161+0000 7fcca619b740 1 -- 172.21.15.44:0/130806255 wait complete. 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.162+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc840a3d20 0x7fcc840a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559238613c70 tx=0x7fcc90005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.162+0000 7fcca619b740 1 -- start start 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.162+0000 7fcca619b740 1 Processor -- start 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 0x55923861a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923861d6a0 0x55923861b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559238620490 con 0x559238614ac0 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559238533740 con 0x5592386141a0 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559238620610 con 0x55923861d6a0 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca619b740 10 client.0 _ll_register_callbacks cb 0x55923852eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 0x55923861a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 0x55923861a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53354/0 (socket says 172.21.15.44:53354) 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 -- 172.21.15.44:0/1732308376 learned_addr learned my addr 172.21.15.44:0/1732308376 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca1400640 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923861d6a0 0x55923861b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.166 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-04-20T11:24:27.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 -- 172.21.15.44:0/1732308376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923861d6a0 msgr2=0x55923861b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923861d6a0 0x55923861b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 -- 172.21.15.44:0/1732308376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 msgr2=0x55923861ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.163+0000 7fcca0a00640 1 -- 172.21.15.44:0/1732308376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55923828c3f0 con 0x5592386141a0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcca0a00640 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 0x55923861a750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc900052c0 tx=0x7fcc9002bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459f1bfa0b505247 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcc82000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcc82000640 1 -- 172.21.15.44:0/1732308376 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90050020 con 0x5592386141a0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcc82000640 1 -- 172.21.15.44:0/1732308376 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc90004a50 con 0x5592386141a0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9400f9a0 con 0x5592386141a0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcc82000640 1 -- 172.21.15.44:0/1732308376 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc9005b050 con 0x5592386141a0 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcca619b740 10 client.25958 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.164+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55923824c0f0 con 0x5592386141a0 2024-04-20T11:24:27.169 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcc82000640 1 -- 172.21.15.44:0/1732308376 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc9400f9a0 con 0x5592386141a0 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcc82000640 1 -- 172.21.15.44:0/1732308376 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fcc900404d0 con 0x5592386141a0 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcca619b740 10 client.25958 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcca619b740 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcca619b740 20 client.25958 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcca619b740 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923862daa0 0x55923862fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcca619b740 4 client.25958 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559238587b70 con 0x55923862daa0 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.165+0000 7fcc9be00640 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923862daa0 0x55923862fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.166+0000 7fcc9be00640 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923862daa0 0x55923862fef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94f27f8125840f5 server_cookie=9524db8bf9027eff in_seq=0 out_seq=0 2024-04-20T11:24:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.166+0000 7fcc82000640 10 client.25958 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:27.171 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/ 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcc82000640 1 -- 172.21.15.44:0/1732308376 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559238587b70 con 0x55923862daa0 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcc82000640 10 client.25958 handle_command_reply: tid=0 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 client.25958 shutdown 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 msgr2=0x55923861a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 0x55923861a750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc900052c0 tx=0x7fcc9002bcc0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 shutdown_connections 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923862daa0 0x55923862fef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923861d6a0 0x55923861b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923861ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 --2- 172.21.15.44:0/1732308376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592386141a0 0x55923861a750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcc82000640 0 client.25958 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.167+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 >> 172.21.15.44:0/1732308376 conn(0x55923860e8d0 msgr2=0x559238610620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.168+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 shutdown_connections 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.168+0000 7fcca619b740 1 -- 172.21.15.44:0/1732308376 wait complete. 2024-04-20T11:24:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.168+0000 7fcca619b740 20 client.25958 trim_cache size 0 max 16384 2024-04-20T11:24:27.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-04-20T11:24:27.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-04-20T11:24:27.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-04-20T11:24:27.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-04-20T11:24:27.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-04-20T11:24:27.175 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-04-20T11:24:27.177 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-04-20T11:24:27.178 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-04-20T11:24:27.180 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-04-20T11:24:27.182 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-04-20T11:24:27.184 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-04-20T11:24:27.185 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-04-20T11:24:27.186 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-04-20T11:24:27.187 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-04-20T11:24:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 Processor -- start 2024-04-20T11:24:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 -- start start 2024-04-20T11:24:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923860e8d0 0x55923861ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592386141a0 0x559238629380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 0x559238619690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1adbf0 con 0x5592386141a0 2024-04-20T11:24:27.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1af730 con 0x559238614ac0 2024-04-20T11:24:27.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca619b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ae2c0 con 0x55923860e8d0 2024-04-20T11:24:27.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 0x559238619690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 0x559238619690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53368/0 (socket says 172.21.15.44:53368) 2024-04-20T11:24:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.189+0000 7fcca1400640 1 -- 172.21.15.44:0/2790434918 learned_addr learned my addr 172.21.15.44:0/2790434918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca0a00640 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923860e8d0 0x55923861ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcc9be00640 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592386141a0 0x559238629380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.190 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca1400640 1 -- 172.21.15.44:0/2790434918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923860e8d0 msgr2=0x55923861ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca1400640 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923860e8d0 0x55923861ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca1400640 1 -- 172.21.15.44:0/2790434918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592386141a0 msgr2=0x559238629380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca1400640 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592386141a0 0x559238629380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca1400640 1 -- 172.21.15.44:0/2790434918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55923860a190 con 0x559238614ac0 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca0a00640 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923860e8d0 0x55923861ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcca1400640 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 0x559238619690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcc94011730 tx=0x7fcc940095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c4f7743393f67d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcc8d200640 1 -- 172.21.15.44:0/2790434918 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401e040 con 0x559238614ac0 2024-04-20T11:24:27.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcc8d200640 1 -- 172.21.15.44:0/2790434918 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc94003b70 con 0x559238614ac0 2024-04-20T11:24:27.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.190+0000 7fcc8d200640 1 -- 172.21.15.44:0/2790434918 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94024440 con 0x559238614ac0 2024-04-20T11:24:27.192 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 -- 172.21.15.44:0/2790434918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 msgr2=0x559238619690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 0x559238619690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcc94011730 tx=0x7fcc940095b0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 -- 172.21.15.44:0/2790434918 shutdown_connections 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559238614ac0 0x559238619690 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5592386141a0 0x559238629380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 --2- 172.21.15.44:0/2790434918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55923860e8d0 0x55923861ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 -- 172.21.15.44:0/2790434918 >> 172.21.15.44:0/2790434918 conn(0x559238616120 msgr2=0x55923860d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.191+0000 7fcca619b740 1 -- 172.21.15.44:0/2790434918 shutdown_connections 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.192+0000 7fcca619b740 1 -- 172.21.15.44:0/2790434918 wait complete. 2024-04-20T11:24:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.192+0000 7fcca619b740 1 -- start start 2024-04-20T11:24:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.192+0000 7fcca619b740 1 Processor -- start 2024-04-20T11:24:27.194 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-04-20T11:24:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 0x5592386915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238691af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923868bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1af060 con 0x559238614ac0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ae990 con 0x55923860e8d0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1abbc0 con 0x5592386141a0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca619b740 10 client.0 _ll_register_callbacks cb 0x7fcc90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 0x5592386915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 0x5592386915b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53380/0 (socket says 172.21.15.44:53380) 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca0a00640 1 -- 172.21.15.44:0/965947574 learned_addr learned my addr 172.21.15.44:0/965947574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.193+0000 7fcca1400640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923868bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcc9be00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238691af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca0a00640 1 -- 172.21.15.44:0/965947574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 msgr2=0x559238691af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca0a00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238691af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca0a00640 1 -- 172.21.15.44:0/965947574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 msgr2=0x55923868bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca0a00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923868bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca0a00640 1 -- 172.21.15.44:0/965947574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55923868c6e0 con 0x55923860e8d0 2024-04-20T11:24:27.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcc9be00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238691af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:27.197 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca0a00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 0x5592386915b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc90006010 tx=0x7fcc90040bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2948fd7348cbc9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcc82000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcc82000640 1 -- 172.21.15.44:0/965947574 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90064060 con 0x55923860e8d0 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcc82000640 1 -- 172.21.15.44:0/965947574 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc90004a50 con 0x55923860e8d0 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.194+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc94003b70 con 0x55923860e8d0 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcc82000640 1 -- 172.21.15.44:0/965947574 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc90050070 con 0x55923860e8d0 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 10 client.25964 fetch_fsmap learned FSMap version 27 2024-04-20T11:24:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559238698d90 con 0x55923860e8d0 2024-04-20T11:24:27.199 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-04-20T11:24:27.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcc82000640 1 -- 172.21.15.44:0/965947574 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc94003b70 con 0x55923860e8d0 2024-04-20T11:24:27.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcc82000640 1 -- 172.21.15.44:0/965947574 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8355+0+0 (secure 0 0 0) 0x7fcc90067020 con 0x55923860e8d0 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 10 client.25964 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 20 client.25964 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923861d6a0 0x55923869b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 4 client.25964 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592385819e0 con 0x55923861d6a0 2024-04-20T11:24:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.195+0000 7fcc9be00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923861d6a0 0x55923869b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:27.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-04-20T11:24:27.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcc9be00640 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923861d6a0 0x55923869b520 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41f7bfa8d8ec20b server_cookie=6408af655c45110c in_seq=0 out_seq=0 2024-04-20T11:24:27.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcc82000640 10 client.25964 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcc82000640 1 -- 172.21.15.44:0/965947574 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5592385819e0 con 0x55923861d6a0 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcc82000640 10 client.25964 handle_command_reply: tid=0 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 client.25964 shutdown 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 msgr2=0x5592386915b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 0x5592386915b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc90006010 tx=0x7fcc90040bd0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 shutdown_connections 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55923861d6a0 0x55923869b520 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559238614ac0 0x55923868bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592386141a0 0x559238691af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 --2- 172.21.15.44:0/965947574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55923860e8d0 0x5592386915b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 >> 172.21.15.44:0/965947574 conn(0x559238616120 msgr2=0x55923861a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.196+0000 7fcc82000640 0 client.25964 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 shutdown_connections 2024-04-20T11:24:27.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca619b740 1 -- 172.21.15.44:0/965947574 wait complete. 2024-04-20T11:24:27.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca619b740 20 client.25964 trim_cache size 0 max 16384 2024-04-20T11:24:27.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-04-20T11:24:27.203 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:24:27.203 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:24:27.203 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:24:27.203 INFO:teuthology.orchestra.run.smithi044.stdout: "2f41d9d6-d02d-4029-9915-f9e14bf9eba3": { 2024-04-20T11:24:27.204 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:24:27.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "2f41d9d6-d02d-4029-9915-f9e14bf9eba3", 2024-04-20T11:24:27.204 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:24:27.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:24:27.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:24:27.204 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc840a3d20 msgr2=0x7fcc840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca3400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc840a3d20 0x7fcc840a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559238613c70 tx=0x7fcc90005e40 comp rx=0 tx=0).stop 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 msgr2=0x7fcc9c19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.197+0000 7fcca3400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c19eb00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800e9f0 tx=0x7fcc8800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 shutdown_connections 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc9c108040 0x7fcc9c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc840a3d20 0x7fcc840a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9c1029c0 0x7fcc9c19eb00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 --2- 172.21.15.44:0/1984539200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9c102000 0x7fcc9c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 >> 172.21.15.44:0/1984539200 conn(0x7fcc9c076d00 msgr2=0x7fcc9c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 shutdown_connections 2024-04-20T11:24:27.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:27.198+0000 7fcca3400640 1 -- 172.21.15.44:0/1984539200 wait complete. 2024-04-20T11:24:27.206 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-04-20T11:24:27.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-04-20T11:24:27.209 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-04-20T11:24:27.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-04-20T11:24:27.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-04-20T11:24:27.213 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-04-20T11:24:27.215 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-04-20T11:24:27.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-04-20T11:24:27.218 INFO:tasks.cephfs.filesystem:scrub status for tag:2f41d9d6-d02d-4029-9915-f9e14bf9eba3 - {'path': '/', 'tag': '2f41d9d6-d02d-4029-9915-f9e14bf9eba3', 'options': 'recursive,force'} 2024-04-20T11:24:27.218 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-04-20T11:24:27.220 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-04-20T11:24:27.221 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-04-20T11:24:27.225 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-04-20T11:24:27.231 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-04-20T11:24:27.235 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-04-20T11:24:27.237 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-04-20T11:24:27.239 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-04-20T11:24:27.240 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-04-20T11:24:27.242 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-04-20T11:24:27.244 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-04-20T11:24:27.245 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-04-20T11:24:27.247 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-04-20T11:24:27.249 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-04-20T11:24:27.251 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-04-20T11:24:27.254 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-04-20T11:24:27.261 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-04-20T11:24:27.263 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-04-20T11:24:27.265 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-04-20T11:24:27.266 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-04-20T11:24:27.268 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-04-20T11:24:27.270 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-04-20T11:24:27.271 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-04-20T11:24:27.273 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-04-20T11:24:27.275 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-04-20T11:24:27.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-04-20T11:24:27.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-04-20T11:24:27.280 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-04-20T11:24:27.281 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-04-20T11:24:27.283 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-04-20T11:24:27.285 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-04-20T11:24:27.287 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-04-20T11:24:27.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-04-20T11:24:27.290 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-04-20T11:24:27.292 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-04-20T11:24:27.294 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-04-20T11:24:27.296 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-04-20T11:24:27.297 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-04-20T11:24:27.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-04-20T11:24:27.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-04-20T11:24:27.302 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-04-20T11:24:27.304 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-04-20T11:24:27.306 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-04-20T11:24:27.307 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-04-20T11:24:27.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-04-20T11:24:27.311 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-04-20T11:24:27.313 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-04-20T11:24:27.315 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-04-20T11:24:27.317 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-04-20T11:24:27.318 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-04-20T11:24:27.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-04-20T11:24:27.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-04-20T11:24:27.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-04-20T11:24:27.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-04-20T11:24:27.327 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-04-20T11:24:27.331 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-04-20T11:24:27.333 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-04-20T11:24:27.335 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-04-20T11:24:27.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-04-20T11:24:27.338 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-04-20T11:24:27.340 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-04-20T11:24:27.342 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-04-20T11:24:27.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-04-20T11:24:27.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-04-20T11:24:27.347 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-04-20T11:24:27.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-04-20T11:24:27.350 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-04-20T11:24:27.353 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-04-20T11:24:27.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-04-20T11:24:27.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-04-20T11:24:27.358 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-04-20T11:24:27.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-04-20T11:24:27.361 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-04-20T11:24:27.363 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-04-20T11:24:27.364 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-04-20T11:24:27.366 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-04-20T11:24:27.367 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-04-20T11:24:27.369 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-04-20T11:24:27.371 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-04-20T11:24:27.373 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-04-20T11:24:27.375 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-04-20T11:24:27.376 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-04-20T11:24:27.378 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-04-20T11:24:27.380 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-04-20T11:24:27.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-04-20T11:24:27.383 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-04-20T11:24:27.384 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-04-20T11:24:27.386 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-04-20T11:24:27.388 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-04-20T11:24:27.390 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-04-20T11:24:27.391 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-04-20T11:24:27.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-04-20T11:24:27.395 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-04-20T11:24:27.396 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-04-20T11:24:27.398 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-04-20T11:24:27.401 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-04-20T11:24:27.401 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-04-20T11:24:27.403 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-04-20T11:24:27.405 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-04-20T11:24:27.406 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-04-20T11:24:27.409 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-04-20T11:24:27.410 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-04-20T11:24:27.412 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-04-20T11:24:27.414 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-04-20T11:24:27.417 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-04-20T11:24:27.420 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-04-20T11:24:27.424 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-04-20T11:24:27.427 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-04-20T11:24:27.430 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-04-20T11:24:27.432 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-04-20T11:24:27.434 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-04-20T11:24:27.436 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-04-20T11:24:27.439 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-04-20T11:24:27.442 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-04-20T11:24:27.444 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-04-20T11:24:27.447 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-04-20T11:24:27.449 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-04-20T11:24:27.451 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-04-20T11:24:27.453 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-04-20T11:24:27.456 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-04-20T11:24:27.458 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-04-20T11:24:27.460 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-04-20T11:24:27.462 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-04-20T11:24:27.465 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-04-20T11:24:27.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-04-20T11:24:27.469 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-04-20T11:24:27.471 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-04-20T11:24:27.473 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-04-20T11:24:27.475 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-04-20T11:24:27.477 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-04-20T11:24:27.480 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-04-20T11:24:27.482 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-04-20T11:24:27.484 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-04-20T11:24:27.487 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-04-20T11:24:27.491 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-04-20T11:24:27.491 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-04-20T11:24:27.493 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-04-20T11:24:27.496 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-04-20T11:24:27.498 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-04-20T11:24:27.500 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-04-20T11:24:27.503 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-04-20T11:24:27.505 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-04-20T11:24:27.507 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-04-20T11:24:27.510 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-04-20T11:24:27.512 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-04-20T11:24:27.516 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-04-20T11:24:27.519 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-04-20T11:24:27.521 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-04-20T11:24:27.523 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-04-20T11:24:27.526 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-04-20T11:24:27.529 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-04-20T11:24:27.531 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-04-20T11:24:27.533 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-04-20T11:24:27.536 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-04-20T11:24:27.538 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-04-20T11:24:27.541 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-04-20T11:24:27.543 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-04-20T11:24:27.545 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-04-20T11:24:27.547 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-04-20T11:24:27.549 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-04-20T11:24:27.552 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-04-20T11:24:27.554 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-04-20T11:24:27.604 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-04-20T11:24:27.606 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-04-20T11:24:27.607 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-04-20T11:24:27.609 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-04-20T11:24:27.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-04-20T11:24:27.613 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-04-20T11:24:27.615 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-04-20T11:24:27.617 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-04-20T11:24:27.619 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-04-20T11:24:27.620 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-04-20T11:24:27.622 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-04-20T11:24:27.624 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-04-20T11:24:27.626 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-04-20T11:24:27.628 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-04-20T11:24:27.630 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-04-20T11:24:27.632 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-04-20T11:24:27.633 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-04-20T11:24:27.635 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-04-20T11:24:27.637 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-04-20T11:24:27.639 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-04-20T11:24:27.640 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-04-20T11:24:27.642 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-04-20T11:24:27.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-04-20T11:24:27.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-04-20T11:24:27.648 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-04-20T11:24:27.649 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-04-20T11:24:27.651 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-04-20T11:24:27.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-04-20T11:24:27.655 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-04-20T11:24:27.657 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-04-20T11:24:27.659 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-04-20T11:24:27.667 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-04-20T11:24:27.670 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-04-20T11:24:27.672 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-04-20T11:24:27.674 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/ 2024-04-20T11:24:27.676 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-04-20T11:24:27.677 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-04-20T11:24:27.679 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-04-20T11:24:27.680 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-04-20T11:24:27.682 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-04-20T11:24:27.684 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-04-20T11:24:27.685 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-04-20T11:24:27.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:27 smithi078 ceph-mon[21464]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-20T11:24:27.687 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-04-20T11:24:27.689 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-04-20T11:24:27.690 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/EDID/hex 2024-04-20T11:24:27.692 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/HOWTO 2024-04-20T11:24:27.694 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-04-20T11:24:27.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:27 smithi044 ceph-mon[16855]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-20T11:24:27.696 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-04-20T11:24:27.697 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-04-20T11:24:27.699 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-04-20T11:24:27.701 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-04-20T11:24:27.702 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/Makefile 2024-04-20T11:24:27.704 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-04-20T11:24:27.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/ 2024-04-20T11:24:27.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-04-20T11:24:27.709 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-04-20T11:24:27.711 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-04-20T11:24:27.712 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-04-20T11:24:27.714 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-04-20T11:24:27.716 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-04-20T11:24:27.718 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-04-20T11:24:27.719 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/ 2024-04-20T11:24:27.721 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-04-20T11:24:27.722 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-04-20T11:24:27.724 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-04-20T11:24:27.726 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-04-20T11:24:27.728 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-04-20T11:24:27.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:27 smithi165 ceph-mon[21975]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-20T11:24:27.730 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-04-20T11:24:27.732 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-04-20T11:24:27.734 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-04-20T11:24:27.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-04-20T11:24:27.738 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-04-20T11:24:27.740 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-04-20T11:24:27.742 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-04-20T11:24:27.745 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-04-20T11:24:27.746 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-04-20T11:24:27.748 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-04-20T11:24:27.750 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-04-20T11:24:27.751 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-04-20T11:24:27.753 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-04-20T11:24:27.755 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/SAK.txt 2024-04-20T11:24:27.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/SM501.txt 2024-04-20T11:24:27.759 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-04-20T11:24:27.760 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-04-20T11:24:27.762 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-04-20T11:24:27.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-04-20T11:24:27.766 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-04-20T11:24:27.768 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/ 2024-04-20T11:24:27.769 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-04-20T11:24:27.771 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-04-20T11:24:27.772 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-04-20T11:24:27.774 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-04-20T11:24:27.775 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-04-20T11:24:27.777 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-04-20T11:24:27.779 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-04-20T11:24:27.781 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/ 2024-04-20T11:24:27.782 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-04-20T11:24:27.784 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-04-20T11:24:27.785 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-04-20T11:24:27.787 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-04-20T11:24:27.789 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-04-20T11:24:27.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-04-20T11:24:27.792 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-04-20T11:24:27.794 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-04-20T11:24:27.795 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-04-20T11:24:27.797 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-04-20T11:24:27.799 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-04-20T11:24:27.800 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-04-20T11:24:27.802 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-04-20T11:24:27.803 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/ 2024-04-20T11:24:27.805 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-04-20T11:24:27.807 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-04-20T11:24:27.809 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-04-20T11:24:27.810 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-04-20T11:24:27.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-04-20T11:24:27.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-04-20T11:24:27.815 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-04-20T11:24:27.816 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/ 2024-04-20T11:24:27.818 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-04-20T11:24:27.820 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-04-20T11:24:27.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-04-20T11:24:27.823 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Booting 2024-04-20T11:24:27.825 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-04-20T11:24:27.827 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-04-20T11:24:27.829 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-04-20T11:24:27.832 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-04-20T11:24:27.834 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-04-20T11:24:27.835 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-04-20T11:24:27.837 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-04-20T11:24:27.839 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-04-20T11:24:27.841 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-04-20T11:24:27.843 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-04-20T11:24:27.845 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Porting 2024-04-20T11:24:27.847 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/README 2024-04-20T11:24:27.849 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-04-20T11:24:27.850 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-04-20T11:24:27.852 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-04-20T11:24:27.854 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-04-20T11:24:27.856 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-04-20T11:24:27.858 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-04-20T11:24:27.860 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-04-20T11:24:27.861 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-04-20T11:24:27.863 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-04-20T11:24:27.864 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-04-20T11:24:27.866 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-04-20T11:24:27.868 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-04-20T11:24:27.869 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-04-20T11:24:27.871 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-04-20T11:24:27.873 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-04-20T11:24:27.874 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-04-20T11:24:27.876 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-04-20T11:24:27.877 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-04-20T11:24:27.879 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-04-20T11:24:27.881 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-04-20T11:24:27.883 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-04-20T11:24:27.884 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-04-20T11:24:27.886 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-04-20T11:24:27.888 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-04-20T11:24:27.889 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-04-20T11:24:27.891 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-04-20T11:24:27.893 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-04-20T11:24:27.894 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-04-20T11:24:27.896 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-04-20T11:24:27.898 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-04-20T11:24:27.899 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-04-20T11:24:27.901 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-04-20T11:24:27.903 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-04-20T11:24:27.904 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-04-20T11:24:27.906 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-04-20T11:24:27.908 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-04-20T11:24:27.910 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-04-20T11:24:27.911 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-04-20T11:24:27.913 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-04-20T11:24:27.914 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-04-20T11:24:27.916 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-04-20T11:24:27.918 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-04-20T11:24:27.919 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-04-20T11:24:27.921 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/Setup 2024-04-20T11:24:27.923 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-04-20T11:24:27.924 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-04-20T11:24:27.926 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-04-20T11:24:27.928 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-04-20T11:24:27.929 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-04-20T11:24:27.931 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-04-20T11:24:27.933 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-04-20T11:24:27.935 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-04-20T11:24:27.937 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-04-20T11:24:27.938 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-04-20T11:24:27.939 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-04-20T11:24:27.941 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-04-20T11:24:27.943 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-04-20T11:24:27.944 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-04-20T11:24:27.946 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-04-20T11:24:27.948 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-04-20T11:24:27.949 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-04-20T11:24:27.951 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-04-20T11:24:27.952 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-04-20T11:24:27.954 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-04-20T11:24:27.955 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-04-20T11:24:27.957 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-04-20T11:24:27.959 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-04-20T11:24:27.960 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-04-20T11:24:27.962 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-04-20T11:24:27.963 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-04-20T11:24:27.965 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-04-20T11:24:27.967 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-04-20T11:24:27.969 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-04-20T11:24:27.970 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-04-20T11:24:27.972 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm64/ 2024-04-20T11:24:27.974 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-04-20T11:24:27.976 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-04-20T11:24:27.977 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-04-20T11:24:27.979 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-04-20T11:24:27.981 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-04-20T11:24:27.983 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-04-20T11:24:27.985 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-04-20T11:24:27.987 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-04-20T11:24:27.989 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-04-20T11:24:27.990 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-04-20T11:24:27.992 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-04-20T11:24:27.994 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-04-20T11:24:27.996 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/backlight/ 2024-04-20T11:24:27.997 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-04-20T11:24:27.999 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-04-20T11:24:28.001 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-04-20T11:24:28.002 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/bcache.txt 2024-04-20T11:24:28.004 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-04-20T11:24:28.006 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blackfin/ 2024-04-20T11:24:28.008 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-04-20T11:24:28.009 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-04-20T11:24:28.011 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-04-20T11:24:28.013 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-04-20T11:24:28.014 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-04-20T11:24:28.016 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/ 2024-04-20T11:24:28.018 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-04-20T11:24:28.019 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-04-20T11:24:28.022 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-04-20T11:24:28.024 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-04-20T11:24:28.025 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-04-20T11:24:28.027 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-04-20T11:24:28.029 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-04-20T11:24:28.031 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-04-20T11:24:28.033 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-04-20T11:24:28.034 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-04-20T11:24:28.036 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-04-20T11:24:28.038 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/request.txt 2024-04-20T11:24:28.040 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-04-20T11:24:28.041 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-04-20T11:24:28.043 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-04-20T11:24:28.045 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/ 2024-04-20T11:24:28.047 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-04-20T11:24:28.048 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-04-20T11:24:28.050 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-04-20T11:24:28.052 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-04-20T11:24:28.053 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-04-20T11:24:28.055 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-04-20T11:24:28.057 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-04-20T11:24:28.058 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-04-20T11:24:28.060 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-04-20T11:24:28.062 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-04-20T11:24:28.063 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-04-20T11:24:28.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-04-20T11:24:28.067 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-04-20T11:24:28.068 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-04-20T11:24:28.070 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-04-20T11:24:28.072 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-04-20T11:24:28.074 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-04-20T11:24:28.075 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-04-20T11:24:28.077 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-04-20T11:24:28.079 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-04-20T11:24:28.081 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-04-20T11:24:28.082 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-04-20T11:24:28.084 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-04-20T11:24:28.086 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-04-20T11:24:28.087 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-04-20T11:24:28.089 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-04-20T11:24:28.091 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cdrom/ 2024-04-20T11:24:28.093 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-04-20T11:24:28.094 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-04-20T11:24:28.096 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-04-20T11:24:28.098 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-04-20T11:24:28.100 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-04-20T11:24:28.101 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/ 2024-04-20T11:24:28.103 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-04-20T11:24:28.104 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-04-20T11:24:28.106 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-04-20T11:24:28.108 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-04-20T11:24:28.110 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-04-20T11:24:28.112 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-04-20T11:24:28.114 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-04-20T11:24:28.116 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-04-20T11:24:28.117 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-04-20T11:24:28.119 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-04-20T11:24:28.121 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-04-20T11:24:28.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-04-20T11:24:28.125 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-04-20T11:24:28.126 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-04-20T11:24:28.128 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/clk.txt 2024-04-20T11:24:28.130 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-04-20T11:24:28.132 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/connector/ 2024-04-20T11:24:28.133 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-04-20T11:24:28.135 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-04-20T11:24:28.137 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-04-20T11:24:28.139 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-04-20T11:24:28.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-04-20T11:24:28.142 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/console/ 2024-04-20T11:24:28.145 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/console/console.txt 2024-04-20T11:24:28.145 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-04-20T11:24:28.147 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-04-20T11:24:28.149 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-04-20T11:24:28.151 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-04-20T11:24:28.152 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-04-20T11:24:28.154 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-04-20T11:24:28.156 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-04-20T11:24:28.158 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-04-20T11:24:28.159 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-04-20T11:24:28.161 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-04-20T11:24:28.163 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-04-20T11:24:28.165 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-04-20T11:24:28.167 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-04-20T11:24:28.169 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-04-20T11:24:28.170 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-04-20T11:24:28.172 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-04-20T11:24:28.173 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-04-20T11:24:28.175 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-04-20T11:24:28.177 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-04-20T11:24:28.178 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-04-20T11:24:28.180 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crc32.txt 2024-04-20T11:24:28.182 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cris/ 2024-04-20T11:24:28.183 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/cris/README 2024-04-20T11:24:28.185 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crypto/ 2024-04-20T11:24:28.187 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-04-20T11:24:28.188 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-04-20T11:24:28.190 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-04-20T11:24:28.192 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-04-20T11:24:28.193 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-04-20T11:24:28.195 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-04-20T11:24:28.197 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-04-20T11:24:28.199 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-04-20T11:24:28.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-04-20T11:24:28.202 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/ 2024-04-20T11:24:28.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-04-20T11:24:28.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-04-20T11:24:28.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-04-20T11:24:28.208 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-04-20T11:24:28.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-04-20T11:24:28.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-04-20T11:24:28.214 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-04-20T11:24:28.215 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-04-20T11:24:28.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-04-20T11:24:28.218 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-04-20T11:24:28.220 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-04-20T11:24:28.222 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-04-20T11:24:28.223 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-04-20T11:24:28.225 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-04-20T11:24:28.226 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-04-20T11:24:28.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-04-20T11:24:28.230 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-04-20T11:24:28.231 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-04-20T11:24:28.233 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-04-20T11:24:28.235 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-04-20T11:24:28.237 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-04-20T11:24:28.239 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-04-20T11:24:28.240 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-04-20T11:24:28.242 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-04-20T11:24:28.244 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-04-20T11:24:28.245 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-04-20T11:24:28.247 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-04-20T11:24:28.249 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-04-20T11:24:28.251 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-04-20T11:24:28.253 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-04-20T11:24:28.254 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-04-20T11:24:28.256 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devices.txt 2024-04-20T11:24:28.260 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/ 2024-04-20T11:24:28.262 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-04-20T11:24:28.264 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-04-20T11:24:28.265 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-04-20T11:24:28.267 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-04-20T11:24:28.268 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-04-20T11:24:28.270 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-04-20T11:24:28.271 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-04-20T11:24:28.273 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-04-20T11:24:28.275 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-04-20T11:24:28.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-04-20T11:24:28.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-04-20T11:24:28.279 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-04-20T11:24:28.281 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-04-20T11:24:28.283 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-04-20T11:24:28.285 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-04-20T11:24:28.286 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-04-20T11:24:28.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-04-20T11:24:28.291 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-04-20T11:24:28.292 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-04-20T11:24:28.293 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-04-20T11:24:28.295 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-04-20T11:24:28.297 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-04-20T11:24:28.298 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-04-20T11:24:28.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-04-20T11:24:28.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-04-20T11:24:28.303 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-04-20T11:24:28.304 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-04-20T11:24:28.306 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-04-20T11:24:28.307 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-04-20T11:24:28.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-04-20T11:24:28.311 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-04-20T11:24:28.312 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-04-20T11:24:28.314 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-04-20T11:24:28.315 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-04-20T11:24:28.317 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-04-20T11:24:28.319 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-04-20T11:24:28.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-04-20T11:24:28.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-04-20T11:24:28.323 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-04-20T11:24:28.325 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-04-20T11:24:28.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-04-20T11:24:28.328 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-04-20T11:24:28.330 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-04-20T11:24:28.331 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-04-20T11:24:28.333 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-04-20T11:24:28.335 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-04-20T11:24:28.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-04-20T11:24:28.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-04-20T11:24:28.340 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-04-20T11:24:28.342 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-04-20T11:24:28.344 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-04-20T11:24:28.346 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-04-20T11:24:28.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-04-20T11:24:28.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-04-20T11:24:28.351 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-04-20T11:24:28.352 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-04-20T11:24:28.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-04-20T11:24:28.355 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-04-20T11:24:28.357 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-04-20T11:24:28.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-04-20T11:24:28.360 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-04-20T11:24:28.362 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-04-20T11:24:28.364 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-04-20T11:24:28.365 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-04-20T11:24:28.367 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-04-20T11:24:28.369 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-04-20T11:24:28.370 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-04-20T11:24:28.372 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-04-20T11:24:28.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-04-20T11:24:28.375 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-04-20T11:24:28.377 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-04-20T11:24:28.379 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-04-20T11:24:28.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-04-20T11:24:28.382 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-04-20T11:24:28.384 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-04-20T11:24:28.386 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-04-20T11:24:28.388 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-04-20T11:24:28.389 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-04-20T11:24:28.391 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-04-20T11:24:28.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-04-20T11:24:28.394 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-04-20T11:24:28.396 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-04-20T11:24:28.397 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-04-20T11:24:28.399 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-04-20T11:24:28.401 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-04-20T11:24:28.402 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-04-20T11:24:28.404 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-04-20T11:24:28.406 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-04-20T11:24:28.408 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-04-20T11:24:28.409 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-04-20T11:24:28.411 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-04-20T11:24:28.412 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-04-20T11:24:28.414 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-04-20T11:24:28.416 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-04-20T11:24:28.418 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-04-20T11:24:28.419 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-04-20T11:24:28.421 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-04-20T11:24:28.423 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-04-20T11:24:28.424 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-04-20T11:24:28.426 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-04-20T11:24:28.428 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-04-20T11:24:28.429 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-04-20T11:24:28.431 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-04-20T11:24:28.433 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-04-20T11:24:28.435 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-04-20T11:24:28.436 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-04-20T11:24:28.438 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-04-20T11:24:28.439 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-04-20T11:24:28.441 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-04-20T11:24:28.443 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-04-20T11:24:28.445 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-04-20T11:24:28.446 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-04-20T11:24:28.448 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-04-20T11:24:28.450 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-04-20T11:24:28.451 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-04-20T11:24:28.453 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-04-20T11:24:28.455 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-04-20T11:24:28.457 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-04-20T11:24:28.459 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-04-20T11:24:28.460 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-04-20T11:24:28.462 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-04-20T11:24:28.463 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-04-20T11:24:28.465 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-04-20T11:24:28.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-04-20T11:24:28.469 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-04-20T11:24:28.470 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-04-20T11:24:28.472 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-04-20T11:24:28.474 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-04-20T11:24:28.475 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-04-20T11:24:28.477 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-04-20T11:24:28.479 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-04-20T11:24:28.480 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-04-20T11:24:28.482 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-04-20T11:24:28.484 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-04-20T11:24:28.486 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-04-20T11:24:28.487 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-04-20T11:24:28.489 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-04-20T11:24:28.491 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-04-20T11:24:28.492 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-04-20T11:24:28.494 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-04-20T11:24:28.496 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-04-20T11:24:28.497 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-04-20T11:24:28.499 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-04-20T11:24:28.501 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-04-20T11:24:28.502 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-04-20T11:24:28.504 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-04-20T11:24:28.506 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-04-20T11:24:28.508 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-04-20T11:24:28.509 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-04-20T11:24:28.511 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-04-20T11:24:28.513 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-04-20T11:24:28.515 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-04-20T11:24:28.516 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-04-20T11:24:28.518 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-04-20T11:24:28.520 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-04-20T11:24:28.522 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-04-20T11:24:28.524 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-04-20T11:24:28.525 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-04-20T11:24:28.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:28 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:24:28.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:28 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:24:28.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:28 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:24:28.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:28 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:24:28.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:28 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:24:28.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:28 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:24:28.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:28 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:24:28.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:28 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:24:28.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:28 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:24:28.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:28 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:24:28.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:28 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:24:28.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:28 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:24:28.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:28 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:24:28.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:28 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:24:28.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:28 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:24:28.733 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-04-20T11:24:28.738 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-04-20T11:24:28.741 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-04-20T11:24:28.744 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-04-20T11:24:28.746 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-04-20T11:24:28.747 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-04-20T11:24:28.749 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-04-20T11:24:28.750 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-04-20T11:24:28.752 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-04-20T11:24:28.754 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-04-20T11:24:28.755 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-04-20T11:24:28.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-04-20T11:24:28.759 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-04-20T11:24:28.760 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-04-20T11:24:28.762 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-04-20T11:24:28.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-04-20T11:24:28.765 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-04-20T11:24:28.767 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-04-20T11:24:28.769 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-04-20T11:24:28.770 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-04-20T11:24:28.772 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-04-20T11:24:28.774 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-04-20T11:24:28.776 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-04-20T11:24:28.777 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-04-20T11:24:28.779 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-04-20T11:24:28.781 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-04-20T11:24:28.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-04-20T11:24:28.784 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-04-20T11:24:28.786 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-04-20T11:24:28.788 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-04-20T11:24:28.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-04-20T11:24:28.792 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-04-20T11:24:28.794 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-04-20T11:24:28.795 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-04-20T11:24:28.797 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-04-20T11:24:28.799 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-04-20T11:24:28.801 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-04-20T11:24:28.802 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-04-20T11:24:28.804 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-04-20T11:24:28.806 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-04-20T11:24:28.808 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-04-20T11:24:28.809 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-04-20T11:24:28.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-04-20T11:24:28.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-04-20T11:24:28.815 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-04-20T11:24:28.816 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-04-20T11:24:28.818 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-04-20T11:24:28.819 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-04-20T11:24:28.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-04-20T11:24:28.823 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-04-20T11:24:28.824 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-04-20T11:24:28.826 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-04-20T11:24:28.828 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-04-20T11:24:28.830 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-04-20T11:24:28.831 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-04-20T11:24:28.833 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-04-20T11:24:28.835 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-04-20T11:24:28.836 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-04-20T11:24:28.838 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-04-20T11:24:28.840 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-04-20T11:24:28.842 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-04-20T11:24:28.843 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-04-20T11:24:28.845 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-04-20T11:24:28.847 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-04-20T11:24:28.848 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-04-20T11:24:28.850 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-04-20T11:24:28.852 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-04-20T11:24:28.853 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-04-20T11:24:28.855 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-04-20T11:24:28.858 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-04-20T11:24:28.860 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-04-20T11:24:28.862 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-04-20T11:24:28.865 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-04-20T11:24:28.868 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-04-20T11:24:28.870 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-04-20T11:24:28.872 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-04-20T11:24:28.875 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-04-20T11:24:28.877 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-04-20T11:24:28.879 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-04-20T11:24:28.882 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-04-20T11:24:28.884 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-04-20T11:24:28.886 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-04-20T11:24:28.889 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-04-20T11:24:28.891 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-04-20T11:24:28.893 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-04-20T11:24:28.895 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-04-20T11:24:28.897 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-04-20T11:24:28.899 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-04-20T11:24:28.902 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-04-20T11:24:28.904 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-04-20T11:24:28.906 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-04-20T11:24:28.908 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-04-20T11:24:28.910 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-04-20T11:24:28.913 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-04-20T11:24:28.915 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-04-20T11:24:28.917 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-04-20T11:24:28.920 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-04-20T11:24:28.922 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-04-20T11:24:28.924 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-04-20T11:24:28.926 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-04-20T11:24:28.929 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-04-20T11:24:28.932 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-04-20T11:24:28.934 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-04-20T11:24:28.936 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-04-20T11:24:28.939 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-04-20T11:24:28.941 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-04-20T11:24:28.943 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-04-20T11:24:28.946 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-04-20T11:24:28.948 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-04-20T11:24:28.950 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-04-20T11:24:28.953 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-04-20T11:24:28.955 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-04-20T11:24:28.958 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-04-20T11:24:28.961 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-04-20T11:24:28.962 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-04-20T11:24:28.965 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-04-20T11:24:28.967 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-04-20T11:24:28.969 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-04-20T11:24:28.972 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-04-20T11:24:28.974 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-04-20T11:24:28.977 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-04-20T11:24:28.979 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-04-20T11:24:28.981 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-04-20T11:24:28.983 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-04-20T11:24:28.986 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-04-20T11:24:28.988 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-04-20T11:24:28.992 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-04-20T11:24:28.994 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-04-20T11:24:28.996 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-04-20T11:24:28.998 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-04-20T11:24:29.001 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-04-20T11:24:29.003 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-04-20T11:24:29.005 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-04-20T11:24:29.007 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-04-20T11:24:29.009 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-04-20T11:24:29.011 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-04-20T11:24:29.013 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-04-20T11:24:29.015 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-04-20T11:24:29.017 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-04-20T11:24:29.020 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-04-20T11:24:29.022 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-04-20T11:24:29.024 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-04-20T11:24:29.026 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-04-20T11:24:29.029 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-04-20T11:24:29.031 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-04-20T11:24:29.033 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-04-20T11:24:29.035 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-04-20T11:24:29.037 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-04-20T11:24:29.040 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-04-20T11:24:29.042 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-04-20T11:24:29.044 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-04-20T11:24:29.046 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-04-20T11:24:29.048 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-04-20T11:24:29.050 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-04-20T11:24:29.053 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-04-20T11:24:29.055 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-04-20T11:24:29.057 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-04-20T11:24:29.059 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-04-20T11:24:29.061 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-04-20T11:24:29.063 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-04-20T11:24:29.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-04-20T11:24:29.067 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-04-20T11:24:29.070 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-04-20T11:24:29.072 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-04-20T11:24:29.074 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-04-20T11:24:29.076 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-04-20T11:24:29.079 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-04-20T11:24:29.081 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-04-20T11:24:29.083 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-04-20T11:24:29.085 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-04-20T11:24:29.088 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-04-20T11:24:29.090 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-04-20T11:24:29.092 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-04-20T11:24:29.094 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-04-20T11:24:29.096 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-04-20T11:24:29.098 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-04-20T11:24:29.100 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-04-20T11:24:29.102 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-04-20T11:24:29.104 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-04-20T11:24:29.106 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-04-20T11:24:29.108 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-04-20T11:24:29.110 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-04-20T11:24:29.112 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-04-20T11:24:29.113 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-04-20T11:24:29.115 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-04-20T11:24:29.117 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-04-20T11:24:29.118 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-04-20T11:24:29.120 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-04-20T11:24:29.122 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-04-20T11:24:29.124 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-04-20T11:24:29.125 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-04-20T11:24:29.127 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-04-20T11:24:29.129 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-04-20T11:24:29.130 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-04-20T11:24:29.132 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-04-20T11:24:29.134 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-04-20T11:24:29.135 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-04-20T11:24:29.137 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-04-20T11:24:29.139 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-04-20T11:24:29.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-04-20T11:24:29.142 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-04-20T11:24:29.144 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-04-20T11:24:29.145 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-04-20T11:24:29.147 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-04-20T11:24:29.149 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-04-20T11:24:29.150 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-04-20T11:24:29.152 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-04-20T11:24:29.153 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-04-20T11:24:29.155 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-04-20T11:24:29.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-04-20T11:24:29.158 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-04-20T11:24:29.159 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-04-20T11:24:29.161 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-04-20T11:24:29.163 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-04-20T11:24:29.164 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-04-20T11:24:29.166 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-04-20T11:24:29.168 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-04-20T11:24:29.169 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-04-20T11:24:29.171 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-04-20T11:24:29.172 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-04-20T11:24:29.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-04-20T11:24:29.176 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-04-20T11:24:29.177 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-04-20T11:24:29.179 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-04-20T11:24:29.180 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-04-20T11:24:29.182 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-04-20T11:24:29.183 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-04-20T11:24:29.185 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-04-20T11:24:29.187 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-04-20T11:24:29.189 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-04-20T11:24:29.190 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-04-20T11:24:29.192 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-04-20T11:24:29.193 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-04-20T11:24:29.195 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-04-20T11:24:29.197 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-04-20T11:24:29.198 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-04-20T11:24:29.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-04-20T11:24:29.201 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-04-20T11:24:29.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-04-20T11:24:29.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-04-20T11:24:29.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-04-20T11:24:29.209 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-04-20T11:24:29.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-04-20T11:24:29.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-04-20T11:24:29.213 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-04-20T11:24:29.215 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-04-20T11:24:29.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-04-20T11:24:29.219 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-04-20T11:24:29.221 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-04-20T11:24:29.223 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-04-20T11:24:29.225 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-04-20T11:24:29.227 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-04-20T11:24:29.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-04-20T11:24:29.230 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-04-20T11:24:29.232 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-04-20T11:24:29.233 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-04-20T11:24:29.235 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-04-20T11:24:29.236 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-04-20T11:24:29.238 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-04-20T11:24:29.239 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-04-20T11:24:29.241 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-04-20T11:24:29.243 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-04-20T11:24:29.244 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-04-20T11:24:29.246 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-04-20T11:24:29.248 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-04-20T11:24:29.249 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-04-20T11:24:29.251 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-04-20T11:24:29.253 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-04-20T11:24:29.255 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-04-20T11:24:29.256 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-04-20T11:24:29.258 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-04-20T11:24:29.260 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-04-20T11:24:29.262 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-04-20T11:24:29.263 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-04-20T11:24:29.265 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-04-20T11:24:29.267 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-04-20T11:24:29.268 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-04-20T11:24:29.270 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-04-20T11:24:29.272 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-04-20T11:24:29.274 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-04-20T11:24:29.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-04-20T11:24:29.277 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-04-20T11:24:29.279 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-04-20T11:24:29.281 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-04-20T11:24:29.282 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-04-20T11:24:29.284 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-04-20T11:24:29.286 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-04-20T11:24:29.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-04-20T11:24:29.289 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-04-20T11:24:29.291 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-04-20T11:24:29.293 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-04-20T11:24:29.295 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-04-20T11:24:29.296 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-04-20T11:24:29.298 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-04-20T11:24:29.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-04-20T11:24:29.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-04-20T11:24:29.304 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-04-20T11:24:29.306 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-04-20T11:24:29.308 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-04-20T11:24:29.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-04-20T11:24:29.311 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-04-20T11:24:29.313 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-04-20T11:24:29.315 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-04-20T11:24:29.317 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-04-20T11:24:29.318 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-04-20T11:24:29.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-04-20T11:24:29.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-04-20T11:24:29.324 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-04-20T11:24:29.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-04-20T11:24:29.328 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-04-20T11:24:29.329 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-04-20T11:24:29.331 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-04-20T11:24:29.333 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-04-20T11:24:29.335 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-04-20T11:24:29.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-04-20T11:24:29.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-04-20T11:24:29.341 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-04-20T11:24:29.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-04-20T11:24:29.344 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-04-20T11:24:29.346 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-04-20T11:24:29.348 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-04-20T11:24:29.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-04-20T11:24:29.351 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-04-20T11:24:29.353 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-04-20T11:24:29.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-04-20T11:24:29.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-04-20T11:24:29.358 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-04-20T11:24:29.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-04-20T11:24:29.361 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-04-20T11:24:29.363 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-04-20T11:24:29.364 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-04-20T11:24:29.366 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-04-20T11:24:29.368 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-04-20T11:24:29.369 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-04-20T11:24:29.371 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-04-20T11:24:29.373 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-04-20T11:24:29.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-04-20T11:24:29.376 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-04-20T11:24:29.378 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-04-20T11:24:29.380 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-04-20T11:24:29.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-04-20T11:24:29.383 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-04-20T11:24:29.385 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-04-20T11:24:29.387 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-04-20T11:24:29.388 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-04-20T11:24:29.390 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-04-20T11:24:29.392 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-04-20T11:24:29.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-04-20T11:24:29.395 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-04-20T11:24:29.397 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-04-20T11:24:29.399 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-04-20T11:24:29.400 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-04-20T11:24:29.402 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-04-20T11:24:29.405 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-04-20T11:24:29.407 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-04-20T11:24:29.408 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-04-20T11:24:29.410 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-04-20T11:24:29.412 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-04-20T11:24:29.414 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-04-20T11:24:29.416 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-04-20T11:24:29.418 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-04-20T11:24:29.419 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-04-20T11:24:29.421 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-04-20T11:24:29.423 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-04-20T11:24:29.424 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-04-20T11:24:29.426 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-04-20T11:24:29.428 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-04-20T11:24:29.430 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-04-20T11:24:29.431 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-04-20T11:24:29.433 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-04-20T11:24:29.435 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-04-20T11:24:29.437 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-04-20T11:24:29.439 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-04-20T11:24:29.441 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-04-20T11:24:29.442 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-04-20T11:24:29.444 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-04-20T11:24:29.446 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-04-20T11:24:29.447 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-04-20T11:24:29.449 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-04-20T11:24:29.451 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-04-20T11:24:29.453 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-04-20T11:24:29.455 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-04-20T11:24:29.456 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-04-20T11:24:29.458 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-04-20T11:24:29.460 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-04-20T11:24:29.462 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-04-20T11:24:29.463 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-04-20T11:24:29.465 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-04-20T11:24:29.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-04-20T11:24:29.468 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-04-20T11:24:29.470 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-04-20T11:24:29.472 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-04-20T11:24:29.473 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-04-20T11:24:29.475 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-04-20T11:24:29.477 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-04-20T11:24:29.478 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-04-20T11:24:29.480 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-04-20T11:24:29.482 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-04-20T11:24:29.484 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-04-20T11:24:29.485 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-04-20T11:24:29.487 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-04-20T11:24:29.489 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-04-20T11:24:29.490 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-04-20T11:24:29.492 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-04-20T11:24:29.494 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-04-20T11:24:29.495 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-04-20T11:24:29.497 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-04-20T11:24:29.499 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-04-20T11:24:29.501 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-04-20T11:24:29.503 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-04-20T11:24:29.505 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-04-20T11:24:29.507 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-04-20T11:24:29.508 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-04-20T11:24:29.510 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-04-20T11:24:29.512 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-04-20T11:24:29.514 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-04-20T11:24:29.515 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-04-20T11:24:29.517 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-04-20T11:24:29.519 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-04-20T11:24:29.521 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-04-20T11:24:29.522 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-04-20T11:24:29.524 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-04-20T11:24:29.526 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-04-20T11:24:29.528 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-04-20T11:24:29.529 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-04-20T11:24:29.531 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-04-20T11:24:29.533 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-04-20T11:24:29.535 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-04-20T11:24:29.537 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-04-20T11:24:29.539 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-04-20T11:24:29.540 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-04-20T11:24:29.542 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-04-20T11:24:29.544 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-04-20T11:24:29.546 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-04-20T11:24:29.548 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-04-20T11:24:29.549 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-04-20T11:24:29.551 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-04-20T11:24:29.553 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-04-20T11:24:29.555 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-04-20T11:24:29.557 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-04-20T11:24:29.559 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-04-20T11:24:29.561 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-04-20T11:24:29.563 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-04-20T11:24:29.565 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-04-20T11:24:29.567 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-04-20T11:24:29.569 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-04-20T11:24:29.571 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-04-20T11:24:29.572 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-04-20T11:24:29.574 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-04-20T11:24:29.578 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-04-20T11:24:29.578 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-04-20T11:24:29.580 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-04-20T11:24:29.582 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-04-20T11:24:29.583 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-04-20T11:24:29.585 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-04-20T11:24:29.587 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-04-20T11:24:29.589 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-04-20T11:24:29.591 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-04-20T11:24:29.593 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-04-20T11:24:29.594 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-04-20T11:24:29.596 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-04-20T11:24:29.598 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-04-20T11:24:29.600 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-04-20T11:24:29.601 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-04-20T11:24:29.603 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-04-20T11:24:29.605 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-04-20T11:24:29.607 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-04-20T11:24:29.609 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-04-20T11:24:29.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-04-20T11:24:29.612 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-04-20T11:24:29.614 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-04-20T11:24:29.616 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-04-20T11:24:29.618 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-04-20T11:24:29.620 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-04-20T11:24:29.621 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-04-20T11:24:29.623 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-04-20T11:24:29.625 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-04-20T11:24:29.627 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-04-20T11:24:29.629 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-04-20T11:24:29.630 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-04-20T11:24:29.632 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-04-20T11:24:29.634 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-04-20T11:24:29.635 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-04-20T11:24:29.637 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-04-20T11:24:29.639 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-04-20T11:24:29.641 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-04-20T11:24:29.642 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-04-20T11:24:29.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-04-20T11:24:29.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-04-20T11:24:29.647 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-04-20T11:24:29.649 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-04-20T11:24:29.651 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-04-20T11:24:29.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-04-20T11:24:29.655 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-04-20T11:24:29.656 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-04-20T11:24:29.658 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-04-20T11:24:29.660 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-04-20T11:24:29.661 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-04-20T11:24:29.663 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-04-20T11:24:29.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-04-20T11:24:29.666 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-04-20T11:24:29.668 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-04-20T11:24:29.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:29 smithi078 ceph-mon[21464]: pgmap v59: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 MiB/s wr, 18 op/s 2024-04-20T11:24:29.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:29 smithi044 ceph-mon[16855]: pgmap v59: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 MiB/s wr, 18 op/s 2024-04-20T11:24:29.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:29 smithi165 ceph-mon[21975]: pgmap v59: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 MiB/s wr, 18 op/s 2024-04-20T11:24:29.915 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-04-20T11:24:29.917 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-04-20T11:24:29.919 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-04-20T11:24:29.920 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-04-20T11:24:29.922 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-04-20T11:24:29.924 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-04-20T11:24:29.925 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-04-20T11:24:29.927 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-04-20T11:24:29.928 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-04-20T11:24:29.930 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-04-20T11:24:29.931 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-04-20T11:24:29.933 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-04-20T11:24:29.935 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-04-20T11:24:29.937 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-04-20T11:24:29.938 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-04-20T11:24:29.940 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-04-20T11:24:29.942 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-04-20T11:24:29.943 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-04-20T11:24:29.945 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-04-20T11:24:29.947 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-04-20T11:24:29.949 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-04-20T11:24:29.952 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-04-20T11:24:29.954 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-04-20T11:24:29.956 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-04-20T11:24:29.958 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-04-20T11:24:29.961 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-04-20T11:24:29.963 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-04-20T11:24:29.965 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-04-20T11:24:29.967 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-04-20T11:24:29.970 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-04-20T11:24:29.972 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-04-20T11:24:29.974 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-04-20T11:24:29.977 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-04-20T11:24:29.979 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-04-20T11:24:29.981 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-04-20T11:24:29.983 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-04-20T11:24:29.986 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-04-20T11:24:29.987 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-04-20T11:24:29.989 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-04-20T11:24:29.992 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-04-20T11:24:29.994 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-04-20T11:24:29.996 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-04-20T11:24:29.999 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-04-20T11:24:30.001 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-04-20T11:24:30.003 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-04-20T11:24:30.006 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-04-20T11:24:30.008 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-04-20T11:24:30.010 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-04-20T11:24:30.012 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-04-20T11:24:30.014 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-04-20T11:24:30.017 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-04-20T11:24:30.019 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-04-20T11:24:30.022 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-04-20T11:24:30.024 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-04-20T11:24:30.026 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-04-20T11:24:30.028 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-04-20T11:24:30.030 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-04-20T11:24:30.031 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-04-20T11:24:30.033 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-04-20T11:24:30.035 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-04-20T11:24:30.037 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-04-20T11:24:30.039 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-04-20T11:24:30.041 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-04-20T11:24:30.042 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-04-20T11:24:30.044 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-04-20T11:24:30.046 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-04-20T11:24:30.047 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-04-20T11:24:30.049 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-04-20T11:24:30.051 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-04-20T11:24:30.053 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-04-20T11:24:30.054 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-04-20T11:24:30.056 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-04-20T11:24:30.058 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-04-20T11:24:30.059 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-04-20T11:24:30.061 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-04-20T11:24:30.063 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-04-20T11:24:30.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-04-20T11:24:30.066 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-04-20T11:24:30.068 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-04-20T11:24:30.070 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-04-20T11:24:30.071 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-04-20T11:24:30.073 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-04-20T11:24:30.075 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-04-20T11:24:30.077 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-04-20T11:24:30.078 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-04-20T11:24:30.080 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-04-20T11:24:30.082 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-04-20T11:24:30.083 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-04-20T11:24:30.085 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-04-20T11:24:30.087 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-04-20T11:24:30.089 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-04-20T11:24:30.090 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-04-20T11:24:30.092 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-04-20T11:24:30.094 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-04-20T11:24:30.096 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-04-20T11:24:30.097 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-04-20T11:24:30.100 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-04-20T11:24:30.101 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-04-20T11:24:30.103 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-04-20T11:24:30.104 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-04-20T11:24:30.106 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-04-20T11:24:30.108 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-04-20T11:24:30.110 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-04-20T11:24:30.111 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-04-20T11:24:30.113 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-04-20T11:24:30.115 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-04-20T11:24:30.117 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-04-20T11:24:30.118 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-04-20T11:24:30.120 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-04-20T11:24:30.122 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-04-20T11:24:30.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-04-20T11:24:30.125 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-04-20T11:24:30.127 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-04-20T11:24:30.129 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-04-20T11:24:30.130 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-04-20T11:24:30.131 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-04-20T11:24:30.133 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-04-20T11:24:30.135 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-04-20T11:24:30.137 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-04-20T11:24:30.138 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-04-20T11:24:30.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-04-20T11:24:30.142 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-04-20T11:24:30.144 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-04-20T11:24:30.146 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-04-20T11:24:30.148 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-04-20T11:24:30.149 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-04-20T11:24:30.151 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-04-20T11:24:30.153 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-04-20T11:24:30.155 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-04-20T11:24:30.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-04-20T11:24:30.158 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-04-20T11:24:30.160 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-04-20T11:24:30.162 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-04-20T11:24:30.164 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-04-20T11:24:30.165 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-04-20T11:24:30.167 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-04-20T11:24:30.169 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-04-20T11:24:30.170 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-04-20T11:24:30.172 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-04-20T11:24:30.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-04-20T11:24:30.175 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-04-20T11:24:30.177 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-04-20T11:24:30.179 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-04-20T11:24:30.180 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-04-20T11:24:30.182 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-04-20T11:24:30.184 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-04-20T11:24:30.186 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-04-20T11:24:30.190 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-04-20T11:24:30.190 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-04-20T11:24:30.193 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-04-20T11:24:30.195 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-04-20T11:24:30.197 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-04-20T11:24:30.199 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-04-20T11:24:30.201 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-04-20T11:24:30.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-04-20T11:24:30.206 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-04-20T11:24:30.208 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-04-20T11:24:30.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-04-20T11:24:30.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-04-20T11:24:30.215 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-04-20T11:24:30.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-04-20T11:24:30.219 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-04-20T11:24:30.221 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-04-20T11:24:30.223 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-04-20T11:24:30.225 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-04-20T11:24:30.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-04-20T11:24:30.230 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-04-20T11:24:30.232 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-04-20T11:24:30.234 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-04-20T11:24:30.237 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-04-20T11:24:30.239 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-04-20T11:24:30.241 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-04-20T11:24:30.244 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-04-20T11:24:30.246 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-04-20T11:24:30.248 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-04-20T11:24:30.251 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-04-20T11:24:30.253 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-04-20T11:24:30.255 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-04-20T11:24:30.257 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-04-20T11:24:30.259 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-04-20T11:24:30.261 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-04-20T11:24:30.264 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-04-20T11:24:30.266 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-04-20T11:24:30.268 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-04-20T11:24:30.269 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-04-20T11:24:30.271 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-04-20T11:24:30.273 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-04-20T11:24:30.274 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-04-20T11:24:30.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-04-20T11:24:30.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-04-20T11:24:30.280 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-04-20T11:24:30.281 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-04-20T11:24:30.283 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-04-20T11:24:30.285 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-04-20T11:24:30.287 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-04-20T11:24:30.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-04-20T11:24:30.290 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-04-20T11:24:30.291 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-04-20T11:24:30.293 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-04-20T11:24:30.295 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-04-20T11:24:30.296 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-04-20T11:24:30.298 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-04-20T11:24:30.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-04-20T11:24:30.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-04-20T11:24:30.303 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-04-20T11:24:30.305 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-04-20T11:24:30.306 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-04-20T11:24:30.308 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-04-20T11:24:30.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-04-20T11:24:30.311 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-04-20T11:24:30.313 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-04-20T11:24:30.316 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-04-20T11:24:30.317 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-04-20T11:24:30.319 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-04-20T11:24:30.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-04-20T11:24:30.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-04-20T11:24:30.324 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-04-20T11:24:30.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-04-20T11:24:30.327 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-04-20T11:24:30.329 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-04-20T11:24:30.331 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-04-20T11:24:30.333 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-04-20T11:24:30.334 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-04-20T11:24:30.336 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-04-20T11:24:30.338 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-04-20T11:24:30.340 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-04-20T11:24:30.342 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-04-20T11:24:30.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-04-20T11:24:30.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-04-20T11:24:30.347 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-04-20T11:24:30.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-04-20T11:24:30.351 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-04-20T11:24:30.353 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-04-20T11:24:30.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-04-20T11:24:30.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-04-20T11:24:30.358 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-04-20T11:24:30.360 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-04-20T11:24:30.362 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-04-20T11:24:30.364 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-04-20T11:24:30.366 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-04-20T11:24:30.368 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-04-20T11:24:30.370 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-04-20T11:24:30.372 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-04-20T11:24:30.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-04-20T11:24:30.376 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-04-20T11:24:30.377 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-04-20T11:24:30.379 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-04-20T11:24:30.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-04-20T11:24:30.383 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-04-20T11:24:30.384 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-04-20T11:24:30.386 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-04-20T11:24:30.388 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-04-20T11:24:30.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-04-20T11:24:30.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-04-20T11:24:30.394 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-04-20T11:24:30.396 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-04-20T11:24:30.398 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-04-20T11:24:30.399 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-04-20T11:24:30.401 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-04-20T11:24:30.403 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-04-20T11:24:30.404 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-04-20T11:24:30.406 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-04-20T11:24:30.408 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-04-20T11:24:30.410 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-04-20T11:24:30.411 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-04-20T11:24:30.413 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-04-20T11:24:30.415 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-04-20T11:24:30.417 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-04-20T11:24:30.418 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-04-20T11:24:30.420 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-04-20T11:24:30.422 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-04-20T11:24:30.424 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-04-20T11:24:30.425 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-04-20T11:24:30.427 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-04-20T11:24:30.429 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-04-20T11:24:30.431 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-04-20T11:24:30.433 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-04-20T11:24:30.434 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-04-20T11:24:30.436 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-04-20T11:24:30.438 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-04-20T11:24:30.440 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-04-20T11:24:30.442 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-04-20T11:24:30.443 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-04-20T11:24:30.445 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-04-20T11:24:30.447 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-04-20T11:24:30.449 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-04-20T11:24:30.451 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-04-20T11:24:30.452 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-04-20T11:24:30.454 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-04-20T11:24:30.456 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-04-20T11:24:30.458 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-04-20T11:24:30.460 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-04-20T11:24:30.462 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-04-20T11:24:30.464 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-04-20T11:24:30.466 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-04-20T11:24:30.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-04-20T11:24:30.469 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-04-20T11:24:30.470 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-04-20T11:24:30.472 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-04-20T11:24:30.474 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-04-20T11:24:30.476 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-04-20T11:24:30.477 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-04-20T11:24:30.479 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-04-20T11:24:30.481 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-04-20T11:24:30.482 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-04-20T11:24:30.484 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-04-20T11:24:30.486 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-04-20T11:24:30.487 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-04-20T11:24:30.489 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-04-20T11:24:30.491 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-04-20T11:24:30.492 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-04-20T11:24:30.494 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-04-20T11:24:30.496 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-04-20T11:24:30.498 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-04-20T11:24:30.500 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-04-20T11:24:30.502 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-04-20T11:24:30.503 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-04-20T11:24:30.505 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-04-20T11:24:30.507 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-04-20T11:24:30.509 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-04-20T11:24:30.510 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-04-20T11:24:30.512 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-04-20T11:24:30.514 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-04-20T11:24:30.515 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-04-20T11:24:30.517 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-04-20T11:24:30.519 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-04-20T11:24:30.521 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-04-20T11:24:30.523 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-04-20T11:24:30.524 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-04-20T11:24:30.526 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-04-20T11:24:30.528 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-04-20T11:24:30.529 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-04-20T11:24:30.531 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-04-20T11:24:30.533 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-04-20T11:24:30.535 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-04-20T11:24:30.536 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-04-20T11:24:30.538 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-04-20T11:24:30.540 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-04-20T11:24:30.541 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-04-20T11:24:30.543 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-04-20T11:24:30.545 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-04-20T11:24:30.547 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-04-20T11:24:30.548 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-04-20T11:24:30.550 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-04-20T11:24:30.552 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-04-20T11:24:30.554 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-04-20T11:24:30.556 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-04-20T11:24:30.557 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-04-20T11:24:30.559 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-04-20T11:24:30.561 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-04-20T11:24:30.563 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-04-20T11:24:30.564 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-04-20T11:24:30.566 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-04-20T11:24:30.568 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-04-20T11:24:30.569 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-04-20T11:24:30.571 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-04-20T11:24:30.573 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-04-20T11:24:30.575 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-04-20T11:24:30.576 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-04-20T11:24:30.578 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-04-20T11:24:30.580 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-04-20T11:24:30.582 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-04-20T11:24:30.584 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-04-20T11:24:30.586 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-04-20T11:24:30.589 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-04-20T11:24:30.591 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-04-20T11:24:30.593 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-04-20T11:24:30.596 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-04-20T11:24:30.598 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-04-20T11:24:30.600 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-04-20T11:24:30.603 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-04-20T11:24:30.606 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-04-20T11:24:30.608 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-04-20T11:24:30.610 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-04-20T11:24:30.612 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-04-20T11:24:30.615 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-04-20T11:24:30.616 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-04-20T11:24:30.619 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-04-20T11:24:30.621 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-04-20T11:24:30.623 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-04-20T11:24:30.626 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-04-20T11:24:30.627 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-04-20T11:24:30.629 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-04-20T11:24:30.631 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-04-20T11:24:30.633 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-04-20T11:24:30.635 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-04-20T11:24:30.638 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-04-20T11:24:30.640 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-04-20T11:24:30.642 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-04-20T11:24:30.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-04-20T11:24:30.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-04-20T11:24:30.648 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-04-20T11:24:30.650 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-04-20T11:24:30.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-04-20T11:24:30.655 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-04-20T11:24:30.657 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-04-20T11:24:30.660 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-04-20T11:24:30.662 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-04-20T11:24:30.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-04-20T11:24:30.667 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-04-20T11:24:30.669 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-04-20T11:24:30.672 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-04-20T11:24:30.675 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-04-20T11:24:30.677 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-04-20T11:24:30.680 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-04-20T11:24:30.682 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-04-20T11:24:30.685 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-04-20T11:24:30.687 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-04-20T11:24:30.689 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-04-20T11:24:30.692 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-04-20T11:24:30.695 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-04-20T11:24:30.697 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-04-20T11:24:30.699 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-04-20T11:24:30.702 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-04-20T11:24:30.705 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-04-20T11:24:30.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-04-20T11:24:30.710 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-04-20T11:24:30.712 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-04-20T11:24:30.715 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-04-20T11:24:30.717 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-04-20T11:24:30.719 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-04-20T11:24:30.722 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-04-20T11:24:30.724 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-04-20T11:24:30.727 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-04-20T11:24:30.729 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-04-20T11:24:30.731 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-04-20T11:24:30.734 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-04-20T11:24:30.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-04-20T11:24:30.739 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-04-20T11:24:30.741 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-04-20T11:24:30.744 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-04-20T11:24:30.747 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-04-20T11:24:30.749 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-04-20T11:24:30.751 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-04-20T11:24:30.753 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-04-20T11:24:30.755 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-04-20T11:24:30.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-04-20T11:24:30.759 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-04-20T11:24:30.761 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-04-20T11:24:30.763 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-04-20T11:24:30.765 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-04-20T11:24:30.767 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-04-20T11:24:30.769 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-04-20T11:24:30.771 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-04-20T11:24:30.773 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-04-20T11:24:30.775 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-04-20T11:24:30.777 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-04-20T11:24:30.779 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-04-20T11:24:30.781 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-04-20T11:24:30.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-04-20T11:24:30.784 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-04-20T11:24:30.786 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-04-20T11:24:30.788 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-04-20T11:24:30.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-04-20T11:24:30.792 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-04-20T11:24:30.794 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-04-20T11:24:30.797 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-04-20T11:24:30.798 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-04-20T11:24:30.800 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-04-20T11:24:30.802 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-04-20T11:24:30.805 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-04-20T11:24:30.806 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-04-20T11:24:30.808 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-04-20T11:24:30.810 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-04-20T11:24:30.812 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-04-20T11:24:30.814 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-04-20T11:24:30.816 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-04-20T11:24:30.819 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-04-20T11:24:30.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-04-20T11:24:30.824 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-04-20T11:24:30.826 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-04-20T11:24:30.828 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-04-20T11:24:30.829 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-04-20T11:24:30.831 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-04-20T11:24:30.834 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-04-20T11:24:30.837 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-04-20T11:24:30.839 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-04-20T11:24:30.841 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-04-20T11:24:30.843 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-04-20T11:24:30.845 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-04-20T11:24:30.847 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-04-20T11:24:30.849 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-04-20T11:24:30.851 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-04-20T11:24:30.853 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-04-20T11:24:30.855 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-04-20T11:24:30.857 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-04-20T11:24:30.858 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-04-20T11:24:30.860 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-04-20T11:24:30.862 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-04-20T11:24:30.863 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-04-20T11:24:30.865 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-04-20T11:24:30.867 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-04-20T11:24:30.869 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-04-20T11:24:30.870 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-04-20T11:24:30.872 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-04-20T11:24:30.873 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-04-20T11:24:30.875 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-04-20T11:24:30.877 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-04-20T11:24:30.879 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-04-20T11:24:30.881 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-04-20T11:24:30.883 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-04-20T11:24:30.885 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-04-20T11:24:30.887 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-04-20T11:24:30.888 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-04-20T11:24:30.890 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-04-20T11:24:30.892 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-04-20T11:24:30.894 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-04-20T11:24:30.896 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-04-20T11:24:30.898 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-04-20T11:24:30.899 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-04-20T11:24:30.901 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-04-20T11:24:30.903 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-04-20T11:24:30.905 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-04-20T11:24:30.906 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-04-20T11:24:30.908 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-04-20T11:24:30.910 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-04-20T11:24:30.912 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-04-20T11:24:30.913 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-04-20T11:24:30.915 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-04-20T11:24:30.917 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-04-20T11:24:30.918 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-04-20T11:24:30.920 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-04-20T11:24:30.922 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-04-20T11:24:30.924 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-04-20T11:24:30.925 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-04-20T11:24:30.927 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-04-20T11:24:30.929 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-04-20T11:24:30.931 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-04-20T11:24:30.933 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-04-20T11:24:30.934 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-04-20T11:24:30.936 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-04-20T11:24:30.937 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-04-20T11:24:30.939 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-04-20T11:24:30.941 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-04-20T11:24:30.942 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-04-20T11:24:30.944 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-04-20T11:24:30.946 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-04-20T11:24:30.948 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-04-20T11:24:30.949 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-04-20T11:24:30.951 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-04-20T11:24:30.953 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-04-20T11:24:30.954 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-04-20T11:24:30.956 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-04-20T11:24:30.958 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-04-20T11:24:30.959 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-04-20T11:24:30.961 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-04-20T11:24:30.963 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-04-20T11:24:30.964 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-04-20T11:24:30.966 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-04-20T11:24:30.968 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-04-20T11:24:30.969 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-04-20T11:24:30.971 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-04-20T11:24:30.972 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-04-20T11:24:30.974 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-04-20T11:24:30.976 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-04-20T11:24:30.977 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-04-20T11:24:30.979 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-04-20T11:24:30.981 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-04-20T11:24:30.982 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-04-20T11:24:30.984 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-04-20T11:24:30.985 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-04-20T11:24:30.987 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-04-20T11:24:30.988 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-04-20T11:24:30.990 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-04-20T11:24:30.992 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-04-20T11:24:30.993 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-04-20T11:24:30.995 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-04-20T11:24:30.996 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-04-20T11:24:30.998 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-04-20T11:24:31.000 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-04-20T11:24:31.001 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-04-20T11:24:31.003 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-04-20T11:24:31.005 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-04-20T11:24:31.006 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-04-20T11:24:31.008 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-04-20T11:24:31.010 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-04-20T11:24:31.011 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-04-20T11:24:31.013 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-04-20T11:24:31.015 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-04-20T11:24:31.016 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-04-20T11:24:31.018 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-04-20T11:24:31.020 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-04-20T11:24:31.021 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-04-20T11:24:31.023 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-04-20T11:24:31.025 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-04-20T11:24:31.027 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-04-20T11:24:31.028 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-04-20T11:24:31.030 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-04-20T11:24:31.031 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-04-20T11:24:31.033 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-04-20T11:24:31.035 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-04-20T11:24:31.037 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-04-20T11:24:31.038 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-04-20T11:24:31.040 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-04-20T11:24:31.041 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-04-20T11:24:31.043 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-04-20T11:24:31.044 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-04-20T11:24:31.046 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-04-20T11:24:31.048 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-04-20T11:24:31.049 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-04-20T11:24:31.051 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-04-20T11:24:31.053 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-04-20T11:24:31.054 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-04-20T11:24:31.056 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-04-20T11:24:31.057 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-04-20T11:24:31.059 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-04-20T11:24:31.060 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-04-20T11:24:31.062 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-04-20T11:24:31.064 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-04-20T11:24:31.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-04-20T11:24:31.067 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-04-20T11:24:31.069 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-04-20T11:24:31.070 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-04-20T11:24:31.072 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-04-20T11:24:31.074 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-04-20T11:24:31.075 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-04-20T11:24:31.077 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-04-20T11:24:31.079 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-04-20T11:24:31.080 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-04-20T11:24:31.082 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-04-20T11:24:31.083 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-04-20T11:24:31.085 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-04-20T11:24:31.087 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-04-20T11:24:31.088 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-04-20T11:24:31.090 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-04-20T11:24:31.092 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-04-20T11:24:31.093 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-04-20T11:24:31.095 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-04-20T11:24:31.097 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-04-20T11:24:31.099 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-04-20T11:24:31.100 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-04-20T11:24:31.102 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-04-20T11:24:31.104 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-04-20T11:24:31.106 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-04-20T11:24:31.107 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-04-20T11:24:31.109 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-04-20T11:24:31.111 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-04-20T11:24:31.112 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-04-20T11:24:31.114 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-04-20T11:24:31.116 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-04-20T11:24:31.118 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-04-20T11:24:31.119 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-04-20T11:24:31.121 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-04-20T11:24:31.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-04-20T11:24:31.124 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-04-20T11:24:31.126 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-04-20T11:24:31.128 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-04-20T11:24:31.130 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-04-20T11:24:31.131 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-04-20T11:24:31.133 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-04-20T11:24:31.134 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-04-20T11:24:31.136 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-04-20T11:24:31.138 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-04-20T11:24:31.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-04-20T11:24:31.141 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-04-20T11:24:31.143 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-04-20T11:24:31.144 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-04-20T11:24:31.146 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-04-20T11:24:31.148 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-04-20T11:24:31.150 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-04-20T11:24:31.151 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-04-20T11:24:31.153 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-04-20T11:24:31.155 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-04-20T11:24:31.156 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-04-20T11:24:31.158 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-04-20T11:24:31.160 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-04-20T11:24:31.162 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-04-20T11:24:31.164 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-04-20T11:24:31.166 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-04-20T11:24:31.169 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-04-20T11:24:31.169 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-04-20T11:24:31.171 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-04-20T11:24:31.173 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-04-20T11:24:31.175 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-04-20T11:24:31.177 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-04-20T11:24:31.178 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-04-20T11:24:31.180 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-04-20T11:24:31.182 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-04-20T11:24:31.183 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-04-20T11:24:31.185 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-04-20T11:24:31.187 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-04-20T11:24:31.188 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-04-20T11:24:31.190 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-04-20T11:24:31.191 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-04-20T11:24:31.193 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-04-20T11:24:31.195 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-04-20T11:24:31.197 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-04-20T11:24:31.199 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-04-20T11:24:31.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-04-20T11:24:31.202 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-04-20T11:24:31.204 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-04-20T11:24:31.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-04-20T11:24:31.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-04-20T11:24:31.209 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-04-20T11:24:31.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-04-20T11:24:31.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-04-20T11:24:31.214 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-04-20T11:24:31.216 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-04-20T11:24:31.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-04-20T11:24:31.219 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-04-20T11:24:31.221 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-04-20T11:24:31.222 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-04-20T11:24:31.224 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-04-20T11:24:31.226 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-04-20T11:24:31.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-04-20T11:24:31.229 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-04-20T11:24:31.231 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-04-20T11:24:31.233 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-04-20T11:24:31.235 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-04-20T11:24:31.236 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-04-20T11:24:31.238 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-04-20T11:24:31.240 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-04-20T11:24:31.241 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-04-20T11:24:31.243 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-04-20T11:24:31.245 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-04-20T11:24:31.247 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-04-20T11:24:31.248 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-04-20T11:24:31.250 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-04-20T11:24:31.252 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-04-20T11:24:31.254 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-04-20T11:24:31.255 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-04-20T11:24:31.257 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-04-20T11:24:31.259 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-04-20T11:24:31.261 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-04-20T11:24:31.262 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-04-20T11:24:31.264 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-04-20T11:24:31.266 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-04-20T11:24:31.267 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-04-20T11:24:31.269 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-04-20T11:24:31.271 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-04-20T11:24:31.272 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-04-20T11:24:31.274 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-04-20T11:24:31.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-04-20T11:24:31.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-04-20T11:24:31.279 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-04-20T11:24:31.281 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-04-20T11:24:31.283 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-04-20T11:24:31.284 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-04-20T11:24:31.286 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-04-20T11:24:31.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-04-20T11:24:31.289 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-04-20T11:24:31.291 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-04-20T11:24:31.293 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-04-20T11:24:31.295 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-04-20T11:24:31.296 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-04-20T11:24:31.298 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-04-20T11:24:31.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-04-20T11:24:31.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-04-20T11:24:31.303 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-04-20T11:24:31.305 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-04-20T11:24:31.306 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-04-20T11:24:31.308 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-04-20T11:24:31.310 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-04-20T11:24:31.312 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-04-20T11:24:31.314 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-04-20T11:24:31.315 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-04-20T11:24:31.317 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-04-20T11:24:31.319 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-04-20T11:24:31.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-04-20T11:24:31.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-04-20T11:24:31.324 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-04-20T11:24:31.325 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-04-20T11:24:31.327 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-04-20T11:24:31.329 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-04-20T11:24:31.330 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-04-20T11:24:31.332 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-04-20T11:24:31.334 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-04-20T11:24:31.336 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-04-20T11:24:31.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-04-20T11:24:31.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-04-20T11:24:31.341 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-04-20T11:24:31.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-04-20T11:24:31.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-04-20T11:24:31.347 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-04-20T11:24:31.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-04-20T11:24:31.351 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-04-20T11:24:31.352 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-04-20T11:24:31.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-04-20T11:24:31.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-04-20T11:24:31.358 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-04-20T11:24:31.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-04-20T11:24:31.361 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-04-20T11:24:31.362 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-04-20T11:24:31.364 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-04-20T11:24:31.366 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-04-20T11:24:31.367 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-04-20T11:24:31.369 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-04-20T11:24:31.371 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-04-20T11:24:31.372 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-04-20T11:24:31.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-04-20T11:24:31.376 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-04-20T11:24:31.378 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-04-20T11:24:31.379 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-04-20T11:24:31.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-04-20T11:24:31.383 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-04-20T11:24:31.385 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-04-20T11:24:31.386 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-04-20T11:24:31.388 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-04-20T11:24:31.390 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-04-20T11:24:31.392 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-04-20T11:24:31.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-04-20T11:24:31.395 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-04-20T11:24:31.397 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-04-20T11:24:31.399 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-04-20T11:24:31.400 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-04-20T11:24:31.402 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-04-20T11:24:31.404 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-04-20T11:24:31.406 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-04-20T11:24:31.408 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-04-20T11:24:31.409 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-04-20T11:24:31.412 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-04-20T11:24:31.421 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-04-20T11:24:31.429 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-04-20T11:24:31.433 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-04-20T11:24:31.436 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-04-20T11:24:31.452 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-04-20T11:24:31.473 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-04-20T11:24:31.533 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-04-20T11:24:31.546 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-04-20T11:24:31.550 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-04-20T11:24:31.553 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-04-20T11:24:31.555 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-04-20T11:24:31.561 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-04-20T11:24:31.562 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-04-20T11:24:31.565 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-04-20T11:24:31.567 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-04-20T11:24:31.569 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-04-20T11:24:31.570 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-04-20T11:24:31.572 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-04-20T11:24:31.573 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-04-20T11:24:31.574 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-04-20T11:24:31.575 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-04-20T11:24:31.576 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-04-20T11:24:31.577 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-04-20T11:24:31.579 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-04-20T11:24:31.581 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-04-20T11:24:31.583 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-04-20T11:24:31.584 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-04-20T11:24:31.585 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-04-20T11:24:31.587 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-04-20T11:24:31.588 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-04-20T11:24:31.590 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-04-20T11:24:31.591 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-04-20T11:24:31.592 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-04-20T11:24:31.594 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-04-20T11:24:31.595 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-04-20T11:24:31.597 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-04-20T11:24:31.598 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-04-20T11:24:31.599 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-04-20T11:24:31.601 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-04-20T11:24:31.602 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-04-20T11:24:31.604 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-04-20T11:24:31.605 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-04-20T11:24:31.607 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-04-20T11:24:31.608 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-04-20T11:24:31.610 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-04-20T11:24:31.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-04-20T11:24:31.613 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-04-20T11:24:31.614 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-04-20T11:24:31.616 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-04-20T11:24:31.617 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-04-20T11:24:31.619 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-04-20T11:24:31.620 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-04-20T11:24:31.622 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-04-20T11:24:31.624 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-04-20T11:24:31.625 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-04-20T11:24:31.627 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-04-20T11:24:31.629 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-04-20T11:24:31.630 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-04-20T11:24:31.632 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-04-20T11:24:31.633 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-04-20T11:24:31.635 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-04-20T11:24:31.637 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-04-20T11:24:31.638 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-04-20T11:24:31.640 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-04-20T11:24:31.641 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-04-20T11:24:31.643 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-04-20T11:24:31.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-04-20T11:24:31.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-04-20T11:24:31.648 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-04-20T11:24:31.650 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-04-20T11:24:31.651 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-04-20T11:24:31.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-04-20T11:24:31.655 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-04-20T11:24:31.656 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-04-20T11:24:31.658 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-04-20T11:24:31.660 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-04-20T11:24:31.662 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-04-20T11:24:31.663 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-04-20T11:24:31.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-04-20T11:24:31.667 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-04-20T11:24:31.669 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-04-20T11:24:31.671 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-04-20T11:24:31.672 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-04-20T11:24:31.674 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-04-20T11:24:31.676 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-04-20T11:24:31.677 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-04-20T11:24:31.679 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-04-20T11:24:31.681 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-04-20T11:24:31.683 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-04-20T11:24:31.684 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-04-20T11:24:31.686 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-04-20T11:24:31.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:31 smithi078 ceph-mon[21464]: pgmap v60: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 14 op/s 2024-04-20T11:24:31.688 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-04-20T11:24:31.689 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-04-20T11:24:31.691 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-04-20T11:24:31.693 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-04-20T11:24:31.694 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-04-20T11:24:31.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:31 smithi044 ceph-mon[16855]: pgmap v60: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 14 op/s 2024-04-20T11:24:31.696 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-04-20T11:24:31.698 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-04-20T11:24:31.699 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-04-20T11:24:31.701 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-04-20T11:24:31.703 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-04-20T11:24:31.704 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-04-20T11:24:31.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-04-20T11:24:31.708 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-04-20T11:24:31.710 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-04-20T11:24:31.711 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-04-20T11:24:31.713 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-04-20T11:24:31.714 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-04-20T11:24:31.716 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-04-20T11:24:31.718 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-04-20T11:24:31.719 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-04-20T11:24:31.721 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-04-20T11:24:31.723 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-04-20T11:24:31.725 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-04-20T11:24:31.726 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-04-20T11:24:31.728 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-04-20T11:24:31.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:31 smithi165 ceph-mon[21975]: pgmap v60: 97 pgs: 97 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 14 op/s 2024-04-20T11:24:31.730 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-04-20T11:24:31.732 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-04-20T11:24:31.733 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-04-20T11:24:31.735 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-04-20T11:24:31.737 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-04-20T11:24:31.739 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-04-20T11:24:31.742 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-04-20T11:24:31.743 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-04-20T11:24:31.745 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-04-20T11:24:31.747 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-04-20T11:24:31.749 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-04-20T11:24:31.751 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-04-20T11:24:31.753 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-04-20T11:24:31.755 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-04-20T11:24:31.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-04-20T11:24:31.759 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-04-20T11:24:31.761 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-04-20T11:24:31.762 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-04-20T11:24:31.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-04-20T11:24:31.766 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-04-20T11:24:31.768 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-04-20T11:24:31.769 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-04-20T11:24:31.771 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-04-20T11:24:31.773 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-04-20T11:24:31.775 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-04-20T11:24:31.776 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-04-20T11:24:31.778 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-04-20T11:24:31.780 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-04-20T11:24:31.782 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-04-20T11:24:31.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-04-20T11:24:31.785 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-04-20T11:24:31.787 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-04-20T11:24:31.789 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-04-20T11:24:31.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-04-20T11:24:31.792 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-04-20T11:24:31.794 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-04-20T11:24:31.795 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-04-20T11:24:31.797 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-04-20T11:24:31.799 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-04-20T11:24:31.801 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-04-20T11:24:31.802 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-04-20T11:24:31.804 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-04-20T11:24:31.806 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-04-20T11:24:31.807 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-04-20T11:24:31.809 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-04-20T11:24:31.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-04-20T11:24:31.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-04-20T11:24:31.814 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-04-20T11:24:31.816 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-04-20T11:24:31.817 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-04-20T11:24:31.819 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-04-20T11:24:31.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-04-20T11:24:31.823 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-04-20T11:24:31.825 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-04-20T11:24:31.827 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-04-20T11:24:31.829 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-04-20T11:24:31.830 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-04-20T11:24:31.832 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-04-20T11:24:31.833 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-04-20T11:24:31.835 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-04-20T11:24:31.837 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-04-20T11:24:31.839 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-04-20T11:24:31.841 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-04-20T11:24:31.842 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-04-20T11:24:31.844 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-04-20T11:24:31.847 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-04-20T11:24:31.849 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-04-20T11:24:31.850 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-04-20T11:24:31.852 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-04-20T11:24:31.854 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-04-20T11:24:31.856 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-04-20T11:24:31.857 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-04-20T11:24:31.859 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-04-20T11:24:31.861 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-04-20T11:24:31.862 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-04-20T11:24:31.864 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-04-20T11:24:32.196 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-04-20T11:24:32.198 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-04-20T11:24:32.201 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-04-20T11:24:32.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-04-20T11:24:32.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-04-20T11:24:32.208 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-04-20T11:24:32.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-04-20T11:24:32.213 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-04-20T11:24:32.214 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-04-20T11:24:32.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-04-20T11:24:32.219 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-04-20T11:24:32.221 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-04-20T11:24:32.224 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-04-20T11:24:32.226 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-04-20T11:24:32.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-04-20T11:24:32.230 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-04-20T11:24:32.233 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-04-20T11:24:32.235 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-04-20T11:24:32.237 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-04-20T11:24:32.239 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-04-20T11:24:32.241 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-04-20T11:24:32.242 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-04-20T11:24:32.245 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-04-20T11:24:32.246 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-04-20T11:24:32.247 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-04-20T11:24:32.249 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-04-20T11:24:32.251 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-04-20T11:24:32.252 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-04-20T11:24:32.254 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-04-20T11:24:32.255 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-04-20T11:24:32.257 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-04-20T11:24:32.259 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-04-20T11:24:32.260 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-04-20T11:24:32.262 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-04-20T11:24:32.264 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-04-20T11:24:32.265 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-04-20T11:24:32.267 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-04-20T11:24:32.269 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-04-20T11:24:32.270 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-04-20T11:24:32.272 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-04-20T11:24:32.274 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-04-20T11:24:32.275 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-04-20T11:24:32.277 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-04-20T11:24:32.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-04-20T11:24:32.280 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-04-20T11:24:32.282 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-04-20T11:24:32.283 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-04-20T11:24:32.285 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-04-20T11:24:32.287 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-04-20T11:24:32.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-04-20T11:24:32.290 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-04-20T11:24:32.292 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-04-20T11:24:32.293 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-04-20T11:24:32.295 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-04-20T11:24:32.297 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-04-20T11:24:32.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-04-20T11:24:32.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-04-20T11:24:32.302 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-04-20T11:24:32.303 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-04-20T11:24:32.305 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-04-20T11:24:32.307 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-04-20T11:24:32.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-04-20T11:24:32.310 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-04-20T11:24:32.312 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-04-20T11:24:32.315 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-04-20T11:24:32.316 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-04-20T11:24:32.318 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-04-20T11:24:32.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-04-20T11:24:32.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-04-20T11:24:32.323 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-04-20T11:24:32.325 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/digsig.txt 2024-04-20T11:24:32.327 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-04-20T11:24:32.329 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-04-20T11:24:32.330 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-04-20T11:24:32.332 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-04-20T11:24:32.333 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-04-20T11:24:32.335 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-04-20T11:24:32.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dontdiff 2024-04-20T11:24:32.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/ 2024-04-20T11:24:32.340 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-04-20T11:24:32.342 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-04-20T11:24:32.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-04-20T11:24:32.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-04-20T11:24:32.347 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-04-20T11:24:32.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-04-20T11:24:32.351 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-04-20T11:24:32.352 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-04-20T11:24:32.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-04-20T11:24:32.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-04-20T11:24:32.357 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/ 2024-04-20T11:24:32.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-04-20T11:24:32.360 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-04-20T11:24:32.362 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-04-20T11:24:32.363 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-04-20T11:24:32.365 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-04-20T11:24:32.367 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-04-20T11:24:32.368 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-04-20T11:24:32.370 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-04-20T11:24:32.371 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-04-20T11:24:32.373 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-04-20T11:24:32.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-04-20T11:24:32.376 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-04-20T11:24:32.377 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-04-20T11:24:32.379 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-04-20T11:24:32.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-04-20T11:24:32.382 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-04-20T11:24:32.384 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-04-20T11:24:32.385 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-04-20T11:24:32.387 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/edac.txt 2024-04-20T11:24:32.389 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-04-20T11:24:32.391 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/eisa.txt 2024-04-20T11:24:32.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-04-20T11:24:32.394 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/extcon/ 2024-04-20T11:24:32.396 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-04-20T11:24:32.397 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-04-20T11:24:32.399 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-04-20T11:24:32.400 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-04-20T11:24:32.402 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-04-20T11:24:32.404 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/ 2024-04-20T11:24:32.405 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-04-20T11:24:32.406 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-04-20T11:24:32.408 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-04-20T11:24:32.410 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-04-20T11:24:32.411 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-04-20T11:24:32.413 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-04-20T11:24:32.414 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-04-20T11:24:32.420 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-04-20T11:24:32.422 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-04-20T11:24:32.424 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-04-20T11:24:32.426 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-04-20T11:24:32.428 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-04-20T11:24:32.430 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-04-20T11:24:32.433 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-04-20T11:24:32.434 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-04-20T11:24:32.437 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-04-20T11:24:32.438 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-04-20T11:24:32.440 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-04-20T11:24:32.444 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-04-20T11:24:32.446 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-04-20T11:24:32.449 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-04-20T11:24:32.450 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-04-20T11:24:32.452 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-04-20T11:24:32.454 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-04-20T11:24:32.457 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-04-20T11:24:32.459 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-04-20T11:24:32.461 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-04-20T11:24:32.463 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-04-20T11:24:32.465 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-04-20T11:24:32.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-04-20T11:24:32.469 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-04-20T11:24:32.471 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-04-20T11:24:32.473 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-04-20T11:24:32.476 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-04-20T11:24:32.478 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-04-20T11:24:32.480 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ 2024-04-20T11:24:32.511 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-04-20T11:24:32.513 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-04-20T11:24:32.515 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-04-20T11:24:32.517 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-04-20T11:24:32.520 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-04-20T11:24:32.522 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-04-20T11:24:32.524 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-04-20T11:24:32.527 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-04-20T11:24:32.529 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-04-20T11:24:32.532 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-04-20T11:24:32.534 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-04-20T11:24:32.536 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-04-20T11:24:32.538 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-04-20T11:24:32.539 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-04-20T11:24:32.541 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-04-20T11:24:32.542 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-04-20T11:24:32.544 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-04-20T11:24:32.546 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-04-20T11:24:32.548 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-04-20T11:24:32.550 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-04-20T11:24:32.551 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-04-20T11:24:32.553 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-04-20T11:24:32.555 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-04-20T11:24:32.556 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-04-20T11:24:32.558 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-04-20T11:24:32.560 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-04-20T11:24:32.562 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-04-20T11:24:32.564 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-04-20T11:24:32.565 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-04-20T11:24:32.567 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-04-20T11:24:32.570 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-04-20T11:24:32.571 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-04-20T11:24:32.572 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-04-20T11:24:32.574 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-04-20T11:24:32.576 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-04-20T11:24:32.578 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-04-20T11:24:32.580 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-04-20T11:24:32.581 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-04-20T11:24:32.583 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-04-20T11:24:32.585 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-04-20T11:24:32.587 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-04-20T11:24:32.588 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-04-20T11:24:32.590 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-04-20T11:24:32.592 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-04-20T11:24:32.593 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-04-20T11:24:32.595 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-04-20T11:24:32.597 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-04-20T11:24:32.599 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-04-20T11:24:32.600 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-04-20T11:24:32.602 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-04-20T11:24:32.604 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-04-20T11:24:32.606 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-04-20T11:24:32.608 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-04-20T11:24:32.610 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-04-20T11:24:32.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-04-20T11:24:32.613 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-04-20T11:24:32.615 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-04-20T11:24:32.616 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-04-20T11:24:32.619 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-04-20T11:24:32.620 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-04-20T11:24:32.622 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-04-20T11:24:32.624 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-04-20T11:24:32.625 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-04-20T11:24:32.627 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-04-20T11:24:32.629 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-04-20T11:24:32.630 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-04-20T11:24:32.632 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-04-20T11:24:32.634 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-04-20T11:24:32.635 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-04-20T11:24:32.637 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-04-20T11:24:32.639 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-04-20T11:24:32.640 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-04-20T11:24:32.642 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-04-20T11:24:32.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-04-20T11:24:32.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-04-20T11:24:32.647 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-04-20T11:24:32.649 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-04-20T11:24:32.651 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-04-20T11:24:32.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-04-20T11:24:32.654 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-04-20T11:24:32.656 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-04-20T11:24:32.658 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-04-20T11:24:32.660 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-04-20T11:24:32.662 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-04-20T11:24:32.664 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-04-20T11:24:32.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-04-20T11:24:32.667 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-04-20T11:24:32.669 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-04-20T11:24:32.670 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-04-20T11:24:32.672 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-04-20T11:24:32.674 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-04-20T11:24:32.676 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-04-20T11:24:32.678 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-04-20T11:24:32.680 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-04-20T11:24:32.682 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-04-20T11:24:32.684 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-04-20T11:24:32.686 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-04-20T11:24:32.688 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-04-20T11:24:32.689 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-04-20T11:24:32.691 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-04-20T11:24:32.693 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-04-20T11:24:32.695 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-04-20T11:24:32.697 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-04-20T11:24:32.699 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-04-20T11:24:32.701 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-04-20T11:24:32.703 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-04-20T11:24:32.705 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-04-20T11:24:32.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-04-20T11:24:32.709 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-04-20T11:24:32.710 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-04-20T11:24:32.712 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-04-20T11:24:32.713 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-04-20T11:24:32.715 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-04-20T11:24:32.717 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-04-20T11:24:32.720 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-04-20T11:24:32.721 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-04-20T11:24:32.723 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-04-20T11:24:32.725 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-04-20T11:24:32.726 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-04-20T11:24:32.728 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/ 2024-04-20T11:24:32.730 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-04-20T11:24:32.731 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-04-20T11:24:32.733 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-04-20T11:24:32.735 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-04-20T11:24:32.737 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-04-20T11:24:32.738 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-04-20T11:24:32.740 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-04-20T11:24:32.742 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-04-20T11:24:32.744 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-04-20T11:24:32.747 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-04-20T11:24:32.750 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-04-20T11:24:32.754 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/ 2024-04-20T11:24:32.755 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-04-20T11:24:32.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-04-20T11:24:32.760 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-04-20T11:24:32.762 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-04-20T11:24:32.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-04-20T11:24:32.766 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-04-20T11:24:32.768 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-04-20T11:24:32.770 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-04-20T11:24:32.772 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-04-20T11:24:32.776 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-04-20T11:24:32.777 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-04-20T11:24:32.780 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gcov.txt 2024-04-20T11:24:32.782 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-04-20T11:24:32.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/ 2024-04-20T11:24:32.785 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-04-20T11:24:32.787 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-04-20T11:24:32.788 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-04-20T11:24:32.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-04-20T11:24:32.792 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-04-20T11:24:32.793 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-04-20T11:24:32.795 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-04-20T11:24:32.797 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hid/ 2024-04-20T11:24:32.798 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-04-20T11:24:32.800 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-04-20T11:24:32.802 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-04-20T11:24:32.804 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-04-20T11:24:32.806 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-04-20T11:24:32.807 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/highuid.txt 2024-04-20T11:24:32.809 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hsi.txt 2024-04-20T11:24:32.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-04-20T11:24:32.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ 2024-04-20T11:24:32.814 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-04-20T11:24:32.816 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-04-20T11:24:32.818 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-04-20T11:24:32.819 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-04-20T11:24:32.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-04-20T11:24:32.823 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-04-20T11:24:32.824 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-04-20T11:24:32.826 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-04-20T11:24:32.828 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-04-20T11:24:32.830 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-04-20T11:24:32.831 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-04-20T11:24:32.833 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-04-20T11:24:32.835 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-04-20T11:24:32.836 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-04-20T11:24:32.838 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-04-20T11:24:32.840 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-04-20T11:24:32.842 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-04-20T11:24:32.844 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-04-20T11:24:32.846 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-04-20T11:24:32.848 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-04-20T11:24:32.850 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-04-20T11:24:32.853 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-04-20T11:24:32.855 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-04-20T11:24:32.857 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-04-20T11:24:32.858 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-04-20T11:24:32.860 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-04-20T11:24:32.862 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-04-20T11:24:32.863 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-04-20T11:24:32.865 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-04-20T11:24:32.867 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-04-20T11:24:32.869 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-04-20T11:24:32.871 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-04-20T11:24:32.872 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-04-20T11:24:32.874 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-04-20T11:24:32.876 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-04-20T11:24:32.877 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-04-20T11:24:32.879 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-04-20T11:24:32.881 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-04-20T11:24:32.883 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-04-20T11:24:32.885 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-04-20T11:24:32.886 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-04-20T11:24:32.888 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-04-20T11:24:32.890 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-04-20T11:24:32.892 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-04-20T11:24:32.894 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-04-20T11:24:32.895 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-04-20T11:24:32.897 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-04-20T11:24:32.899 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-04-20T11:24:32.901 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-04-20T11:24:32.903 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-04-20T11:24:32.905 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-04-20T11:24:32.907 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-04-20T11:24:32.909 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-04-20T11:24:32.911 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-04-20T11:24:32.913 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-04-20T11:24:32.915 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-04-20T11:24:32.916 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-04-20T11:24:32.918 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-04-20T11:24:32.920 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-04-20T11:24:32.922 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-04-20T11:24:32.924 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-04-20T11:24:32.926 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-04-20T11:24:32.928 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-04-20T11:24:32.930 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-04-20T11:24:32.931 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-04-20T11:24:32.933 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-04-20T11:24:32.935 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-04-20T11:24:32.937 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-04-20T11:24:32.939 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-04-20T11:24:32.940 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-04-20T11:24:32.942 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-04-20T11:24:32.944 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-04-20T11:24:32.945 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-04-20T11:24:32.947 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-04-20T11:24:32.949 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-04-20T11:24:32.951 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-04-20T11:24:32.952 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-04-20T11:24:32.954 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-04-20T11:24:32.956 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-04-20T11:24:32.957 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-04-20T11:24:32.959 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-04-20T11:24:32.961 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-04-20T11:24:32.962 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-04-20T11:24:32.964 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-04-20T11:24:32.966 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-04-20T11:24:32.967 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-04-20T11:24:32.969 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-04-20T11:24:32.971 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-04-20T11:24:32.972 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-04-20T11:24:32.976 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-04-20T11:24:32.978 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-04-20T11:24:32.980 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-04-20T11:24:32.982 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-04-20T11:24:32.984 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-04-20T11:24:32.985 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-04-20T11:24:32.987 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-04-20T11:24:32.989 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-04-20T11:24:32.991 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-04-20T11:24:32.993 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-04-20T11:24:32.995 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-04-20T11:24:32.997 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-04-20T11:24:32.998 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-04-20T11:24:33.000 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-04-20T11:24:33.002 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-04-20T11:24:33.003 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-04-20T11:24:33.005 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-04-20T11:24:33.009 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-04-20T11:24:33.011 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-04-20T11:24:33.012 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-04-20T11:24:33.014 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-04-20T11:24:33.016 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-04-20T11:24:33.019 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-04-20T11:24:33.021 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-04-20T11:24:33.022 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-04-20T11:24:33.024 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-04-20T11:24:33.025 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-04-20T11:24:33.027 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-04-20T11:24:33.028 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-04-20T11:24:33.030 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-04-20T11:24:33.032 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-04-20T11:24:33.033 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-04-20T11:24:33.035 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-04-20T11:24:33.036 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-04-20T11:24:33.038 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-04-20T11:24:33.039 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-04-20T11:24:33.041 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-04-20T11:24:33.042 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-04-20T11:24:33.044 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-04-20T11:24:33.046 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-04-20T11:24:33.048 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-04-20T11:24:33.049 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-04-20T11:24:33.051 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-04-20T11:24:33.053 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-04-20T11:24:33.055 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-04-20T11:24:33.056 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-04-20T11:24:33.058 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-04-20T11:24:33.060 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-04-20T11:24:33.062 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/ 2024-04-20T11:24:33.063 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-04-20T11:24:33.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-04-20T11:24:33.066 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-04-20T11:24:33.068 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-04-20T11:24:33.070 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-04-20T11:24:33.071 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-04-20T11:24:33.073 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-04-20T11:24:33.075 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-04-20T11:24:33.077 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-04-20T11:24:33.079 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-04-20T11:24:33.080 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-04-20T11:24:33.082 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-04-20T11:24:33.084 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-04-20T11:24:33.086 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-04-20T11:24:33.087 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-04-20T11:24:33.089 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-04-20T11:24:33.091 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-04-20T11:24:33.093 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-04-20T11:24:33.094 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-04-20T11:24:33.096 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-04-20T11:24:33.098 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-04-20T11:24:33.099 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-04-20T11:24:33.101 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-04-20T11:24:33.103 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-04-20T11:24:33.105 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-04-20T11:24:33.107 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-04-20T11:24:33.109 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-04-20T11:24:33.110 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-04-20T11:24:33.112 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-04-20T11:24:33.114 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-04-20T11:24:33.115 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-04-20T11:24:33.117 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-04-20T11:24:33.120 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/summary 2024-04-20T11:24:33.121 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-04-20T11:24:33.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-04-20T11:24:33.124 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-04-20T11:24:33.126 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2o/ 2024-04-20T11:24:33.128 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2o/README 2024-04-20T11:24:33.130 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-04-20T11:24:33.132 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/ 2024-04-20T11:24:33.134 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-04-20T11:24:33.135 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-04-20T11:24:33.137 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-04-20T11:24:33.138 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/README 2024-04-20T11:24:33.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-04-20T11:24:33.142 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-04-20T11:24:33.144 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-04-20T11:24:33.145 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-04-20T11:24:33.147 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-04-20T11:24:33.149 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-04-20T11:24:33.151 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-04-20T11:24:33.152 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-04-20T11:24:33.154 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/ 2024-04-20T11:24:33.156 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-04-20T11:24:33.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-04-20T11:24:33.161 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-04-20T11:24:33.162 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-04-20T11:24:33.164 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-04-20T11:24:33.166 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-04-20T11:24:33.168 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-04-20T11:24:33.170 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/infiniband/ 2024-04-20T11:24:33.172 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-04-20T11:24:33.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-04-20T11:24:33.176 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-04-20T11:24:33.177 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-04-20T11:24:33.179 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-04-20T11:24:33.181 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/init.txt 2024-04-20T11:24:33.183 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/initrd.txt 2024-04-20T11:24:33.185 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/ 2024-04-20T11:24:33.187 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-04-20T11:24:33.188 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-04-20T11:24:33.191 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-04-20T11:24:33.192 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-04-20T11:24:33.194 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-04-20T11:24:33.196 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-04-20T11:24:33.198 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-04-20T11:24:33.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-04-20T11:24:33.201 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-04-20T11:24:33.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-04-20T11:24:33.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-04-20T11:24:33.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-04-20T11:24:33.209 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-04-20T11:24:33.211 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-04-20T11:24:33.213 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-04-20T11:24:33.215 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-04-20T11:24:33.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-04-20T11:24:33.219 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/input.txt 2024-04-20T11:24:33.221 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-04-20T11:24:33.222 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-04-20T11:24:33.224 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-04-20T11:24:33.226 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-04-20T11:24:33.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-04-20T11:24:33.231 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-04-20T11:24:33.232 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-04-20T11:24:33.234 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-04-20T11:24:33.236 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-04-20T11:24:33.238 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-04-20T11:24:33.240 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-04-20T11:24:33.241 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-04-20T11:24:33.243 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-04-20T11:24:33.245 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-04-20T11:24:33.246 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-04-20T11:24:33.248 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-04-20T11:24:33.250 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/ 2024-04-20T11:24:33.252 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-04-20T11:24:33.253 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-04-20T11:24:33.255 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-04-20T11:24:33.257 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-04-20T11:24:33.259 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-04-20T11:24:33.260 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-04-20T11:24:33.262 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/iostats.txt 2024-04-20T11:24:33.264 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-04-20T11:24:33.266 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-04-20T11:24:33.268 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/ 2024-04-20T11:24:33.269 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-04-20T11:24:33.271 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-04-20T11:24:33.273 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-04-20T11:24:33.274 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-04-20T11:24:33.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-04-20T11:24:33.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-04-20T11:24:33.280 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README 2024-04-20T11:24:33.282 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-04-20T11:24:33.283 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-04-20T11:24:33.285 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-04-20T11:24:33.287 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-04-20T11:24:33.289 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-04-20T11:24:33.290 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-04-20T11:24:33.292 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-04-20T11:24:33.294 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-04-20T11:24:33.295 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-04-20T11:24:33.297 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-04-20T11:24:33.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-04-20T11:24:33.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-04-20T11:24:33.303 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-04-20T11:24:33.305 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-04-20T11:24:33.307 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-04-20T11:24:33.308 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-04-20T11:24:33.310 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-04-20T11:24:33.312 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-04-20T11:24:33.314 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-04-20T11:24:33.316 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-04-20T11:24:33.318 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-04-20T11:24:33.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-04-20T11:24:33.322 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-04-20T11:24:33.324 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-04-20T11:24:33.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/java.txt 2024-04-20T11:24:33.328 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kasan.txt 2024-04-20T11:24:33.330 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/ 2024-04-20T11:24:33.332 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-04-20T11:24:33.334 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-04-20T11:24:33.336 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-04-20T11:24:33.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-04-20T11:24:33.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-04-20T11:24:33.341 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-04-20T11:24:33.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-04-20T11:24:33.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kdump/ 2024-04-20T11:24:33.348 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-04-20T11:24:33.350 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-04-20T11:24:33.352 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-04-20T11:24:33.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-04-20T11:24:33.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-04-20T11:24:33.364 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-04-20T11:24:33.384 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-04-20T11:24:33.429 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-04-20T11:24:33.604 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-04-20T11:24:33.630 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-04-20T11:24:33.641 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-04-20T11:24:33.645 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kobject.txt 2024-04-20T11:24:33.650 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-04-20T11:24:33.654 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kref.txt 2024-04-20T11:24:33.657 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-04-20T11:24:33.662 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/ 2024-04-20T11:24:33.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-04-20T11:24:33.670 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-04-20T11:24:33.674 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-04-20T11:24:33.678 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-04-20T11:24:33.682 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-04-20T11:24:33.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:33 smithi078 ceph-mon[21464]: pgmap v61: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 7.1 MiB/s wr, 37 op/s 2024-04-20T11:24:33.687 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-04-20T11:24:33.692 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-04-20T11:24:33.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:33 smithi044 ceph-mon[16855]: pgmap v61: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 7.1 MiB/s wr, 37 op/s 2024-04-20T11:24:33.696 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-04-20T11:24:33.701 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-04-20T11:24:33.706 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-04-20T11:24:33.709 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-04-20T11:24:33.713 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-04-20T11:24:33.717 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ldm.txt 2024-04-20T11:24:33.722 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/ 2024-04-20T11:24:33.726 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-04-20T11:24:33.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:33 smithi165 ceph-mon[21975]: pgmap v61: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 7.1 MiB/s wr, 37 op/s 2024-04-20T11:24:33.730 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-04-20T11:24:33.734 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-04-20T11:24:33.739 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-04-20T11:24:33.744 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-04-20T11:24:33.749 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-04-20T11:24:33.754 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-04-20T11:24:33.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-04-20T11:24:33.761 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-04-20T11:24:33.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-04-20T11:24:33.767 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-04-20T11:24:33.770 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-04-20T11:24:33.773 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/ 2024-04-20T11:24:33.776 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-04-20T11:24:33.779 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-04-20T11:24:33.781 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-04-20T11:24:33.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-04-20T11:24:33.785 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-04-20T11:24:33.787 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-04-20T11:24:33.790 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-04-20T11:24:33.794 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-04-20T11:24:33.796 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-04-20T11:24:33.798 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-04-20T11:24:33.800 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-04-20T11:24:33.803 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/logo.gif 2024-04-20T11:24:33.805 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/logo.txt 2024-04-20T11:24:33.807 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/lzo.txt 2024-04-20T11:24:33.809 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/m68k/ 2024-04-20T11:24:33.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-04-20T11:24:33.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-04-20T11:24:33.814 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-04-20T11:24:33.816 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-04-20T11:24:33.819 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-04-20T11:24:33.821 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/md.txt 2024-04-20T11:24:33.824 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-04-20T11:24:33.826 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-04-20T11:24:34.113 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-04-20T11:24:34.115 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-04-20T11:24:34.118 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-04-20T11:24:34.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/metag/ 2024-04-20T11:24:34.125 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-04-20T11:24:34.127 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-04-20T11:24:34.130 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/ 2024-04-20T11:24:34.131 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-04-20T11:24:34.133 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-04-20T11:24:34.135 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-04-20T11:24:34.136 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-04-20T11:24:34.138 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-04-20T11:24:34.140 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-04-20T11:24:34.142 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-04-20T11:24:34.144 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-04-20T11:24:34.146 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-04-20T11:24:34.151 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-04-20T11:24:34.153 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mips/ 2024-04-20T11:24:34.155 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-04-20T11:24:34.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-04-20T11:24:34.158 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-04-20T11:24:34.160 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-04-20T11:24:34.163 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-04-20T11:24:34.165 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-04-20T11:24:34.167 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-04-20T11:24:34.168 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-04-20T11:24:34.170 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-04-20T11:24:34.172 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-04-20T11:24:34.174 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-04-20T11:24:34.176 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-04-20T11:24:34.179 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-04-20T11:24:34.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-04-20T11:24:34.202 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-04-20T11:24:34.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-04-20T11:24:34.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-04-20T11:24:34.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-04-20T11:24:34.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-04-20T11:24:34.213 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-04-20T11:24:34.215 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-04-20T11:24:34.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mmc/ 2024-04-20T11:24:34.219 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-04-20T11:24:34.220 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-04-20T11:24:34.222 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-04-20T11:24:34.224 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-04-20T11:24:34.226 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mn10300/ 2024-04-20T11:24:34.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-04-20T11:24:34.229 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-04-20T11:24:34.231 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-04-20T11:24:34.233 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mono.txt 2024-04-20T11:24:34.235 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mtd/ 2024-04-20T11:24:34.236 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-04-20T11:24:34.238 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-04-20T11:24:34.239 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-04-20T11:24:34.241 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-04-20T11:24:34.243 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/namespaces/ 2024-04-20T11:24:34.244 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-04-20T11:24:34.246 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-04-20T11:24:34.248 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/netlabel/ 2024-04-20T11:24:34.249 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-04-20T11:24:34.251 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-04-20T11:24:34.252 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-04-20T11:24:34.254 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-04-20T11:24:34.256 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-04-20T11:24:34.258 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ 2024-04-20T11:24:34.259 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-04-20T11:24:34.261 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-04-20T11:24:34.263 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-04-20T11:24:34.264 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-04-20T11:24:34.266 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-04-20T11:24:34.268 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-04-20T11:24:34.269 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-04-20T11:24:34.271 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-04-20T11:24:34.273 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-04-20T11:24:34.274 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-04-20T11:24:34.276 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-04-20T11:24:34.278 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-04-20T11:24:34.280 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-04-20T11:24:34.281 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-04-20T11:24:34.285 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-04-20T11:24:34.287 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-04-20T11:24:34.288 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-04-20T11:24:34.290 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-04-20T11:24:34.292 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-04-20T11:24:34.293 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-04-20T11:24:34.297 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-04-20T11:24:34.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-04-20T11:24:34.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-04-20T11:24:34.302 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-04-20T11:24:34.304 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-04-20T11:24:34.305 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-04-20T11:24:34.307 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-04-20T11:24:34.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-04-20T11:24:34.311 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-04-20T11:24:34.313 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-04-20T11:24:34.314 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-04-20T11:24:34.316 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-04-20T11:24:34.318 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-04-20T11:24:34.319 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-04-20T11:24:34.321 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-04-20T11:24:34.323 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-04-20T11:24:34.325 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-04-20T11:24:34.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-04-20T11:24:34.328 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-04-20T11:24:34.330 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-04-20T11:24:34.332 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-04-20T11:24:34.334 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-04-20T11:24:34.335 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-04-20T11:24:34.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-04-20T11:24:34.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-04-20T11:24:34.341 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-04-20T11:24:34.343 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-04-20T11:24:34.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-04-20T11:24:34.347 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-04-20T11:24:34.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-04-20T11:24:34.351 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-04-20T11:24:34.352 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-04-20T11:24:34.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-04-20T11:24:34.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-04-20T11:24:34.358 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-04-20T11:24:34.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-04-20T11:24:34.361 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-04-20T11:24:34.363 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-04-20T11:24:34.367 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-04-20T11:24:34.370 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-04-20T11:24:34.371 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-04-20T11:24:34.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-04-20T11:24:34.375 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-04-20T11:24:34.377 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-04-20T11:24:34.379 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-04-20T11:24:34.380 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-04-20T11:24:34.382 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-04-20T11:24:34.384 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-04-20T11:24:34.386 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-04-20T11:24:34.387 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-04-20T11:24:34.389 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-04-20T11:24:34.391 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-04-20T11:24:34.393 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-04-20T11:24:34.395 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-04-20T11:24:34.396 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-04-20T11:24:34.398 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-04-20T11:24:34.399 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-04-20T11:24:34.401 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-04-20T11:24:34.402 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-04-20T11:24:34.404 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-04-20T11:24:34.405 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-04-20T11:24:34.407 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-04-20T11:24:34.408 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-04-20T11:24:34.410 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-04-20T11:24:34.412 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-04-20T11:24:34.413 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-04-20T11:24:34.415 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-04-20T11:24:34.416 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-04-20T11:24:34.418 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-04-20T11:24:34.420 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-04-20T11:24:34.421 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-04-20T11:24:34.423 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-04-20T11:24:34.425 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-04-20T11:24:34.426 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-04-20T11:24:34.428 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-04-20T11:24:34.430 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-04-20T11:24:34.431 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-04-20T11:24:34.433 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-04-20T11:24:34.435 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-04-20T11:24:34.436 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-04-20T11:24:34.438 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-04-20T11:24:34.440 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-04-20T11:24:34.441 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-04-20T11:24:34.443 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-04-20T11:24:34.445 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-04-20T11:24:34.447 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-04-20T11:24:34.449 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-04-20T11:24:34.450 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-04-20T11:24:34.452 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-04-20T11:24:34.453 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-04-20T11:24:34.455 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-04-20T11:24:34.457 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-04-20T11:24:34.459 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-04-20T11:24:34.460 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-04-20T11:24:34.462 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-04-20T11:24:34.464 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-04-20T11:24:34.466 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-04-20T11:24:34.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-04-20T11:24:34.469 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-04-20T11:24:34.471 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-04-20T11:24:34.473 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-04-20T11:24:34.474 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-04-20T11:24:34.476 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-04-20T11:24:34.478 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-04-20T11:24:34.480 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-04-20T11:24:34.481 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-04-20T11:24:34.483 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-04-20T11:24:34.485 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-04-20T11:24:34.486 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-04-20T11:24:34.488 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-04-20T11:24:34.490 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-04-20T11:24:34.492 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-04-20T11:24:34.493 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-04-20T11:24:34.495 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-04-20T11:24:34.497 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-04-20T11:24:34.499 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/nfc/ 2024-04-20T11:24:34.500 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-04-20T11:24:34.502 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-04-20T11:24:34.503 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/nios2/ 2024-04-20T11:24:34.505 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/nios2/README 2024-04-20T11:24:34.509 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-04-20T11:24:34.511 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/numastat.txt 2024-04-20T11:24:34.513 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-04-20T11:24:34.514 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/padata.txt 2024-04-20T11:24:34.516 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/parisc/ 2024-04-20T11:24:34.517 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-04-20T11:24:34.519 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-04-20T11:24:34.520 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/parisc/registers 2024-04-20T11:24:34.522 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-04-20T11:24:34.524 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/parport.txt 2024-04-20T11:24:34.526 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-04-20T11:24:34.528 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-04-20T11:24:34.529 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-04-20T11:24:34.531 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-04-20T11:24:34.533 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-04-20T11:24:34.534 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-04-20T11:24:34.536 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-04-20T11:24:34.538 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-04-20T11:24:34.539 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-04-20T11:24:34.541 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/phy.txt 2024-04-20T11:24:34.543 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/phy/ 2024-04-20T11:24:34.545 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-04-20T11:24:34.546 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-04-20T11:24:34.548 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-04-20T11:24:34.550 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/platform/ 2024-04-20T11:24:34.552 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-04-20T11:24:34.554 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pnp.txt 2024-04-20T11:24:34.556 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/ 2024-04-20T11:24:34.557 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-04-20T11:24:34.559 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-04-20T11:24:34.561 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-04-20T11:24:34.562 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-04-20T11:24:34.564 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-04-20T11:24:34.566 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-04-20T11:24:34.568 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-04-20T11:24:34.570 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-04-20T11:24:34.571 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-04-20T11:24:34.573 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-04-20T11:24:34.575 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-04-20T11:24:34.577 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-04-20T11:24:34.578 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-04-20T11:24:34.580 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-04-20T11:24:34.582 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-04-20T11:24:34.583 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-04-20T11:24:34.585 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-04-20T11:24:34.587 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-04-20T11:24:34.588 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-04-20T11:24:34.590 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-04-20T11:24:34.592 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-04-20T11:24:34.593 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-04-20T11:24:34.596 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-04-20T11:24:34.597 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/states.txt 2024-04-20T11:24:34.599 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-04-20T11:24:34.600 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-04-20T11:24:34.602 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-04-20T11:24:34.604 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-04-20T11:24:34.606 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-04-20T11:24:34.608 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-04-20T11:24:34.610 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-04-20T11:24:34.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/power/video.txt 2024-04-20T11:24:34.613 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/ 2024-04-20T11:24:34.615 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-04-20T11:24:34.616 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-04-20T11:24:34.618 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-04-20T11:24:34.620 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-04-20T11:24:34.622 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-04-20T11:24:34.623 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-04-20T11:24:34.625 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-04-20T11:24:34.627 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-04-20T11:24:34.629 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-04-20T11:24:34.631 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-04-20T11:24:34.632 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-04-20T11:24:34.634 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-04-20T11:24:34.636 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-04-20T11:24:34.638 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pps/ 2024-04-20T11:24:34.639 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-04-20T11:24:34.641 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/ 2024-04-20T11:24:34.643 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-04-20T11:24:34.644 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-04-20T11:24:34.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-04-20T11:24:34.647 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-04-20T11:24:34.649 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-04-20T11:24:34.651 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-04-20T11:24:34.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-04-20T11:24:34.654 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-04-20T11:24:34.656 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-04-20T11:24:34.658 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pti/ 2024-04-20T11:24:34.660 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-04-20T11:24:34.661 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ptp/ 2024-04-20T11:24:34.663 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-04-20T11:24:34.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-04-20T11:24:34.666 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-04-20T11:24:34.668 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-04-20T11:24:34.670 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-04-20T11:24:34.671 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/pwm.txt 2024-04-20T11:24:34.674 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-04-20T11:24:34.675 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rapidio/ 2024-04-20T11:24:34.677 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-04-20T11:24:34.679 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-04-20T11:24:34.681 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-04-20T11:24:34.682 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-04-20T11:24:34.684 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-04-20T11:24:34.686 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-04-20T11:24:34.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:34 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:34.688 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-04-20T11:24:34.690 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-04-20T11:24:34.691 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-04-20T11:24:34.694 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/rtc.txt 2024-04-20T11:24:34.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:34 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:34.695 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/ 2024-04-20T11:24:34.697 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-04-20T11:24:34.699 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-04-20T11:24:34.700 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-04-20T11:24:34.702 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-04-20T11:24:34.705 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/DASD 2024-04-20T11:24:34.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-04-20T11:24:34.710 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-04-20T11:24:34.713 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-04-20T11:24:34.714 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-04-20T11:24:34.716 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-04-20T11:24:34.718 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-04-20T11:24:34.721 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-04-20T11:24:34.722 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-04-20T11:24:34.725 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-04-20T11:24:34.727 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/ 2024-04-20T11:24:34.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:34 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:34.729 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-04-20T11:24:34.731 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-04-20T11:24:34.734 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-04-20T11:24:34.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-04-20T11:24:34.738 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-04-20T11:24:34.739 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-04-20T11:24:34.741 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-04-20T11:24:34.743 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-04-20T11:24:34.745 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-04-20T11:24:34.746 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-04-20T11:24:34.748 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ 2024-04-20T11:24:34.749 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-04-20T11:24:34.754 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-04-20T11:24:34.756 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-04-20T11:24:34.758 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-04-20T11:24:34.760 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-04-20T11:24:34.762 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-04-20T11:24:34.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-04-20T11:24:34.768 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-04-20T11:24:34.770 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-04-20T11:24:34.772 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-04-20T11:24:34.774 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-04-20T11:24:34.776 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-04-20T11:24:34.778 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-04-20T11:24:34.780 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-04-20T11:24:34.782 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-04-20T11:24:34.784 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-04-20T11:24:34.786 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-04-20T11:24:34.787 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-04-20T11:24:34.789 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-04-20T11:24:34.791 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-04-20T11:24:34.793 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-04-20T11:24:34.795 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-04-20T11:24:34.797 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-04-20T11:24:34.799 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-04-20T11:24:34.801 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-04-20T11:24:34.803 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-04-20T11:24:34.804 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-04-20T11:24:34.806 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-04-20T11:24:34.808 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-04-20T11:24:34.810 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-04-20T11:24:34.811 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-04-20T11:24:34.813 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-04-20T11:24:34.815 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-04-20T11:24:34.817 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-04-20T11:24:34.819 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-04-20T11:24:34.820 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-04-20T11:24:34.822 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-04-20T11:24:34.824 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-04-20T11:24:34.826 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-04-20T11:24:34.828 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-04-20T11:24:34.830 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-04-20T11:24:34.832 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-04-20T11:24:34.834 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-04-20T11:24:34.835 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-04-20T11:24:34.837 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-04-20T11:24:34.840 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-04-20T11:24:34.841 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-04-20T11:24:34.843 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-04-20T11:24:34.845 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-04-20T11:24:34.847 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-04-20T11:24:34.850 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-04-20T11:24:34.851 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-04-20T11:24:34.853 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-04-20T11:24:34.854 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-04-20T11:24:34.856 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-04-20T11:24:34.858 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-04-20T11:24:34.860 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-04-20T11:24:34.862 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-04-20T11:24:34.863 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-04-20T11:24:34.865 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/ 2024-04-20T11:24:34.867 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-04-20T11:24:34.868 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-04-20T11:24:34.870 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-04-20T11:24:34.873 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-04-20T11:24:34.875 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-04-20T11:24:34.877 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-04-20T11:24:34.879 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-04-20T11:24:34.882 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-04-20T11:24:34.884 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-04-20T11:24:34.886 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-04-20T11:24:34.889 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-04-20T11:24:34.891 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-04-20T11:24:34.894 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-04-20T11:24:34.897 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-04-20T11:24:34.898 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/ 2024-04-20T11:24:34.900 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-04-20T11:24:34.902 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-04-20T11:24:34.905 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/driver 2024-04-20T11:24:34.907 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-04-20T11:24:34.909 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-04-20T11:24:34.911 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-04-20T11:24:34.914 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-04-20T11:24:34.916 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-04-20T11:24:34.918 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-04-20T11:24:34.921 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sh/ 2024-04-20T11:24:34.923 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-04-20T11:24:34.925 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-04-20T11:24:34.927 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-04-20T11:24:34.929 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/ 2024-04-20T11:24:34.931 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-04-20T11:24:34.933 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-04-20T11:24:34.936 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-04-20T11:24:34.938 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-04-20T11:24:34.940 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-04-20T11:24:34.942 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-04-20T11:24:34.945 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-04-20T11:24:34.947 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-04-20T11:24:34.949 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-04-20T11:24:34.952 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-04-20T11:24:34.954 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-04-20T11:24:34.956 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-04-20T11:24:34.958 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-04-20T11:24:34.961 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-04-20T11:24:34.963 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-04-20T11:24:34.965 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-04-20T11:24:34.967 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-04-20T11:24:34.969 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-04-20T11:24:34.972 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-04-20T11:24:34.974 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-04-20T11:24:34.976 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-04-20T11:24:34.978 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-04-20T11:24:34.980 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-04-20T11:24:34.983 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-04-20T11:24:34.985 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-04-20T11:24:34.987 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-04-20T11:24:34.989 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-04-20T11:24:34.991 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-04-20T11:24:34.993 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-04-20T11:24:34.996 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-04-20T11:24:34.998 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-04-20T11:24:35.000 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-04-20T11:24:35.002 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-04-20T11:24:35.005 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-04-20T11:24:35.007 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-04-20T11:24:35.010 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-04-20T11:24:35.012 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-04-20T11:24:35.014 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-04-20T11:24:35.016 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-04-20T11:24:35.018 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-04-20T11:24:35.020 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-04-20T11:24:35.022 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-04-20T11:24:35.024 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-04-20T11:24:35.027 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-04-20T11:24:35.029 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-04-20T11:24:35.031 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-04-20T11:24:35.033 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-04-20T11:24:35.035 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-04-20T11:24:35.038 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-04-20T11:24:35.040 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-04-20T11:24:35.042 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-04-20T11:24:35.045 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-04-20T11:24:35.047 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-04-20T11:24:35.049 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-04-20T11:24:35.051 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-04-20T11:24:35.053 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-04-20T11:24:35.055 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-04-20T11:24:35.058 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-04-20T11:24:35.060 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-04-20T11:24:35.062 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-04-20T11:24:35.065 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-04-20T11:24:35.067 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-04-20T11:24:35.069 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sparse.txt 2024-04-20T11:24:35.073 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/ 2024-04-20T11:24:35.076 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-04-20T11:24:35.078 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-04-20T11:24:35.080 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-04-20T11:24:35.082 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-04-20T11:24:35.084 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-04-20T11:24:35.085 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-04-20T11:24:35.087 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-04-20T11:24:35.089 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-04-20T11:24:35.091 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-04-20T11:24:35.093 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/spidev 2024-04-20T11:24:35.095 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-04-20T11:24:35.097 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-04-20T11:24:35.098 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-04-20T11:24:35.102 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-04-20T11:24:35.102 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-04-20T11:24:35.105 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/svga.txt 2024-04-20T11:24:35.106 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/ 2024-04-20T11:24:35.108 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-04-20T11:24:35.110 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/README 2024-04-20T11:24:35.112 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-04-20T11:24:35.113 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-04-20T11:24:35.115 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-04-20T11:24:35.117 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-04-20T11:24:35.119 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-04-20T11:24:35.121 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-04-20T11:24:35.123 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-04-20T11:24:35.125 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-04-20T11:24:35.127 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/target/ 2024-04-20T11:24:35.129 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-04-20T11:24:35.131 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-04-20T11:24:35.133 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-04-20T11:24:35.134 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/ 2024-04-20T11:24:35.136 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-04-20T11:24:35.138 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-04-20T11:24:35.139 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-04-20T11:24:35.141 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-04-20T11:24:35.143 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-04-20T11:24:35.145 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-04-20T11:24:35.146 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-04-20T11:24:35.148 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-04-20T11:24:35.150 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/ 2024-04-20T11:24:35.152 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-04-20T11:24:35.153 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-04-20T11:24:35.155 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-04-20T11:24:35.157 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-04-20T11:24:35.158 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-04-20T11:24:35.160 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-04-20T11:24:35.162 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-04-20T11:24:35.164 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-04-20T11:24:35.166 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-04-20T11:24:35.168 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-04-20T11:24:35.170 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-04-20T11:24:35.171 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/tpm/ 2024-04-20T11:24:35.173 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-04-20T11:24:35.175 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/ 2024-04-20T11:24:35.176 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-04-20T11:24:35.178 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-04-20T11:24:35.180 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-04-20T11:24:35.182 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-04-20T11:24:35.183 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-04-20T11:24:35.185 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-04-20T11:24:35.187 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-04-20T11:24:35.191 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-04-20T11:24:35.193 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-04-20T11:24:35.195 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-04-20T11:24:35.197 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-04-20T11:24:35.198 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-04-20T11:24:35.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-04-20T11:24:35.202 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-04-20T11:24:35.204 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-04-20T11:24:35.206 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-04-20T11:24:35.209 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-04-20T11:24:35.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-04-20T11:24:35.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/unicode.txt 2024-04-20T11:24:35.214 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/unshare.txt 2024-04-20T11:24:35.217 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/ 2024-04-20T11:24:35.218 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-04-20T11:24:35.220 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-04-20T11:24:35.222 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-04-20T11:24:35.225 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-04-20T11:24:35.226 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-04-20T11:24:35.228 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-04-20T11:24:35.230 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-04-20T11:24:35.233 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-04-20T11:24:35.299 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-04-20T11:24:35.300 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-04-20T11:24:35.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-04-20T11:24:35.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-04-20T11:24:35.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-04-20T11:24:35.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vDSO/ 2024-04-20T11:24:35.301 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-04-20T11:24:35.302 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-04-20T11:24:35.304 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-04-20T11:24:35.307 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-04-20T11:24:35.309 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-04-20T11:24:35.310 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vfio.txt 2024-04-20T11:24:35.313 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-04-20T11:24:35.315 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video-output.txt 2024-04-20T11:24:35.316 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/ 2024-04-20T11:24:35.318 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-04-20T11:24:35.320 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-04-20T11:24:35.321 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-04-20T11:24:35.323 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-04-20T11:24:35.325 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-04-20T11:24:35.326 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-04-20T11:24:35.328 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-04-20T11:24:35.330 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-04-20T11:24:35.332 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-04-20T11:24:35.333 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-04-20T11:24:35.335 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-04-20T11:24:35.337 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-04-20T11:24:35.339 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-04-20T11:24:35.340 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-04-20T11:24:35.342 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-04-20T11:24:35.344 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-04-20T11:24:35.345 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-04-20T11:24:35.347 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-04-20T11:24:35.349 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-04-20T11:24:35.352 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-04-20T11:24:35.354 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-04-20T11:24:35.356 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-04-20T11:24:35.359 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-04-20T11:24:35.360 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-04-20T11:24:35.362 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-04-20T11:24:35.365 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-04-20T11:24:35.367 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-04-20T11:24:35.369 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-04-20T11:24:35.371 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-04-20T11:24:35.374 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-04-20T11:24:35.376 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-04-20T11:24:35.378 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-04-20T11:24:35.381 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-04-20T11:24:35.383 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-04-20T11:24:35.385 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-04-20T11:24:35.387 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-04-20T11:24:35.390 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-04-20T11:24:35.392 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-04-20T11:24:35.394 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-04-20T11:24:35.396 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-04-20T11:24:35.398 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-04-20T11:24:35.401 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-04-20T11:24:35.403 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-04-20T11:24:35.405 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-04-20T11:24:35.407 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-04-20T11:24:35.409 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-04-20T11:24:35.411 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-04-20T11:24:35.414 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-04-20T11:24:35.416 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-04-20T11:24:35.418 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-04-20T11:24:35.421 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-04-20T11:24:35.423 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-04-20T11:24:35.425 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-04-20T11:24:35.428 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-04-20T11:24:35.430 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-04-20T11:24:35.432 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-04-20T11:24:35.435 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-04-20T11:24:35.437 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-04-20T11:24:35.439 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-04-20T11:24:35.441 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-04-20T11:24:35.442 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-04-20T11:24:35.444 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-04-20T11:24:35.446 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-04-20T11:24:35.448 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-04-20T11:24:35.449 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-04-20T11:24:35.451 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-04-20T11:24:35.453 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-04-20T11:24:35.454 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-04-20T11:24:35.456 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-04-20T11:24:35.458 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-04-20T11:24:35.460 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-04-20T11:24:35.461 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-04-20T11:24:35.463 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-04-20T11:24:35.465 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-04-20T11:24:35.467 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-04-20T11:24:35.469 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-04-20T11:24:35.470 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-04-20T11:24:35.472 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-04-20T11:24:35.474 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/ 2024-04-20T11:24:35.476 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-04-20T11:24:35.477 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-04-20T11:24:35.479 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-04-20T11:24:35.480 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-04-20T11:24:35.484 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-04-20T11:24:35.485 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-04-20T11:24:35.487 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-04-20T11:24:35.488 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-04-20T11:24:35.490 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-04-20T11:24:35.492 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-04-20T11:24:35.494 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-04-20T11:24:35.495 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-04-20T11:24:35.497 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-04-20T11:24:35.499 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-04-20T11:24:35.500 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-04-20T11:24:35.503 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-04-20T11:24:35.525 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-04-20T11:24:35.535 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-04-20T11:24:35.546 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-04-20T11:24:35.557 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-04-20T11:24:35.568 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-04-20T11:24:35.570 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-04-20T11:24:35.573 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-04-20T11:24:35.575 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-04-20T11:24:35.577 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-04-20T11:24:35.582 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/ 2024-04-20T11:24:35.584 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-04-20T11:24:35.586 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-04-20T11:24:35.589 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-04-20T11:24:35.591 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/balance 2024-04-20T11:24:35.593 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-04-20T11:24:35.595 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-04-20T11:24:35.598 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-04-20T11:24:35.600 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-04-20T11:24:35.603 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-04-20T11:24:35.605 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-04-20T11:24:35.607 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/numa 2024-04-20T11:24:35.609 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-04-20T11:24:35.611 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-04-20T11:24:35.613 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-04-20T11:24:35.615 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-04-20T11:24:35.617 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-04-20T11:24:35.620 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-04-20T11:24:35.622 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-04-20T11:24:35.624 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-04-20T11:24:35.626 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-04-20T11:24:35.628 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-04-20T11:24:35.630 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-04-20T11:24:35.632 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-04-20T11:24:35.634 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-04-20T11:24:35.636 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-04-20T11:24:35.638 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/ 2024-04-20T11:24:35.640 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-04-20T11:24:35.642 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-04-20T11:24:35.643 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-04-20T11:24:35.645 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-04-20T11:24:35.646 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-04-20T11:24:35.648 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-04-20T11:24:35.650 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-04-20T11:24:35.652 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-04-20T11:24:35.653 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-04-20T11:24:35.655 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-04-20T11:24:35.657 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-04-20T11:24:35.658 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-04-20T11:24:35.660 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-04-20T11:24:35.662 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-04-20T11:24:35.664 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-04-20T11:24:35.665 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-04-20T11:24:35.667 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/ 2024-04-20T11:24:35.669 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-04-20T11:24:35.671 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-04-20T11:24:35.673 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-04-20T11:24:35.674 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-04-20T11:24:35.676 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-04-20T11:24:35.678 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-04-20T11:24:35.679 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-04-20T11:24:35.681 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-04-20T11:24:35.683 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-04-20T11:24:35.684 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-04-20T11:24:35.686 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-04-20T11:24:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:35 smithi078 ceph-mon[21464]: pgmap v62: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-04-20T11:24:35.688 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-04-20T11:24:35.690 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-04-20T11:24:35.692 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/wimax/ 2024-04-20T11:24:35.693 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-04-20T11:24:35.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:35 smithi044 ceph-mon[16855]: pgmap v62: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-04-20T11:24:35.698 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-04-20T11:24:35.700 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-04-20T11:24:35.702 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/ 2024-04-20T11:24:35.703 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-04-20T11:24:35.705 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-04-20T11:24:35.707 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-04-20T11:24:35.709 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-04-20T11:24:35.710 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-04-20T11:24:35.712 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-04-20T11:24:35.714 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-04-20T11:24:35.717 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-04-20T11:24:35.717 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-04-20T11:24:35.719 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-04-20T11:24:35.721 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-04-20T11:24:35.723 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-04-20T11:24:35.724 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-04-20T11:24:35.726 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-04-20T11:24:35.728 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-04-20T11:24:35.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:35 smithi165 ceph-mon[21975]: pgmap v62: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-04-20T11:24:35.729 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-04-20T11:24:35.731 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-04-20T11:24:35.733 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-04-20T11:24:35.735 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-04-20T11:24:35.736 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-04-20T11:24:35.738 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-04-20T11:24:35.740 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-04-20T11:24:35.741 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-04-20T11:24:35.744 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-04-20T11:24:35.746 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/xtensa/ 2024-04-20T11:24:35.748 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-04-20T11:24:35.750 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-04-20T11:24:35.752 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/xz.txt 2024-04-20T11:24:35.754 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-04-20T11:24:35.755 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-04-20T11:24:35.757 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-04-20T11:24:35.759 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-04-20T11:24:35.761 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-04-20T11:24:35.762 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-04-20T11:24:35.764 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-04-20T11:24:35.766 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-04-20T11:24:35.768 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-04-20T11:24:35.769 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-04-20T11:24:35.771 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-04-20T11:24:35.773 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-04-20T11:24:35.775 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-04-20T11:24:35.776 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-04-20T11:24:35.778 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-04-20T11:24:35.780 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-04-20T11:24:35.782 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-04-20T11:24:35.783 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-04-20T11:24:35.785 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-04-20T11:24:36.198 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-04-20T11:24:36.200 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-04-20T11:24:36.202 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-04-20T11:24:36.203 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-04-20T11:24:36.205 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-04-20T11:24:36.207 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-04-20T11:24:36.209 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-04-20T11:24:36.210 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-04-20T11:24:36.212 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-04-20T11:24:36.214 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-04-20T11:24:36.216 INFO:tasks.workunit.client.0.smithi044.stdout:linux-4.0.5/Documentation/zorro.txt 2024-04-20T11:24:37.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:37 smithi078 ceph-mon[21464]: pgmap v63: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-04-20T11:24:37.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:37 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:37 smithi044 ceph-mon[16855]: pgmap v63: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-04-20T11:24:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:37 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:37 smithi165 ceph-mon[21975]: pgmap v63: 97 pgs: 97 active+clean; 86 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 7.1 MiB/s wr, 32 op/s 2024-04-20T11:24:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:37 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:39.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:39 smithi078 ceph-mon[21464]: pgmap v64: 97 pgs: 97 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.7 MiB/s wr, 68 op/s 2024-04-20T11:24:39.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:39 smithi044 ceph-mon[16855]: pgmap v64: 97 pgs: 97 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.7 MiB/s wr, 68 op/s 2024-04-20T11:24:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:39 smithi165 ceph-mon[21975]: pgmap v64: 97 pgs: 97 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.7 MiB/s wr, 68 op/s 2024-04-20T11:24:40.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:40 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:40.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:40 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:40.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:40 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:41 smithi078 ceph-mon[21464]: pgmap v65: 97 pgs: 97 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 59 op/s 2024-04-20T11:24:41.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:41 smithi044 ceph-mon[16855]: pgmap v65: 97 pgs: 97 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 59 op/s 2024-04-20T11:24:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:41 smithi165 ceph-mon[21975]: pgmap v65: 97 pgs: 97 active+clean; 93 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 59 op/s 2024-04-20T11:24:41.869 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar cf doc.tar linux-4.0.5 2024-04-20T11:24:43.607 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar xf doc.tar 2024-04-20T11:24:43.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:43 smithi078 ceph-mon[21464]: pgmap v66: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2024-04-20T11:24:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:43 smithi044 ceph-mon[16855]: pgmap v66: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2024-04-20T11:24:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:43 smithi165 ceph-mon[21975]: pgmap v66: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2024-04-20T11:24:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:45 smithi044 ceph-mon[16855]: pgmap v67: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 729 KiB/s wr, 51 op/s 2024-04-20T11:24:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:45 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:45 smithi165 ceph-mon[21975]: pgmap v67: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 729 KiB/s wr, 51 op/s 2024-04-20T11:24:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:45 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:45.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:45 smithi078 ceph-mon[21464]: pgmap v67: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 729 KiB/s wr, 51 op/s 2024-04-20T11:24:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:45 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:24:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:47 smithi044 ceph-mon[16855]: pgmap v68: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 729 KiB/s wr, 51 op/s 2024-04-20T11:24:47.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:47 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:24:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:47 smithi165 ceph-mon[21975]: pgmap v68: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 729 KiB/s wr, 51 op/s 2024-04-20T11:24:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:47 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:24:47.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:47 smithi078 ceph-mon[21464]: pgmap v68: 97 pgs: 97 active+clean; 94 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 729 KiB/s wr, 51 op/s 2024-04-20T11:24:47.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:47 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:24:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:48 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:24:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:48 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:24:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:48 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:24:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:48 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:24:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:48 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:24:48.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:48 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:24:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:48 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:24:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:48 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:24:48.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:48 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:24:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:49 smithi165 ceph-mon[21975]: pgmap v69: 97 pgs: 97 active+clean; 99 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-20T11:24:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:49 smithi078 ceph-mon[21464]: pgmap v69: 97 pgs: 97 active+clean; 99 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-20T11:24:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:49 smithi044 ceph-mon[16855]: pgmap v69: 97 pgs: 97 active+clean; 99 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-04-20T11:24:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:51 smithi165 ceph-mon[21975]: pgmap v70: 97 pgs: 97 active+clean; 99 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 567 KiB/s wr, 42 op/s 2024-04-20T11:24:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:51 smithi078 ceph-mon[21464]: pgmap v70: 97 pgs: 97 active+clean; 99 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 567 KiB/s wr, 42 op/s 2024-04-20T11:24:51.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:51 smithi044 ceph-mon[16855]: pgmap v70: 97 pgs: 97 active+clean; 99 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 567 KiB/s wr, 42 op/s 2024-04-20T11:24:53.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:53 smithi078 ceph-mon[21464]: pgmap v71: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-04-20T11:24:53.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:53 smithi044 ceph-mon[16855]: pgmap v71: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-04-20T11:24:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:53 smithi165 ceph-mon[21975]: pgmap v71: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-04-20T11:24:55.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:55 smithi078 ceph-mon[21464]: pgmap v72: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 43 op/s 2024-04-20T11:24:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:55 smithi044 ceph-mon[16855]: pgmap v72: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 43 op/s 2024-04-20T11:24:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:55 smithi165 ceph-mon[21975]: pgmap v72: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 43 op/s 2024-04-20T11:24:57.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:24:57.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:24:57.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.454+0000 7f4231a00640 1 -- 172.21.15.44:0/2038492859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c10d3d0 msgr2=0x7f422c10d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.454+0000 7f4231a00640 1 --2- 172.21.15.44:0/2038492859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c10d3d0 0x7f422c10d7b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f421c009900 tx=0x7f421c02bcd0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.454+0000 7f4231a00640 1 -- 172.21.15.44:0/2038492859 shutdown_connections 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.454+0000 7f4231a00640 1 --2- 172.21.15.44:0/2038492859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c108ea0 0x7f422c1152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.455+0000 7f4231a00640 1 --2- 172.21.15.44:0/2038492859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.455+0000 7f4231a00640 1 --2- 172.21.15.44:0/2038492859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c10d3d0 0x7f422c10d7b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.455+0000 7f4231a00640 1 -- 172.21.15.44:0/2038492859 >> 172.21.15.44:0/2038492859 conn(0x7f422c077d80 msgr2=0x7f422c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.455+0000 7f4231a00640 1 -- 172.21.15.44:0/2038492859 shutdown_connections 2024-04-20T11:24:57.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.455+0000 7f4231a00640 1 -- 172.21.15.44:0/2038492859 wait complete. 2024-04-20T11:24:57.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 Processor -- start 2024-04-20T11:24:57.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 -- start start 2024-04-20T11:24:57.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c108ea0 0x7f422c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c10d3d0 0x7f422c1a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f422c118930 con 0x7f422c108ea0 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1187b0 con 0x7f422c108500 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4231a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f422c118ab0 con 0x7f422c10d3d0 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f422be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f422be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60172/0 (socket says 172.21.15.44:60172) 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f422be00640 1 -- 172.21.15.44:0/440901791 learned_addr learned my addr 172.21.15.44:0/440901791 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.456+0000 7f4223e00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c108ea0 0x7f422c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4231000640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c10d3d0 0x7f422c1a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f422be00640 1 -- 172.21.15.44:0/440901791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c10d3d0 msgr2=0x7f422c1a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f422be00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c10d3d0 0x7f422c1a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f422be00640 1 -- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c108ea0 msgr2=0x7f422c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f422be00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c108ea0 0x7f422c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f422be00640 1 -- 172.21.15.44:0/440901791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c1a3e40 con 0x7f422c108500 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4223e00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c108ea0 0x7f422c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4231000640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c10d3d0 0x7f422c1a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f422be00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c19ee50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f421c005940 tx=0x7f421c0049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4229600640 1 -- 172.21.15.44:0/440901791 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c043020 con 0x7f422c108500 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4229600640 1 -- 172.21.15.44:0/440901791 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f421c004250 con 0x7f422c108500 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c1a40d0 con 0x7f422c108500 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.457+0000 7f4229600640 1 -- 172.21.15.44:0/440901791 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c003a80 con 0x7f422c108500 2024-04-20T11:24:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.458+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c1ab9c0 con 0x7f422c108500 2024-04-20T11:24:57.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.461+0000 7f4229600640 1 -- 172.21.15.44:0/440901791 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f421c046020 con 0x7f422c108500 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.461+0000 7f4235d78740 1 Processor -- start 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.461+0000 7f4235d78740 1 -- start start 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.461+0000 7f4235d78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 0x560074943760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.461+0000 7f4235d78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943ca0 0x56007494c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4235d78740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x56007494ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4235d78740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5600748a9150 con 0x56007494c880 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4235d78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56007484f370 con 0x560074943380 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4235d78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5600748a5110 con 0x560074943ca0 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 0x560074943760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 0x560074943760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60176/0 (socket says 172.21.15.44:60176) 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 -- 172.21.15.44:0/139231494 learned_addr learned my addr 172.21.15.44:0/139231494 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:57.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f422be00640 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943ca0 0x56007494c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4223e00640 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x56007494ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 -- 172.21.15.44:0/139231494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943ca0 msgr2=0x56007494c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943ca0 0x56007494c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 -- 172.21.15.44:0/139231494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 msgr2=0x56007494ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x56007494ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4231000640 1 -- 172.21.15.44:0/139231494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600746857d0 con 0x560074943380 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.462+0000 7f4223e00640 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x56007494ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f4231000640 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 0x560074943760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4224009f20 tx=0x7f4224010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b062656f5826e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f4229600640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42100a3da0 0x7f42100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f4221600640 1 -- 172.21.15.44:0/139231494 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224016070 con 0x560074943380 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f4221600640 1 -- 172.21.15.44:0/139231494 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f422401bb60 con 0x560074943380 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f4221600640 1 -- 172.21.15.44:0/139231494 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224012d70 con 0x560074943380 2024-04-20T11:24:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f4229600640 1 -- 172.21.15.44:0/440901791 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f421c0f2910 con 0x7f422c108500 2024-04-20T11:24:57.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.463+0000 7f422be00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42100a3da0 0x7f42100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 -- 172.21.15.44:0/139231494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 msgr2=0x560074943760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 0x560074943760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4224009f20 tx=0x7f4224010800 comp rx=0 tx=0).stop 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 -- 172.21.15.44:0/139231494 shutdown_connections 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x56007494ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943ca0 0x56007494c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 --2- 172.21.15.44:0/139231494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943380 0x560074943760 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 -- 172.21.15.44:0/139231494 >> 172.21.15.44:0/139231494 conn(0x56007493dab0 msgr2=0x56007493de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f422be00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42100a3da0 0x7f42100a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x560074942e50 tx=0x7f421c0f8530 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 -- 172.21.15.44:0/139231494 shutdown_connections 2024-04-20T11:24:57.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.464+0000 7f4235d78740 1 -- 172.21.15.44:0/139231494 wait complete. 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 -- start start 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 Processor -- start 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074949900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x560074949e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x56007494a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56007494f670 con 0x560074943ca0 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560074862740 con 0x56007494c880 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56007494f7f0 con 0x560074943380 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.465+0000 7f4235d78740 10 client.0 _ll_register_callbacks cb 0x56007485db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x560074949e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x560074949e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39942/0 (socket says 172.21.15.44:39942) 2024-04-20T11:24:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 -- 172.21.15.44:0/2422276760 learned_addr learned my addr 172.21.15.44:0/2422276760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4231000640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x56007494a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4231000640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074949900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 -- 172.21.15.44:0/2422276760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 msgr2=0x560074949900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074949900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 -- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 msgr2=0x56007494a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x56007494a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4223e00640 1 -- 172.21.15.44:0/2422276760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600745bb3f0 con 0x560074943ca0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.466+0000 7f4231000640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074949900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f4223e00640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x560074949e40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4214013790 tx=0x7f4214013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621c77e7ed97cc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f421a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f421a000640 1 -- 172.21.15.44:0/2422276760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214020060 con 0x560074943ca0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f421a000640 1 -- 172.21.15.44:0/2422276760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4214011dd0 con 0x560074943ca0 2024-04-20T11:24:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422401bb60 con 0x560074943ca0 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f421a000640 1 -- 172.21.15.44:0/2422276760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f421400f450 con 0x560074943ca0 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f4235d78740 10 client.15855 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.467+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56007457b0f0 con 0x560074943ca0 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f421a000640 1 -- 172.21.15.44:0/2422276760 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f422401bb60 con 0x560074943ca0 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f421a000640 1 -- 172.21.15.44:0/2422276760 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f4214023020 con 0x560074943ca0 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4235d78740 10 client.15855 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4235d78740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4235d78740 20 client.15855 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4235d78740 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56007495cdb0 0x56007495f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4235d78740 4 client.15855 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600748b6b70 con 0x56007495cdb0 2024-04-20T11:24:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.468+0000 7f4231000640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56007495cdb0 0x56007495f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.469+0000 7f4231000640 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56007495cdb0 0x56007495f1d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eae8b078f9a5df9 server_cookie=1fc8c7c8ee45fd5b in_seq=0 out_seq=0 2024-04-20T11:24:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.469+0000 7f421a000640 10 client.15855 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f421a000640 1 -- 172.21.15.44:0/2422276760 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600748b6b70 con 0x56007495cdb0 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f421a000640 10 client.15855 handle_command_reply: tid=0 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 client.15855 shutdown 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 msgr2=0x560074949e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x560074949e40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4214013790 tx=0x7f4214013c50 comp rx=0 tx=0).stop 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 shutdown_connections 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56007495cdb0 0x56007495f1d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x56007494a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x560074949e40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 --2- 172.21.15.44:0/2422276760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074949900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.471+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 >> 172.21.15.44:0/2422276760 conn(0x56007493dab0 msgr2=0x56007493f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.472+0000 7f421a000640 0 client.15855 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.472+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 shutdown_connections 2024-04-20T11:24:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.472+0000 7f4235d78740 1 -- 172.21.15.44:0/2422276760 wait complete. 2024-04-20T11:24:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.472+0000 7f4235d78740 20 client.15855 trim_cache size 0 max 16384 2024-04-20T11:24:57.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.493+0000 7f4235d78740 1 Processor -- start 2024-04-20T11:24:57.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 -- start start 2024-04-20T11:24:57.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074948840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943ca0 0x560074944ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x560074958560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b4120 con 0x56007494c880 2024-04-20T11:24:57.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b5d20 con 0x560074943ca0 2024-04-20T11:24:57.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4235d78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b47f0 con 0x560074943380 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4231000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074948840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4231000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074948840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:56636/0 (socket says 172.21.15.44:56636) 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4231000640 1 -- 172.21.15.44:0/898776044 learned_addr learned my addr 172.21.15.44:0/898776044 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4223e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943ca0 0x560074944ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.494+0000 7f4231000640 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x560074958560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4231000640 1 -- 172.21.15.44:0/898776044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943ca0 msgr2=0x560074944ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4231000640 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943ca0 0x560074944ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4231000640 1 -- 172.21.15.44:0/898776044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 msgr2=0x560074958560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4231000640 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x560074958560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4231000640 1 -- 172.21.15.44:0/898776044 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600749426a0 con 0x560074943380 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4223e00640 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943ca0 0x560074944ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:57.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.495+0000 7f4231000640 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074948840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4224012a60 tx=0x7f4224012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcade6a3074de970 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.498+0000 7f4220c00640 1 -- 172.21.15.44:0/898776044 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224016040 con 0x560074943380 2024-04-20T11:24:57.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.498+0000 7f4220c00640 1 -- 172.21.15.44:0/898776044 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f422402b350 con 0x560074943380 2024-04-20T11:24:57.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.498+0000 7f4220c00640 1 -- 172.21.15.44:0/898776044 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422401b660 con 0x560074943380 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 -- 172.21.15.44:0/898776044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 msgr2=0x560074948840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074948840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4224012a60 tx=0x7f4224012990 comp rx=0 tx=0).stop 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 -- 172.21.15.44:0/898776044 shutdown_connections 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56007494c880 0x560074958560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560074943ca0 0x560074944ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 --2- 172.21.15.44:0/898776044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074948840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 -- 172.21.15.44:0/898776044 >> 172.21.15.44:0/898776044 conn(0x56007493dab0 msgr2=0x56007493f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.499+0000 7f4235d78740 1 -- 172.21.15.44:0/898776044 shutdown_connections 2024-04-20T11:24:57.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.500+0000 7f4235d78740 1 -- 172.21.15.44:0/898776044 wait complete. 2024-04-20T11:24:57.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.500+0000 7f4235d78740 1 -- start start 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.500+0000 7f4235d78740 1 Processor -- start 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074954fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x5600749463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x560074946b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b5590 con 0x560074943ca0 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b4ec0 con 0x56007494c880 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b2080 con 0x560074943380 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4235d78740 10 client.0 _ll_register_callbacks cb 0x7f4214009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4231000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x560074946b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4231000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x560074946b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60194/0 (socket says 172.21.15.44:60194) 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4231000640 1 -- 172.21.15.44:0/1662507601 learned_addr learned my addr 172.21.15.44:0/1662507601 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4223e00640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x5600749463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.501+0000 7f4231000640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074954fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4231000640 1 -- 172.21.15.44:0/1662507601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 msgr2=0x560074954fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4231000640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074954fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4231000640 1 -- 172.21.15.44:0/1662507601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 msgr2=0x5600749463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4231000640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x5600749463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4231000640 1 -- 172.21.15.44:0/1662507601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600749c2350 con 0x56007494c880 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4223e00640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x5600749463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4231000640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x560074946b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4224010d80 tx=0x7f422401b500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c772b3fdd9461cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f421a000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f421a000640 1 -- 172.21.15.44:0/1662507601 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224016070 con 0x56007494c880 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f421a000640 1 -- 172.21.15.44:0/1662507601 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f422402b700 con 0x56007494c880 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422402b350 con 0x56007494c880 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.502+0000 7f421a000640 1 -- 172.21.15.44:0/1662507601 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4224041050 con 0x56007494c880 2024-04-20T11:24:57.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 10 client.25982 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5600749c3480 con 0x56007494c880 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f421a000640 1 -- 172.21.15.44:0/1662507601 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f422402b350 con 0x56007494c880 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f421a000640 1 -- 172.21.15.44:0/1662507601 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f422400f040 con 0x56007494c880 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 10 client.25982 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 20 client.25982 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600749c3900 0x5600749c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 4 client.25982 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600748b09e0 con 0x5600749c3900 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.503+0000 7f4223e00640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600749c3900 0x5600749c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.504+0000 7f4223e00640 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600749c3900 0x5600749c5d20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d65aaba05fb5f7 server_cookie=107b6d9d6cc59a96 in_seq=0 out_seq=0 2024-04-20T11:24:57.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.504+0000 7f421a000640 10 client.25982 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:57.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.505+0000 7f421a000640 1 -- 172.21.15.44:0/1662507601 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600748b09e0 con 0x5600749c3900 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.505+0000 7f421a000640 10 client.25982 handle_command_reply: tid=0 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.505+0000 7f4235d78740 1 client.25982 shutdown 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 msgr2=0x560074946b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x560074946b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4224010d80 tx=0x7f422401b500 comp rx=0 tx=0).stop 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 shutdown_connections 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600749c3900 0x5600749c5d20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56007494c880 0x560074946b10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560074943ca0 0x5600749463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f421a000640 0 client.25982 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 --2- 172.21.15.44:0/1662507601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560074943380 0x560074954fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 >> 172.21.15.44:0/1662507601 conn(0x56007493dab0 msgr2=0x56007493c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 shutdown_connections 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 1 -- 172.21.15.44:0/1662507601 wait complete. 2024-04-20T11:24:57.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.506+0000 7f4235d78740 20 client.25982 trim_cache size 0 max 16384 2024-04-20T11:24:57.509 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:24:57.509 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:24:57.509 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:24:57.509 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42100a3da0 msgr2=0x7f42100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42100a3da0 0x7f42100a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x560074942e50 tx=0x7f421c0f8530 comp rx=0 tx=0).stop 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 msgr2=0x7f422c19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c19ee50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f421c005940 tx=0x7f421c0049e0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 shutdown_connections 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422c10d3d0 0x7f422c1a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42100a3da0 0x7f42100a6260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f422c108ea0 0x7f422c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 --2- 172.21.15.44:0/440901791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f422c108500 0x7f422c19ee50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 >> 172.21.15.44:0/440901791 conn(0x7f422c077d80 msgr2=0x7f422c078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 shutdown_connections 2024-04-20T11:24:57.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.507+0000 7f4231a00640 1 -- 172.21.15.44:0/440901791 wait complete. 2024-04-20T11:24:57.533 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:24:57.533 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:24:57.635 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:57 smithi044 ceph-mon[16855]: pgmap v73: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 43 op/s 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 -- 172.21.15.44:0/3121360079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 msgr2=0x7f9cc81045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 --2- 172.21.15.44:0/3121360079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 0x7f9cc81045d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009900 tx=0x7f9cb002bc30 comp rx=0 tx=0).stop 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 -- 172.21.15.44:0/3121360079 shutdown_connections 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 --2- 172.21.15.44:0/3121360079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc81092c0 0x7f9cc8110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 --2- 172.21.15.44:0/3121360079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8104ba0 0x7f9cc8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 --2- 172.21.15.44:0/3121360079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 0x7f9cc81045d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.877+0000 7f9ccd000640 1 -- 172.21.15.44:0/3121360079 >> 172.21.15.44:0/3121360079 conn(0x7f9cc80ffa30 msgr2=0x7f9cc8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.878+0000 7f9ccd000640 1 -- 172.21.15.44:0/3121360079 shutdown_connections 2024-04-20T11:24:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.878+0000 7f9ccd000640 1 -- 172.21.15.44:0/3121360079 wait complete. 2024-04-20T11:24:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.879+0000 7f9ccd000640 1 Processor -- start 2024-04-20T11:24:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 -- start start 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 0x7f9cc81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc81092c0 0x7f9cc81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 0x7f9cc819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8114540 con 0x7f9cc81092c0 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81143c0 con 0x7f9cc819a760 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.880+0000 7f9ccd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81146c0 con 0x7f9cc81041f0 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.881+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 0x7f9cc819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.881+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 0x7f9cc819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60220/0 (socket says 172.21.15.44:60220) 2024-04-20T11:24:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.881+0000 7f9cc7e00640 1 -- 172.21.15.44:0/4011179114 learned_addr learned my addr 172.21.15.44:0/4011179114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.881+0000 7f9cc7400640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 0x7f9cc81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.881+0000 7f9cc6a00640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc81092c0 0x7f9cc81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc7e00640 1 -- 172.21.15.44:0/4011179114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 msgr2=0x7f9cc81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc7e00640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 0x7f9cc81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc7e00640 1 -- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc81092c0 msgr2=0x7f9cc81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc7e00640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc81092c0 0x7f9cc81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc7e00640 1 -- 172.21.15.44:0/4011179114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc819b360 con 0x7f9cc819a760 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc6a00640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc81092c0 0x7f9cc81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.882+0000 7f9cc7e00640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 0x7f9cc819abe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000b970 tx=0x7f9cc000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.883+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0019070 con 0x7f9cc819a760 2024-04-20T11:24:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.883+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9cc000cb40 con 0x7f9cc819a760 2024-04-20T11:24:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.883+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0007700 con 0x7f9cc819a760 2024-04-20T11:24:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.884+0000 7f9ccd000640 1 -- 172.21.15.44:0/4011179114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc81a73f0 con 0x7f9cc819a760 2024-04-20T11:24:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.884+0000 7f9ccd000640 1 -- 172.21.15.44:0/4011179114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81a77a0 con 0x7f9cc819a760 2024-04-20T11:24:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.886+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc81056c0 con 0x7f9cc819a760 2024-04-20T11:24:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.894+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9cc000ccf0 con 0x7f9cc819a760 2024-04-20T11:24:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.897+0000 7f9cbfe00640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9cac0a3e20 0x7f9cac0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.897+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9cc00cfd50 con 0x7f9cc819a760 2024-04-20T11:24:57.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.898+0000 7f9cc7400640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9cac0a3e20 0x7f9cac0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:57.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.898+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9cc009bec0 con 0x7f9cc819a760 2024-04-20T11:24:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:57.904+0000 7f9cc7400640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9cac0a3e20 0x7f9cac0a62e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00096c0 tx=0x7f9cb0000bc0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:57 smithi078 ceph-mon[21464]: pgmap v73: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 43 op/s 2024-04-20T11:24:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:57 smithi165 ceph-mon[21975]: pgmap v73: 97 pgs: 97 active+clean; 106 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.1 MiB/s wr, 43 op/s 2024-04-20T11:24:58.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.383+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc819bc40 con 0x7f9cc819a760 2024-04-20T11:24:58.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.384+0000 7f9cbfe00640 1 -- 172.21.15.44:0/4011179114 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11954 (secure 0 0 0) 0x7f9cc00a01c0 con 0x7f9cc819a760 2024-04-20T11:24:58.388 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:24:58.388 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:24:43.721503+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:24:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 31 2024-04-20T11:24:58.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.388+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9cac0a3e20 msgr2=0x7f9cac0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.388+0000 7f9cbd600640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9cac0a3e20 0x7f9cac0a62e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00096c0 tx=0x7f9cb0000bc0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.388+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 msgr2=0x7f9cc819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.388+0000 7f9cbd600640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 0x7f9cc819abe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000b970 tx=0x7f9cc000be30 comp rx=0 tx=0).stop 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 shutdown_connections 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cc819a760 0x7f9cc819abe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9cac0a3e20 0x7f9cac0a62e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc81092c0 0x7f9cc81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 --2- 172.21.15.44:0/4011179114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cc81041f0 0x7f9cc81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 >> 172.21.15.44:0/4011179114 conn(0x7f9cc80ffa30 msgr2=0x7f9cc8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 shutdown_connections 2024-04-20T11:24:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.389+0000 7f9cbd600640 1 -- 172.21.15.44:0/4011179114 wait complete. 2024-04-20T11:24:58.419 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:24:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 -- 172.21.15.44:0/1592149474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c101680 msgr2=0x7f731c10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 --2- 172.21.15.44:0/1592149474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c101680 0x7f731c10e3d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7308009900 tx=0x7f730802bc40 comp rx=0 tx=0).stop 2024-04-20T11:24:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 -- 172.21.15.44:0/1592149474 shutdown_connections 2024-04-20T11:24:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 --2- 172.21.15.44:0/1592149474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c10e910 0x7f731c110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 --2- 172.21.15.44:0/1592149474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c101680 0x7f731c10e3d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 --2- 172.21.15.44:0/1592149474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f731c100d60 0x7f731c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.668+0000 7f7324600640 1 -- 172.21.15.44:0/1592149474 >> 172.21.15.44:0/1592149474 conn(0x7f731c077f60 msgr2=0x7f731c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.669+0000 7f7324600640 1 -- 172.21.15.44:0/1592149474 shutdown_connections 2024-04-20T11:24:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.669+0000 7f7324600640 1 -- 172.21.15.44:0/1592149474 wait complete. 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 Processor -- start 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 -- start start 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 0x7f731c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c101680 0x7f731c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f731c10e910 0x7f731c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f731c114660 con 0x7f731c101680 2024-04-20T11:24:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1144e0 con 0x7f731c10e910 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7324600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1147e0 con 0x7f731c100d60 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7321c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 0x7f731c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7321c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 0x7f731c1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:39956/0 (socket says 172.21.15.44:39956) 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7321c00640 1 -- 172.21.15.44:0/37810127 learned_addr learned my addr 172.21.15.44:0/37810127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.670+0000 7f7321200640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c101680 0x7f731c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7322600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f731c10e910 0x7f731c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321c00640 1 -- 172.21.15.44:0/37810127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f731c10e910 msgr2=0x7f731c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321c00640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f731c10e910 0x7f731c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321c00640 1 -- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c101680 msgr2=0x7f731c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321c00640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c101680 0x7f731c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321c00640 1 -- 172.21.15.44:0/37810127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c19b0a0 con 0x7f731c100d60 2024-04-20T11:24:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321200640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c101680 0x7f731c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7321c00640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 0x7f731c1a0770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9f0 tx=0x7f731000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f731a000640 1 -- 172.21.15.44:0/37810127 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731000cd80 con 0x7f731c100d60 2024-04-20T11:24:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f731a000640 1 -- 172.21.15.44:0/37810127 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7310005520 con 0x7f731c100d60 2024-04-20T11:24:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f731a000640 1 -- 172.21.15.44:0/37810127 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310010690 con 0x7f731c100d60 2024-04-20T11:24:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c19b390 con 0x7f731c100d60 2024-04-20T11:24:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.671+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c1a7630 con 0x7f731c100d60 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.674+0000 7f7327418740 1 Processor -- start 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 -- start start 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f731a000640 1 -- 172.21.15.44:0/37810127 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f73100046c0 con 0x7f731c100d60 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889180390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891808d0 0x560889188f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 0x56088918b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5608890e5050 con 0x5608891808d0 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56088908b490 con 0x5608891894b0 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7327418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5608890e1010 con 0x56088917ffb0 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7321200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 0x56088918b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7322600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889180390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7321200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 0x56088918b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39512/0 (socket says 172.21.15.44:39512) 2024-04-20T11:24:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7321200640 1 -- 172.21.15.44:0/3500193756 learned_addr learned my addr 172.21.15.44:0/3500193756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.675+0000 7f7321c00640 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891808d0 0x560889188f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f7321200640 1 -- 172.21.15.44:0/3500193756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 msgr2=0x560889180390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f7321200640 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889180390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f7321200640 1 -- 172.21.15.44:0/3500193756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891808d0 msgr2=0x560889188f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f7321200640 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891808d0 0x560889188f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f7321200640 1 -- 172.21.15.44:0/3500193756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560889093070 con 0x5608891894b0 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f7321200640 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 0x56088918b8a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f730802c180 tx=0x7f7308005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d9b509969252f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f730ea00640 1 -- 172.21.15.44:0/3500193756 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730802c9e0 con 0x5608891894b0 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.676+0000 7f730ea00640 1 -- 172.21.15.44:0/3500193756 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f730802cb80 con 0x5608891894b0 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f730ea00640 1 -- 172.21.15.44:0/3500193756 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004920 con 0x5608891894b0 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f731a000640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73040a3da0 0x7f73040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7322600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73040a3da0 0x7f73040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 -- 172.21.15.44:0/3500193756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 msgr2=0x56088918b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 0x56088918b8a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f730802c180 tx=0x7f7308005e50 comp rx=0 tx=0).stop 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 -- 172.21.15.44:0/3500193756 shutdown_connections 2024-04-20T11:24:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891894b0 0x56088918b8a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891808d0 0x560889188f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 --2- 172.21.15.44:0/3500193756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889180390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f7327418740 1 -- 172.21.15.44:0/3500193756 >> 172.21.15.44:0/3500193756 conn(0x56088917a6e0 msgr2=0x56088917aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.677+0000 7f731a000640 1 -- 172.21.15.44:0/37810127 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f73100d0c30 con 0x7f731c100d60 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.678+0000 7f7327418740 1 -- 172.21.15.44:0/3500193756 shutdown_connections 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.678+0000 7f7327418740 1 -- 172.21.15.44:0/3500193756 wait complete. 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.678+0000 7f7327418740 1 -- start start 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.678+0000 7f7322600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73040a3da0 0x7f73040a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f731c06b6f0 tx=0x7f7314009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.678+0000 7f7327418740 1 Processor -- start 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x5608891864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x5608891869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x560889186f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56088918c240 con 0x5608891894b0 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56088909e860 con 0x56088917ffb0 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56088918c3c0 con 0x5608891808d0 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7327418740 10 client.0 _ll_register_callbacks cb 0x56088906cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x5608891864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x5608891864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39526/0 (socket says 172.21.15.44:39526) 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 -- 172.21.15.44:0/887648806 learned_addr learned my addr 172.21.15.44:0/887648806 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7322600640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x560889186f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321200640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x5608891869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 -- 172.21.15.44:0/887648806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 msgr2=0x5608891869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x5608891869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 -- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 msgr2=0x560889186f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x560889186f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.679+0000 7f7321c00640 1 -- 172.21.15.44:0/887648806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608890a2950 con 0x56088917ffb0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7321c00640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x5608891864b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9c0 tx=0x7f7310098f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5927a15f0d353cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7303400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7303400640 1 -- 172.21.15.44:0/887648806 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73100ce810 con 0x56088917ffb0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7303400640 1 -- 172.21.15.44:0/887648806 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f73100a53c0 con 0x56088917ffb0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f730802cb80 con 0x56088917ffb0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7303400640 1 -- 172.21.15.44:0/887648806 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f731009d2e0 con 0x56088917ffb0 2024-04-20T11:24:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7327418740 10 client.25997 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.680+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560889070820 con 0x56088917ffb0 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7303400640 1 -- 172.21.15.44:0/887648806 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f730802cb80 con 0x56088917ffb0 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7303400640 1 -- 172.21.15.44:0/887648806 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f731009c030 con 0x56088917ffb0 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7327418740 10 client.25997 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7327418740 10 client.25997 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7327418740 20 client.25997 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7327418740 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891993b0 0x56088919b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7327418740 4 client.25997 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608890f2a70 con 0x5608891993b0 2024-04-20T11:24:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.681+0000 7f7321200640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891993b0 0x56088919b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.682+0000 7f7321200640 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891993b0 0x56088919b7a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c9c27198d27b5a5 server_cookie=f4cb0b9b29a57909 in_seq=0 out_seq=0 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.682+0000 7f7303400640 10 client.25997 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7303400640 1 -- 172.21.15.44:0/887648806 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608890f2a70 con 0x5608891993b0 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7303400640 10 client.25997 handle_command_reply: tid=0 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 client.25997 shutdown 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 msgr2=0x5608891864b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x5608891864b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9c0 tx=0x7f7310098f40 comp rx=0 tx=0).stop 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 shutdown_connections 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x560889186f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x5608891869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891993b0 0x56088919b7a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 --2- 172.21.15.44:0/887648806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x5608891864b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 >> 172.21.15.44:0/887648806 conn(0x56088917a6e0 msgr2=0x56088917c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7303400640 0 client.25997 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:58.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 shutdown_connections 2024-04-20T11:24:58.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 1 -- 172.21.15.44:0/887648806 wait complete. 2024-04-20T11:24:58.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.683+0000 7f7327418740 20 client.25997 trim_cache size 0 max 16384 2024-04-20T11:24:58.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4011179114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.697+0000 7f7327418740 1 Processor -- start 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.697+0000 7f7327418740 1 -- start start 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7327418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889188be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7327418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891808d0 0x560889195100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7327418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x5608891795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7327418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1afe60 con 0x5608891894b0 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7327418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1b1a60 con 0x5608891808d0 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7327418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1b0530 con 0x56088917ffb0 2024-04-20T11:24:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889188be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889188be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:39990/0 (socket says 172.21.15.44:39990) 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 -- 172.21.15.44:0/1116765831 learned_addr learned my addr 172.21.15.44:0/1116765831 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7322600640 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x5608891795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321200640 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891808d0 0x560889195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 -- 172.21.15.44:0/1116765831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891808d0 msgr2=0x560889195100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891808d0 0x560889195100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 -- 172.21.15.44:0/1116765831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 msgr2=0x5608891795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x5608891795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 -- 172.21.15.44:0/1116765831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560888df73f0 con 0x56088917ffb0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.698+0000 7f7321c00640 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889188be0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73100995b0 tx=0x7f73100bd050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c74c39a99f3319 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f730e000640 1 -- 172.21.15.44:0/1116765831 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731009cbe0 con 0x56088917ffb0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f730e000640 1 -- 172.21.15.44:0/1116765831 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f73100a5850 con 0x56088917ffb0 2024-04-20T11:24:58.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f730e000640 1 -- 172.21.15.44:0/1116765831 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310098a70 con 0x56088917ffb0 2024-04-20T11:24:58.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 -- 172.21.15.44:0/1116765831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 msgr2=0x560889188be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889188be0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73100995b0 tx=0x7f73100bd050 comp rx=0 tx=0).stop 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 -- 172.21.15.44:0/1116765831 shutdown_connections 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x5608891795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608891808d0 0x560889195100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 --2- 172.21.15.44:0/1116765831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56088917ffb0 0x560889188be0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 -- 172.21.15.44:0/1116765831 >> 172.21.15.44:0/1116765831 conn(0x56088917a6e0 msgr2=0x560889178eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.699+0000 7f7327418740 1 -- 172.21.15.44:0/1116765831 shutdown_connections 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.700+0000 7f7327418740 1 -- 172.21.15.44:0/1116765831 wait complete. 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.700+0000 7f7327418740 1 -- start start 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.700+0000 7f7327418740 1 Processor -- start 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x56088918d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x56088918d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x56088918dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1b12d0 con 0x5608891894b0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1b0c00 con 0x56088917ffb0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1addc0 con 0x5608891808d0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7327418740 10 client.0 _ll_register_callbacks cb 0x7f7310009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x56088918d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x56088918d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39544/0 (socket says 172.21.15.44:39544) 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321c00640 1 -- 172.21.15.44:0/3114711381 learned_addr learned my addr 172.21.15.44:0/3114711381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x56088918d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7322600640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x56088918dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 -- 172.21.15.44:0/3114711381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 msgr2=0x56088918d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x56088918d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 -- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 msgr2=0x56088918dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x56088918dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 -- 172.21.15.44:0/3114711381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560888ec17d0 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321c00640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x56088918d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7321200640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x56088918d570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7308000c00 tx=0x7f7308005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4f0ddd02f432c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.701+0000 7f7303400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7303400640 1 -- 172.21.15.44:0/3114711381 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730802cd00 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7303400640 1 -- 172.21.15.44:0/3114711381 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7308004040 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73100a5850 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7303400640 1 -- 172.21.15.44:0/3114711381 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7308004240 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7327418740 10 client.26011 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5608891ff9f0 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7303400640 1 -- 172.21.15.44:0/3114711381 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f73100a5850 con 0x5608891808d0 2024-04-20T11:24:58.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.702+0000 7f7303400640 1 -- 172.21.15.44:0/3114711381 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f7308052020 con 0x5608891808d0 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7327418740 10 client.26011 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7327418740 10 client.26011 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7327418740 20 client.26011 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7327418740 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891ffe90 0x560889202280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7327418740 4 client.26011 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608890ec8e0 con 0x5608891ffe90 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7321c00640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891ffe90 0x560889202280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7321c00640 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891ffe90 0x560889202280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b030842bc1bbe43 server_cookie=649fbe9dd4b9617c in_seq=0 out_seq=0 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.703+0000 7f7303400640 10 client.26011 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7303400640 1 -- 172.21.15.44:0/3114711381 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608890ec8e0 con 0x5608891ffe90 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7303400640 10 client.26011 handle_command_reply: tid=0 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 client.26011 shutdown 2024-04-20T11:24:58.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 msgr2=0x56088918d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x56088918d570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7308000c00 tx=0x7f7308005a20 comp rx=0 tx=0).stop 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 shutdown_connections 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608891894b0 0x56088918dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608891808d0 0x56088918d570 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5608891ffe90 0x560889202280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 --2- 172.21.15.44:0/3114711381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56088917ffb0 0x56088918d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7303400640 0 client.26011 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 >> 172.21.15.44:0/3114711381 conn(0x56088917a6e0 msgr2=0x56088917c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 shutdown_connections 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 1 -- 172.21.15.44:0/3114711381 wait complete. 2024-04-20T11:24:58.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.704+0000 7f7327418740 20 client.26011 trim_cache size 0 max 16384 2024-04-20T11:24:58.710 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:58.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73040a3da0 msgr2=0x7f73040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73040a3da0 0x7f73040a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f731c06b6f0 tx=0x7f7314009500 comp rx=0 tx=0).stop 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 msgr2=0x7f731c1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 0x7f731c1a0770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9f0 tx=0x7f731000edf0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 shutdown_connections 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f731c10e910 0x7f731c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73040a3da0 0x7f73040a6260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f731c101680 0x7f731c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 --2- 172.21.15.44:0/37810127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f731c100d60 0x7f731c1a0770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 >> 172.21.15.44:0/37810127 conn(0x7f731c077f60 msgr2=0x7f731c10fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 shutdown_connections 2024-04-20T11:24:58.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.705+0000 7f7324600640 1 -- 172.21.15.44:0/37810127 wait complete. 2024-04-20T11:24:58.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:24:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 -- 172.21.15.44:0/542643839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140112c80 msgr2=0x7f1140115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 --2- 172.21.15.44:0/542643839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140112c80 0x7f1140115070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1138009ca0 tx=0x7f113802c0f0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 -- 172.21.15.44:0/542643839 shutdown_connections 2024-04-20T11:24:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 --2- 172.21.15.44:0/542643839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140112c80 0x7f1140115070 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 --2- 172.21.15.44:0/542643839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140076db0 0x7f1140112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 --2- 172.21.15.44:0/542643839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140076490 0x7f1140076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.911+0000 7f1144c00640 1 -- 172.21.15.44:0/542643839 >> 172.21.15.44:0/542643839 conn(0x7f11400ffa70 msgr2=0x7f1140101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.912+0000 7f1144c00640 1 -- 172.21.15.44:0/542643839 shutdown_connections 2024-04-20T11:24:58.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.912+0000 7f1144c00640 1 -- 172.21.15.44:0/542643839 wait complete. 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.912+0000 7f1144c00640 1 Processor -- start 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 -- start start 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140076490 0x7f114019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140076db0 0x7f114019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 0x7f11401a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1140118900 con 0x7f1140076db0 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1140118780 con 0x7f1140076490 2024-04-20T11:24:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f1144c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1140118a80 con 0x7f1140112c80 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f113fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 0x7f11401a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f113f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140076490 0x7f114019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f113ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140076db0 0x7f114019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f113fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 0x7f11401a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40016/0 (socket says 172.21.15.44:40016) 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.913+0000 7f113fe00640 1 -- 172.21.15.44:0/57376753 learned_addr learned my addr 172.21.15.44:0/57376753 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113fe00640 1 -- 172.21.15.44:0/57376753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140076490 msgr2=0x7f114019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113fe00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140076490 0x7f114019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113fe00640 1 -- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140076db0 msgr2=0x7f114019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113fe00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140076db0 0x7f114019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113fe00640 1 -- 172.21.15.44:0/57376753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11401a3d70 con 0x7f1140112c80 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113ea00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140076db0 0x7f114019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f113fe00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 0x7f11401a3640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f11380098d0 tx=0x7f11380325e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.914+0000 7f1137e00640 1 -- 172.21.15.44:0/57376753 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138039050 con 0x7f1140112c80 2024-04-20T11:24:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.915+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401a4000 con 0x7f1140112c80 2024-04-20T11:24:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.915+0000 7f1137e00640 1 -- 172.21.15.44:0/57376753 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1138045070 con 0x7f1140112c80 2024-04-20T11:24:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.915+0000 7f1137e00640 1 -- 172.21.15.44:0/57376753 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138004570 con 0x7f1140112c80 2024-04-20T11:24:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.915+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401ab8f0 con 0x7f1140112c80 2024-04-20T11:24:58.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1137e00640 1 -- 172.21.15.44:0/57376753 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1138004710 con 0x7f1140112c80 2024-04-20T11:24:58.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 Processor -- start 2024-04-20T11:24:58.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 -- start start 2024-04-20T11:24:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa464550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464a90 0x5648aa46d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa46d670 0x5648aa46fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648aa3c9050 con 0x5648aa46d670 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5648aa36f490 con 0x5648aa464170 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.918+0000 7f1148f67740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648aa3c5010 con 0x5648aa464a90 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa464550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464a90 0x5648aa46d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa464550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39552/0 (socket says 172.21.15.44:39552) 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113ea00640 1 -- 172.21.15.44:0/850709028 learned_addr learned my addr 172.21.15.44:0/850709028 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113ea00640 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa46d670 0x5648aa46fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 -- 172.21.15.44:0/850709028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 msgr2=0x5648aa464550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa464550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 -- 172.21.15.44:0/850709028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa46d670 msgr2=0x5648aa46fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa46d670 0x5648aa46fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 -- 172.21.15.44:0/850709028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648aa377070 con 0x5648aa464a90 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f1137e00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f11240a3d20 0x7f11240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f1137e00640 1 -- 172.21.15.44:0/57376753 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f11380f2d90 con 0x7f1140112c80 2024-04-20T11:24:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.919+0000 7f113f400640 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464a90 0x5648aa46d130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1130009c20 tx=0x7f1130011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5c97cbfe71f6cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f113ea00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f11240a3d20 0x7f11240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f112be00640 1 -- 172.21.15.44:0/850709028 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130004050 con 0x5648aa464a90 2024-04-20T11:24:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f112be00640 1 -- 172.21.15.44:0/850709028 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1130005520 con 0x5648aa464a90 2024-04-20T11:24:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f112be00640 1 -- 172.21.15.44:0/850709028 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130004e00 con 0x5648aa464a90 2024-04-20T11:24:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f1148f67740 1 -- 172.21.15.44:0/850709028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464a90 msgr2=0x5648aa46d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f113ea00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f11240a3d20 0x7f11240a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5648aa45fa30 tx=0x7f112c005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.920+0000 7f1148f67740 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464a90 0x5648aa46d130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1130009c20 tx=0x7f1130011810 comp rx=0 tx=0).stop 2024-04-20T11:24:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 -- 172.21.15.44:0/850709028 shutdown_connections 2024-04-20T11:24:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa46d670 0x5648aa46fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464a90 0x5648aa46d130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 --2- 172.21.15.44:0/850709028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa464550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 -- 172.21.15.44:0/850709028 >> 172.21.15.44:0/850709028 conn(0x5648aa45e8a0 msgr2=0x5648aa45ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 -- 172.21.15.44:0/850709028 shutdown_connections 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 -- 172.21.15.44:0/850709028 wait complete. 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 -- start start 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.921+0000 7f1148f67740 1 Processor -- start 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa46a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa46abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa46b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648aa470460 con 0x5648aa464a90 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5648aa382860 con 0x5648aa464170 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648aa4705e0 con 0x5648aa46d670 2024-04-20T11:24:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f1148f67740 10 client.0 _ll_register_callbacks cb 0x5648aa350fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa46a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa46b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa46abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa46b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40030/0 (socket says 172.21.15.44:40030) 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 -- 172.21.15.44:0/3473960745 learned_addr learned my addr 172.21.15.44:0/3473960745 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa46a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39560/0 (socket says 172.21.15.44:39560) 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 -- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 msgr2=0x5648aa46a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa46a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 -- 172.21.15.44:0/3473960745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 msgr2=0x5648aa46abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa46abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.922+0000 7f113fe00640 1 -- 172.21.15.44:0/3473960745 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648aa386950 con 0x5648aa46d670 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f113fe00640 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa46b130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f11380bb820 tx=0x7f11380bf690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc60ab31ab8f84ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1128c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1128c00640 1 -- 172.21.15.44:0/3473960745 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11380d8060 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1128c00640 1 -- 172.21.15.44:0/3473960745 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f11380f0770 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1130005520 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1128c00640 1 -- 172.21.15.44:0/3473960745 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11380f0970 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1148f67740 10 client.26023 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5648aa354820 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1128c00640 1 -- 172.21.15.44:0/3473960745 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1130005520 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.923+0000 7f1128c00640 1 -- 172.21.15.44:0/3473960745 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f1138004b40 con 0x5648aa46d670 2024-04-20T11:24:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1148f67740 10 client.26023 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1148f67740 10 client.26023 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1148f67740 20 client.26023 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1148f67740 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa47d800 0x5648aa47fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1148f67740 4 client.26023 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648aa3d6a70 con 0x5648aa47d800 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f113f400640 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa47d800 0x5648aa47fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f113f400640 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa47d800 0x5648aa47fc80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e57d93a198ecfb9 server_cookie=fefefce1b1209707 in_seq=0 out_seq=0 2024-04-20T11:24:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.924+0000 7f1128c00640 10 client.26023 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.926+0000 7f1128c00640 1 -- 172.21.15.44:0/3473960745 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648aa3d6a70 con 0x5648aa47d800 2024-04-20T11:24:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.926+0000 7f1128c00640 10 client.26023 handle_command_reply: tid=0 2024-04-20T11:24:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.926+0000 7f1148f67740 1 client.26023 shutdown 2024-04-20T11:24:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 msgr2=0x5648aa46b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa46b130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f11380bb820 tx=0x7f11380bf690 comp rx=0 tx=0).stop 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 shutdown_connections 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa46b130 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa47d800 0x5648aa47fc80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa46abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 --2- 172.21.15.44:0/3473960745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464170 0x5648aa46a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1128c00640 0 client.26023 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 >> 172.21.15.44:0/3473960745 conn(0x5648aa45e8a0 msgr2=0x5648aa4605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 shutdown_connections 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 1 -- 172.21.15.44:0/3473960745 wait complete. 2024-04-20T11:24:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.927+0000 7f1148f67740 20 client.26023 trim_cache size 0 max 16384 2024-04-20T11:24:58.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:58 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4011179114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 Processor -- start 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 -- start start 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464170 0x5648aa46cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 0x5648aa479350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa45ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b3fe0 con 0x5648aa464170 2024-04-20T11:24:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b5be0 con 0x5648aa464a90 2024-04-20T11:24:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.943+0000 7f1148f67740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b46b0 con 0x5648aa46d670 2024-04-20T11:24:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464170 0x5648aa46cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 0x5648aa479350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 0x5648aa479350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39570/0 (socket says 172.21.15.44:39570) 2024-04-20T11:24:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 -- 172.21.15.44:0/3237142667 learned_addr learned my addr 172.21.15.44:0/3237142667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa45ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 -- 172.21.15.44:0/3237142667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 msgr2=0x5648aa45ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa45ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 -- 172.21.15.44:0/3237142667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464170 msgr2=0x5648aa46cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464170 0x5648aa46cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 -- 172.21.15.44:0/3237142667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648aa469660 con 0x5648aa464a90 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113fe00640 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa45ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113f400640 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464170 0x5648aa46cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.944+0000 7f113ea00640 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 0x5648aa479350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f112c012850 tx=0x7f112c014070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36419281bced9a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f112b400640 1 -- 172.21.15.44:0/3237142667 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0108b0 con 0x5648aa464a90 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f112b400640 1 -- 172.21.15.44:0/3237142667 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f112c005020 con 0x5648aa464a90 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f112b400640 1 -- 172.21.15.44:0/3237142667 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c004bc0 con 0x5648aa464a90 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 -- 172.21.15.44:0/3237142667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 msgr2=0x5648aa479350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 0x5648aa479350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f112c012850 tx=0x7f112c014070 comp rx=0 tx=0).stop 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 -- 172.21.15.44:0/3237142667 shutdown_connections 2024-04-20T11:24:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa46d670 0x5648aa45ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa464a90 0x5648aa479350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 --2- 172.21.15.44:0/3237142667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464170 0x5648aa46cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.945+0000 7f1148f67740 1 -- 172.21.15.44:0/3237142667 >> 172.21.15.44:0/3237142667 conn(0x5648aa45e8a0 msgr2=0x5648aa45d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.946+0000 7f1148f67740 1 -- 172.21.15.44:0/3237142667 shutdown_connections 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.946+0000 7f1148f67740 1 -- 172.21.15.44:0/3237142667 wait complete. 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.946+0000 7f1148f67740 1 -- start start 2024-04-20T11:24:58.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.946+0000 7f1148f67740 1 Processor -- start 2024-04-20T11:24:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 0x5648aa471930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa471e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa46d670 0x5648aa4e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b5450 con 0x5648aa464a90 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b4d80 con 0x5648aa46d670 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b1f60 con 0x5648aa464170 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f1148f67740 10 client.0 _ll_register_callbacks cb 0x7f11380be7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa471e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa46d670 0x5648aa4e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 0x5648aa471930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa471e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60514/0 (socket says 172.21.15.44:60514) 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa46d670 0x5648aa4e2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39572/0 (socket says 172.21.15.44:39572) 2024-04-20T11:24:58.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113fe00640 1 -- 172.21.15.44:0/1337899899 learned_addr learned my addr 172.21.15.44:0/1337899899 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 0x5648aa471930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40042/0 (socket says 172.21.15.44:40042) 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 -- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa46d670 msgr2=0x5648aa4e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa46d670 0x5648aa4e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 -- 172.21.15.44:0/1337899899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 msgr2=0x5648aa471e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa471e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.947+0000 7f113f400640 1 -- 172.21.15.44:0/1337899899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648aa0db3f0 con 0x5648aa464170 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f113f400640 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 0x5648aa471930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f113001b040 tx=0x7f1130011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=330bbc5ac2c7baf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1128c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1128c00640 1 -- 172.21.15.44:0/1337899899 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130032020 con 0x5648aa464170 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1128c00640 1 -- 172.21.15.44:0/1337899899 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f113002edd0 con 0x5648aa464170 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f112c005020 con 0x5648aa464170 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1128c00640 1 -- 172.21.15.44:0/1337899899 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f113002e5e0 con 0x5648aa464170 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1148f67740 10 client.26029 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:58.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5648aa1a57d0 con 0x5648aa464170 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.948+0000 7f1128c00640 1 -- 172.21.15.44:0/1337899899 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f112c005020 con 0x5648aa464170 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1128c00640 1 -- 172.21.15.44:0/1337899899 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f1130004160 con 0x5648aa464170 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1148f67740 10 client.26029 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1148f67740 10 client.26029 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1148f67740 20 client.26029 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1148f67740 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa4e4800 0x5648aa4e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1148f67740 4 client.26029 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648aa3d08e0 con 0x5648aa4e4800 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.949+0000 7f113ea00640 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa4e4800 0x5648aa4e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.950+0000 7f113ea00640 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa4e4800 0x5648aa4e6bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88f1442f8ba45c3e server_cookie=523990cc39bddb8e in_seq=0 out_seq=0 2024-04-20T11:24:58.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.950+0000 7f1128c00640 10 client.26029 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:58.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1128c00640 1 -- 172.21.15.44:0/1337899899 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648aa3d08e0 con 0x5648aa4e4800 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1128c00640 10 client.26029 handle_command_reply: tid=0 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 client.26029 shutdown 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 msgr2=0x5648aa471930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 0x5648aa471930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f113001b040 tx=0x7f1130011dc0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 shutdown_connections 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648aa46d670 0x5648aa4e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5648aa4e4800 0x5648aa4e6bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648aa464a90 0x5648aa471e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 --2- 172.21.15.44:0/1337899899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648aa464170 0x5648aa471930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 >> 172.21.15.44:0/1337899899 conn(0x5648aa45e8a0 msgr2=0x5648aa46af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1128c00640 0 client.26029 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 shutdown_connections 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.951+0000 7f1148f67740 1 -- 172.21.15.44:0/1337899899 wait complete. 2024-04-20T11:24:58.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1148f67740 20 client.26029 trim_cache size 0 max 16384 2024-04-20T11:24:58.956 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:58.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f11240a3d20 msgr2=0x7f11240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f11240a3d20 0x7f11240a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5648aa45fa30 tx=0x7f112c005f90 comp rx=0 tx=0).stop 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 msgr2=0x7f11401a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 0x7f11401a3640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f11380098d0 tx=0x7f11380325e0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 shutdown_connections 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1140112c80 0x7f11401a3640 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f11240a3d20 0x7f11240a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1140076db0 0x7f114019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 --2- 172.21.15.44:0/57376753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140076490 0x7f114019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 >> 172.21.15.44:0/57376753 conn(0x7f11400ffa70 msgr2=0x7f1140114710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 shutdown_connections 2024-04-20T11:24:58.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:58.952+0000 7f1144c00640 1 -- 172.21.15.44:0/57376753 wait complete. 2024-04-20T11:24:58.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:24:58.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:58 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4011179114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 -- 172.21.15.44:0/2990825549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb01094e0 msgr2=0x7f6cb01110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2990825549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb01094e0 0x7f6cb01110b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ca0 tx=0x7f6ca802c100 comp rx=0 tx=0).stop 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 -- 172.21.15.44:0/2990825549 shutdown_connections 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2990825549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb01094e0 0x7f6cb01110b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2990825549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104dc0 0x7f6cb0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2990825549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb0104410 0x7f6cb01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 -- 172.21.15.44:0/2990825549 >> 172.21.15.44:0/2990825549 conn(0x7f6cb00ffcf0 msgr2=0x7f6cb0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 -- 172.21.15.44:0/2990825549 shutdown_connections 2024-04-20T11:24:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.175+0000 7f6cb8200640 1 -- 172.21.15.44:0/2990825549 wait complete. 2024-04-20T11:24:59.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 Processor -- start 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 -- start start 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104410 0x7f6cb0076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb01094e0 0x7f6cb00769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 0x7f6cb01ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01146b0 con 0x7f6cb01094e0 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0114530 con 0x7f6cb0076ef0 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.176+0000 7f6cb8200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0114830 con 0x7f6cb0104410 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 0x7f6cb01ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb01094e0 0x7f6cb00769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104410 0x7f6cb0076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 0x7f6cb01ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39592/0 (socket says 172.21.15.44:39592) 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb6200640 1 -- 172.21.15.44:0/2774645262 learned_addr learned my addr 172.21.15.44:0/2774645262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 -- 172.21.15.44:0/2774645262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 msgr2=0x7f6cb01ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 0x7f6cb01ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 -- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb01094e0 msgr2=0x7f6cb00769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb01094e0 0x7f6cb00769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 -- 172.21.15.44:0/2774645262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb01ab680 con 0x7f6cb0104410 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb6200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 0x7f6cb01ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb01094e0 0x7f6cb00769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cb5800640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104410 0x7f6cb0076470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e9f0 tx=0x7f6ca400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.177+0000 7f6cae000640 1 -- 172.21.15.44:0/2774645262 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400ccc0 con 0x7f6cb0104410 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.178+0000 7f6cae000640 1 -- 172.21.15.44:0/2774645262 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ca4010930 con 0x7f6cb0104410 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.178+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb01ab910 con 0x7f6cb0104410 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.178+0000 7f6cae000640 1 -- 172.21.15.44:0/2774645262 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40189f0 con 0x7f6cb0104410 2024-04-20T11:24:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.178+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01abe30 con 0x7f6cb0104410 2024-04-20T11:24:59.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 Processor -- start 2024-04-20T11:24:59.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 -- start start 2024-04-20T11:24:59.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cae000640 1 -- 172.21.15.44:0/2774645262 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6ca4010ae0 con 0x7f6cb0104410 2024-04-20T11:24:59.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15f7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d15f78d0 0x55b3d15fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d16004b0 0x55b3d16028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b3d155c050 con 0x55b3d15f6fb0 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b3d1502490 con 0x55b3d15f78d0 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cbae56740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b3d1558010 con 0x55b3d16004b0 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d15f78d0 0x55b3d15fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cb6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d15f78d0 0x55b3d15fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39602/0 (socket says 172.21.15.44:39602) 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cb4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d16004b0 0x55b3d16028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cb5800640 1 -- 172.21.15.44:0/1649384695 learned_addr learned my addr 172.21.15.44:0/1649384695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.181+0000 7f6cb6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15f7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60532/0 (socket says 172.21.15.44:60532) 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6cb4e00640 1 -- 172.21.15.44:0/1649384695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d15f78d0 msgr2=0x55b3d15fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d15f78d0 0x55b3d15fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6cb4e00640 1 -- 172.21.15.44:0/1649384695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 msgr2=0x55b3d15f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6cb4e00640 1 -- 172.21.15.44:0/1649384695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d150a070 con 0x55b3d16004b0 2024-04-20T11:24:59.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d16004b0 0x55b3d16028a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0077c70 tx=0x7f6c9c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e32fd0f723cc7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6ca2a00640 1 -- 172.21.15.44:0/1649384695 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c011700 con 0x55b3d16004b0 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6ca2a00640 1 -- 172.21.15.44:0/1649384695 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6c9c011da0 con 0x55b3d16004b0 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.182+0000 7f6ca2a00640 1 -- 172.21.15.44:0/1649384695 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c01a380 con 0x55b3d16004b0 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cae000640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6c980a3d20 0x7f6c980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 -- 172.21.15.44:0/1649384695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d16004b0 msgr2=0x55b3d16028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cae000640 1 -- 172.21.15.44:0/2774645262 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f6ca40d0a80 con 0x7f6cb0104410 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d16004b0 0x55b3d16028a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0077c70 tx=0x7f6c9c0107a0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cb6200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6c980a3d20 0x7f6c980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 -- 172.21.15.44:0/1649384695 shutdown_connections 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d16004b0 0x55b3d16028a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d15f78d0 0x55b3d15fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 --2- 172.21.15.44:0/1649384695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 -- 172.21.15.44:0/1649384695 >> 172.21.15.44:0/1649384695 conn(0x55b3d15f16e0 msgr2=0x55b3d15f1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 -- 172.21.15.44:0/1649384695 shutdown_connections 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.183+0000 7f6cbae56740 1 -- 172.21.15.44:0/1649384695 wait complete. 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 -- start start 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cb6200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6c980a3d20 0x7f6c980a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8035fd0 tx=0x7f6ca8005d80 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 Processor -- start 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d15fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b3d16032a0 con 0x55b3d15f6fb0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b3d1515860 con 0x55b3d16004b0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.184+0000 7f6cbae56740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b3d1603420 con 0x55b3d15f78d0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cbae56740 10 client.0 _ll_register_callbacks cb 0x55b3d14e3fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60548/0 (socket says 172.21.15.44:60548) 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 -- 172.21.15.44:0/715065645 learned_addr learned my addr 172.21.15.44:0/715065645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d15fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 -- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 msgr2=0x55b3d15fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d15fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 -- 172.21.15.44:0/715065645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 msgr2=0x55b3d15fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 -- 172.21.15.44:0/715065645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d1519950 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb6200640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d15fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6cb5800640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15fd510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c6a0 tx=0x7f6ca4010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f6584c9e670ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.185+0000 7f6c97400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6c97400640 1 -- 172.21.15.44:0/715065645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40da040 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6c97400640 1 -- 172.21.15.44:0/715065645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f6ca4018d90 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c011da0 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6c97400640 1 -- 172.21.15.44:0/715065645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6ca409c620 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 10 client.15909 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b3d14e7820 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6c97400640 1 -- 172.21.15.44:0/715065645 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6c9c011da0 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6c97400640 1 -- 172.21.15.44:0/715065645 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f6ca4098830 con 0x55b3d15f6fb0 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 10 client.15909 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:59.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 20 client.15909 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.186+0000 7f6cbae56740 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1610470 0x55b3d16128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.187+0000 7f6cbae56740 4 client.15909 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.187+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3d1569a70 con 0x55b3d1610470 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.187+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1610470 0x55b3d16128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.187+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1610470 0x55b3d16128e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49d662980c6d9a46 server_cookie=9704ab8b8b245f46 in_seq=0 out_seq=0 2024-04-20T11:24:59.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.187+0000 7f6c97400640 10 client.15909 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6c97400640 1 -- 172.21.15.44:0/715065645 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3d1569a70 con 0x55b3d1610470 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6c97400640 10 client.15909 handle_command_reply: tid=0 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 client.15909 shutdown 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 msgr2=0x55b3d15fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15fd510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c6a0 tx=0x7f6ca4010dd0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 shutdown_connections 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1610470 0x55b3d16128e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d15fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 --2- 172.21.15.44:0/715065645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15fd510 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6c97400640 0 client.15909 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.188+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 >> 172.21.15.44:0/715065645 conn(0x55b3d15f16e0 msgr2=0x55b3d15f3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.189+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 shutdown_connections 2024-04-20T11:24:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.189+0000 7f6cbae56740 1 -- 172.21.15.44:0/715065645 wait complete. 2024-04-20T11:24:59.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.189+0000 7f6cbae56740 20 client.15909 trim_cache size 0 max 16384 2024-04-20T11:24:59.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 Processor -- start 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 -- start start 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d160c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01b4310 con 0x55b3d15f6fb0 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.206+0000 7f6cbae56740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01b5f10 con 0x55b3d16004b0 2024-04-20T11:24:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.207+0000 7f6cbae56740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01b49e0 con 0x55b3d15f78d0 2024-04-20T11:24:59.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.207+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.207+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60554/0 (socket says 172.21.15.44:60554) 2024-04-20T11:24:59.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.207+0000 7f6cb5800640 1 -- 172.21.15.44:0/3370719553 learned_addr learned my addr 172.21.15.44:0/3370719553 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.207+0000 7f6cb6200640 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.207+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d160c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.208+0000 7f6cb5800640 1 -- 172.21.15.44:0/3370719553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 msgr2=0x55b3d160c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.208+0000 7f6cb5800640 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d160c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.208+0000 7f6cb5800640 1 -- 172.21.15.44:0/3370719553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 msgr2=0x55b3d15fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.208+0000 7f6cb5800640 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.208+0000 7f6cb5800640 1 -- 172.21.15.44:0/3370719553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d15f7700 con 0x55b3d15f6fb0 2024-04-20T11:24:59.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.208+0000 7f6cb4e00640 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d160c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:59.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.209+0000 7f6cb5800640 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15ffc40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ca409ca70 tx=0x7f6ca40d0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f5b6ca55923747 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.209+0000 7f6ca2000640 1 -- 172.21.15.44:0/3370719553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40ce670 con 0x55b3d15f6fb0 2024-04-20T11:24:59.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.209+0000 7f6ca2000640 1 -- 172.21.15.44:0/3370719553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ca409cae0 con 0x55b3d15f6fb0 2024-04-20T11:24:59.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.209+0000 7f6ca2000640 1 -- 172.21.15.44:0/3370719553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca409c440 con 0x55b3d15f6fb0 2024-04-20T11:24:59.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 -- 172.21.15.44:0/3370719553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 msgr2=0x55b3d15ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15ffc40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ca409ca70 tx=0x7f6ca40d0d20 comp rx=0 tx=0).stop 2024-04-20T11:24:59.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 -- 172.21.15.44:0/3370719553 shutdown_connections 2024-04-20T11:24:59.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d15fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d160c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 --2- 172.21.15.44:0/3370719553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d15ffc40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.211+0000 7f6cbae56740 1 -- 172.21.15.44:0/3370719553 >> 172.21.15.44:0/3370719553 conn(0x55b3d15f16e0 msgr2=0x55b3d15fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.212+0000 7f6cbae56740 1 -- 172.21.15.44:0/3370719553 shutdown_connections 2024-04-20T11:24:59.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.212+0000 7f6cbae56740 1 -- 172.21.15.44:0/3370719553 wait complete. 2024-04-20T11:24:59.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.213+0000 7f6cbae56740 1 -- start start 2024-04-20T11:24:59.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.213+0000 7f6cbae56740 1 Processor -- start 2024-04-20T11:24:59.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.213+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d1672110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.213+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d1608670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.213+0000 7f6cbae56740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d1608bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cbae56740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01b5780 con 0x55b3d15f6fb0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cbae56740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01b50b0 con 0x55b3d16004b0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cbae56740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01b2290 con 0x55b3d15f78d0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cbae56740 10 client.0 _ll_register_callbacks cb 0x7f6ca4010ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d1608670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d1672110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d1608bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d1608670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40090/0 (socket says 172.21.15.44:40090) 2024-04-20T11:24:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb4e00640 1 -- 172.21.15.44:0/2925844976 learned_addr learned my addr 172.21.15.44:0/2925844976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb6200640 1 -- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 msgr2=0x55b3d1608670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb6200640 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d1608670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb6200640 1 -- 172.21.15.44:0/2925844976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 msgr2=0x55b3d1672110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb6200640 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d1672110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.214+0000 7f6cb6200640 1 -- 172.21.15.44:0/2925844976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d126e3f0 con 0x55b3d16004b0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.215+0000 7f6cb6200640 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d1608bb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005e30 tx=0x7f6ca80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba20508c56a2222 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.215+0000 7f6c97400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.215+0000 7f6c97400640 1 -- 172.21.15.44:0/2925844976 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8039040 con 0x55b3d16004b0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.215+0000 7f6c97400640 1 -- 172.21.15.44:0/2925844976 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f6ca8032a10 con 0x55b3d16004b0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.215+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca409cae0 con 0x55b3d16004b0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.216+0000 7f6c97400640 1 -- 172.21.15.44:0/2925844976 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6ca8003fc0 con 0x55b3d16004b0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.216+0000 7f6cbae56740 10 client.26039 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.216+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b3d13387d0 con 0x55b3d16004b0 2024-04-20T11:24:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.216+0000 7f6c97400640 1 -- 172.21.15.44:0/2925844976 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ca409cae0 con 0x55b3d16004b0 2024-04-20T11:24:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6c97400640 1 -- 172.21.15.44:0/2925844976 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f6ca8060050 con 0x55b3d16004b0 2024-04-20T11:24:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6cbae56740 10 client.26039 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6cbae56740 10 client.26039 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:24:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6cbae56740 20 client.26039 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6cbae56740 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1688090 0x55b3d168a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6cbae56740 4 client.26039 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:59.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.217+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3d15638e0 con 0x55b3d1688090 2024-04-20T11:24:59.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.218+0000 7f6cb5800640 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1688090 0x55b3d168a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.218+0000 7f6c97400640 10 client.26039 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:59.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.219+0000 7f6cb5800640 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1688090 0x55b3d168a4b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e658c65e032f2f96 server_cookie=db8eb577fe204e63 in_seq=0 out_seq=0 2024-04-20T11:24:59.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6c97400640 1 -- 172.21.15.44:0/2925844976 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3d15638e0 con 0x55b3d1688090 2024-04-20T11:24:59.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6c97400640 10 client.26039 handle_command_reply: tid=0 2024-04-20T11:24:59.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6cbae56740 1 client.26039 shutdown 2024-04-20T11:24:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 msgr2=0x55b3d1608bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6cbae56740 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d1608bb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005e30 tx=0x7f6ca80095c0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 shutdown_connections 2024-04-20T11:24:59.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.220+0000 7f6cbae56740 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55b3d1688090 0x55b3d168a4b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b3d16004b0 0x55b3d1608bb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b3d15f78d0 0x55b3d1608670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 1 --2- 172.21.15.44:0/2925844976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3d15f6fb0 0x55b3d1672110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 >> 172.21.15.44:0/2925844976 conn(0x55b3d15f16e0 msgr2=0x55b3d15fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6c97400640 0 client.26039 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 shutdown_connections 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 1 -- 172.21.15.44:0/2925844976 wait complete. 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.221+0000 7f6cbae56740 20 client.26039 trim_cache size 0 max 16384 2024-04-20T11:24:59.222 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6c980a3d20 msgr2=0x7f6c980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6c980a3d20 0x7f6c980a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8035fd0 tx=0x7f6ca8005d80 comp rx=0 tx=0).stop 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104410 msgr2=0x7f6cb0076470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104410 0x7f6cb0076470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e9f0 tx=0x7f6ca400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 shutdown_connections 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cb0076ef0 0x7f6cb01ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6c980a3d20 0x7f6c980a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cb01094e0 0x7f6cb00769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 --2- 172.21.15.44:0/2774645262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cb0104410 0x7f6cb0076470 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 >> 172.21.15.44:0/2774645262 conn(0x7f6cb00ffcf0 msgr2=0x7f6cb0100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 shutdown_connections 2024-04-20T11:24:59.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.222+0000 7f6cb8200640 1 -- 172.21.15.44:0/2774645262 wait complete. 2024-04-20T11:24:59.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:24:59.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 -- 172.21.15.44:0/1372888837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5220104410 msgr2=0x7f52201047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 --2- 172.21.15.44:0/1372888837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5220104410 0x7f52201047f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5210009900 tx=0x7f521002bc30 comp rx=0 tx=0).stop 2024-04-20T11:24:59.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 -- 172.21.15.44:0/1372888837 shutdown_connections 2024-04-20T11:24:59.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 --2- 172.21.15.44:0/1372888837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52201094e0 0x7f52201110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 --2- 172.21.15.44:0/1372888837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5220104dc0 0x7f5220108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 --2- 172.21.15.44:0/1372888837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5220104410 0x7f52201047f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 -- 172.21.15.44:0/1372888837 >> 172.21.15.44:0/1372888837 conn(0x7f52200ffcf0 msgr2=0x7f5220102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 -- 172.21.15.44:0/1372888837 shutdown_connections 2024-04-20T11:24:59.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.425+0000 7f5226200640 1 -- 172.21.15.44:0/1372888837 wait complete. 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.426+0000 7f5226200640 1 Processor -- start 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.426+0000 7f5226200640 1 -- start start 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f5226200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5220104410 0x7f52201a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f5226200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52201094e0 0x7f52201a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f5226200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 0x7f522019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f5226200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5220114680 con 0x7f52201094e0 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f5226200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5220114500 con 0x7f5220104410 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f5226200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5220114800 con 0x7f522019a9e0 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52201094e0 0x7f52201a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 0x7f522019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 0x7f522019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40102/0 (socket says 172.21.15.44:40102) 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 -- 172.21.15.44:0/3190554088 learned_addr learned my addr 172.21.15.44:0/3190554088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 -- 172.21.15.44:0/3190554088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5220104410 msgr2=0x7f52201a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5220104410 0x7f52201a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 -- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52201094e0 msgr2=0x7f52201a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52201094e0 0x7f52201a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.427+0000 7f521fe00640 1 -- 172.21.15.44:0/3190554088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522019b500 con 0x7f522019a9e0 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.428+0000 7f521fe00640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 0x7f522019ae90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f521800eaa0 tx=0x7f521800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.428+0000 7f5217e00640 1 -- 172.21.15.44:0/3190554088 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521800cdb0 con 0x7f522019a9e0 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.428+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522019b7f0 con 0x7f522019a9e0 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.428+0000 7f5217e00640 1 -- 172.21.15.44:0/3190554088 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5218004b20 con 0x7f522019a9e0 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.428+0000 7f5217e00640 1 -- 172.21.15.44:0/3190554088 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218011760 con 0x7f522019a9e0 2024-04-20T11:24:59.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.428+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52201a7a40 con 0x7f522019a9e0 2024-04-20T11:24:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.431+0000 7f5228e82740 1 Processor -- start 2024-04-20T11:24:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5217e00640 1 -- 172.21.15.44:0/3190554088 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5218011900 con 0x7f522019a9e0 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 -- start start 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c08925fb0 0x564c08926390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0892ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c089318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564c0888b050 con 0x564c089268d0 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c08831490 con 0x564c08925fb0 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.432+0000 7f5228e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564c08887010 con 0x564c0892f4b0 2024-04-20T11:24:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c08925fb0 0x564c08926390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0892ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c08925fb0 0x564c08926390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39636/0 (socket says 172.21.15.44:39636) 2024-04-20T11:24:59.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521ea00640 1 -- 172.21.15.44:0/2112795695 learned_addr learned my addr 172.21.15.44:0/2112795695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521ea00640 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c089318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521f400640 1 -- 172.21.15.44:0/2112795695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 msgr2=0x564c089318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521f400640 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c089318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521f400640 1 -- 172.21.15.44:0/2112795695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c08925fb0 msgr2=0x564c08926390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521f400640 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c08925fb0 0x564c08926390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.433+0000 7f521f400640 1 -- 172.21.15.44:0/2112795695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c08839070 con 0x564c089268d0 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.434+0000 7f5217e00640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f52040a3d20 0x7f52040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.434+0000 7f521f400640 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0892ef70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f521002c170 tx=0x7f52100095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79f074894b591b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.434+0000 7f5217e00640 1 -- 172.21.15.44:0/3190554088 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f52180d09e0 con 0x7f522019a9e0 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.434+0000 7f521ea00640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f52040a3d20 0x7f52040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.434+0000 7f520fe00640 1 -- 172.21.15.44:0/2112795695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210042020 con 0x564c089268d0 2024-04-20T11:24:59.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.434+0000 7f520fe00640 1 -- 172.21.15.44:0/2112795695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f521002c7d0 con 0x564c089268d0 2024-04-20T11:24:59.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f520fe00640 1 -- 172.21.15.44:0/2112795695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210003950 con 0x564c089268d0 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 -- 172.21.15.44:0/2112795695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 msgr2=0x564c0892ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0892ef70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f521002c170 tx=0x7f52100095f0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f521ea00640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f52040a3d20 0x7f52040a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x564c08921870 tx=0x7f52080096c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 -- 172.21.15.44:0/2112795695 shutdown_connections 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c089318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0892ef70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 --2- 172.21.15.44:0/2112795695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c08925fb0 0x564c08926390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 -- 172.21.15.44:0/2112795695 >> 172.21.15.44:0/2112795695 conn(0x564c089206e0 msgr2=0x564c08920ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 -- 172.21.15.44:0/2112795695 shutdown_connections 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.435+0000 7f5228e82740 1 -- 172.21.15.44:0/2112795695 wait complete. 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.436+0000 7f5228e82740 1 -- start start 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.436+0000 7f5228e82740 1 Processor -- start 2024-04-20T11:24:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.436+0000 7f5228e82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c0892c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f5228e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c089268d0 0x564c0892c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f5228e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f5228e82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564c08932290 con 0x564c08925fb0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f5228e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c08844860 con 0x564c0892f4b0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f5228e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564c08932410 con 0x564c089268d0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f5228e82740 10 client.0 _ll_register_callbacks cb 0x564c08812fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c0892c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c089268d0 0x564c0892c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c0892c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60612/0 (socket says 172.21.15.44:60612) 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 -- 172.21.15.44:0/955352450 learned_addr learned my addr 172.21.15.44:0/955352450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521fe00640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 -- 172.21.15.44:0/955352450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c089268d0 msgr2=0x564c0892c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c089268d0 0x564c0892c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 -- 172.21.15.44:0/955352450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 msgr2=0x564c0892ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521f400640 1 -- 172.21.15.44:0/955352450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c08848950 con 0x564c08925fb0 2024-04-20T11:24:59.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521ea00640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c089268d0 0x564c0892c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.437+0000 7f521fe00640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:24:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f521f400640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c0892c3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f521002c700 tx=0x7f521002c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fc2dc4fe54d44c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f520cc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:24:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f520cc00640 1 -- 172.21.15.44:0/955352450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210042030 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521002c7d0 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f520cc00640 1 -- 172.21.15.44:0/955352450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f52100038f0 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f520cc00640 1 -- 172.21.15.44:0/955352450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5210004490 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f5228e82740 10 client.15927 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.438+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c08816820 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f520cc00640 1 -- 172.21.15.44:0/955352450 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f521002c7d0 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f520cc00640 1 -- 172.21.15.44:0/955352450 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f5210047420 con 0x564c08925fb0 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f5228e82740 10 client.15927 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f5228e82740 10 client.15927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f5228e82740 20 client.15927 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f5228e82740 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c0893f260 0x564c08941650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f5228e82740 4 client.15927 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.439+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c08898a70 con 0x564c0893f260 2024-04-20T11:24:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.440+0000 7f521ea00640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c0893f260 0x564c08941650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.440+0000 7f521ea00640 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c0893f260 0x564c08941650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edc209cf051f88da server_cookie=2be8abb0c11da2db in_seq=0 out_seq=0 2024-04-20T11:24:59.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.441+0000 7f520cc00640 10 client.15927 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f520cc00640 1 -- 172.21.15.44:0/955352450 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c08898a70 con 0x564c0893f260 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f520cc00640 10 client.15927 handle_command_reply: tid=0 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 client.15927 shutdown 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 msgr2=0x564c0892c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c0892c3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f521002c700 tx=0x7f521002c730 comp rx=0 tx=0).stop 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 shutdown_connections 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c0893f260 0x564c08941650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c089268d0 0x564c0892c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 --2- 172.21.15.44:0/955352450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c0892c3c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.442+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 >> 172.21.15.44:0/955352450 conn(0x564c089206e0 msgr2=0x564c08922430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.443+0000 7f520cc00640 0 client.15927 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:59.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.443+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 shutdown_connections 2024-04-20T11:24:59.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.443+0000 7f5228e82740 1 -- 172.21.15.44:0/955352450 wait complete. 2024-04-20T11:24:59.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.443+0000 7f5228e82740 20 client.15927 trim_cache size 0 max 16384 2024-04-20T11:24:59.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.461+0000 7f5228e82740 1 Processor -- start 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 -- start start 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 0x564c0892eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0893b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52201afed0 con 0x564c089268d0 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52201b1ad0 con 0x564c0892f4b0 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f5228e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52201b05a0 con 0x564c08925fb0 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f521f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 0x564c0892eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f521f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 0x564c0892eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40126/0 (socket says 172.21.15.44:40126) 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f521f400640 1 -- 172.21.15.44:0/2325093872 learned_addr learned my addr 172.21.15.44:0/2325093872 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f521fe00640 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.462+0000 7f521ea00640 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0893b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521f400640 1 -- 172.21.15.44:0/2325093872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 msgr2=0x564c0892ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521f400640 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521f400640 1 -- 172.21.15.44:0/2325093872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 msgr2=0x564c0893b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521f400640 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0893b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521f400640 1 -- 172.21.15.44:0/2325093872 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0859d3f0 con 0x564c08925fb0 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521fe00640 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f521f400640 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 0x564c0892eaf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5210004650 tx=0x7f5210005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9856279620c838 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f520f400640 1 -- 172.21.15.44:0/2325093872 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521005c070 con 0x564c08925fb0 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f520f400640 1 -- 172.21.15.44:0/2325093872 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f521004b070 con 0x564c08925fb0 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.463+0000 7f520f400640 1 -- 172.21.15.44:0/2325093872 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210042070 con 0x564c08925fb0 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 -- 172.21.15.44:0/2325093872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 msgr2=0x564c0892eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 0x564c0892eaf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5210004650 tx=0x7f5210005bd0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 -- 172.21.15.44:0/2325093872 shutdown_connections 2024-04-20T11:24:59.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0892f4b0 0x564c0892ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c089268d0 0x564c0893b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 --2- 172.21.15.44:0/2325093872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c08925fb0 0x564c0892eaf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 -- 172.21.15.44:0/2325093872 >> 172.21.15.44:0/2325093872 conn(0x564c089206e0 msgr2=0x564c0892c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 -- 172.21.15.44:0/2325093872 shutdown_connections 2024-04-20T11:24:59.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.464+0000 7f5228e82740 1 -- 172.21.15.44:0/2325093872 wait complete. 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.465+0000 7f5228e82740 1 -- start start 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.465+0000 7f5228e82740 1 Processor -- start 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.465+0000 7f5228e82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c089376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f5228e82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c089268d0 0x564c08937bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f5228e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c08928bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f5228e82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52201b1340 con 0x564c08925fb0 2024-04-20T11:24:59.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f5228e82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52201b0c70 con 0x564c089268d0 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f5228e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52201ade30 con 0x564c0892f4b0 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f5228e82740 10 client.0 _ll_register_callbacks cb 0x7f5210009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c08928bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c08928bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40130/0 (socket says 172.21.15.44:40130) 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 -- 172.21.15.44:0/2092935002 learned_addr learned my addr 172.21.15.44:0/2092935002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521ea00640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c089268d0 0x564c08937bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521f400640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c089376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 -- 172.21.15.44:0/2092935002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c089268d0 msgr2=0x564c08937bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c089268d0 0x564c08937bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 -- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 msgr2=0x564c089376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c089376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.466+0000 7f521fe00640 1 -- 172.21.15.44:0/2092935002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c086677d0 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.467+0000 7f521fe00640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c08928bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f521800efd0 tx=0x7f521809d1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9392c3e4c2ad321d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.467+0000 7f520cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.467+0000 7f520cc00640 1 -- 172.21.15.44:0/2092935002 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521801f040 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.467+0000 7f520cc00640 1 -- 172.21.15.44:0/2092935002 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f52180a5170 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.467+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521004b070 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f520cc00640 1 -- 172.21.15.44:0/2092935002 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52180a5600 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f5228e82740 10 client.26065 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c089a4f80 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f520cc00640 1 -- 172.21.15.44:0/2092935002 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f521004b070 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f520cc00640 1 -- 172.21.15.44:0/2092935002 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f521800f040 con 0x564c0892f4b0 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f5228e82740 10 client.26065 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.468+0000 7f5228e82740 10 client.26065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:24:59.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.469+0000 7f5228e82740 20 client.26065 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:59.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.469+0000 7f5228e82740 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c089a5650 0x564c089a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.469+0000 7f5228e82740 4 client.26065 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:59.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.469+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c088928e0 con 0x564c089a5650 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.469+0000 7f521f400640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c089a5650 0x564c089a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.470+0000 7f521f400640 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c089a5650 0x564c089a7a60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9ebd6720cebb290 server_cookie=f8a4ce23b523d397 in_seq=0 out_seq=0 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.470+0000 7f520cc00640 10 client.26065 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f520cc00640 1 -- 172.21.15.44:0/2092935002 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c088928e0 con 0x564c089a5650 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f520cc00640 10 client.26065 handle_command_reply: tid=0 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 client.26065 shutdown 2024-04-20T11:24:59.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 msgr2=0x564c08928bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c08928bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f521800efd0 tx=0x7f521809d1d0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 shutdown_connections 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x564c089a5650 0x564c089a7a60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c0892f4b0 0x564c08928bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c089268d0 0x564c08937bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 --2- 172.21.15.44:0/2092935002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c08925fb0 0x564c089376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f520cc00640 0 client.26065 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 >> 172.21.15.44:0/2092935002 conn(0x564c089206e0 msgr2=0x564c0891f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 shutdown_connections 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 1 -- 172.21.15.44:0/2092935002 wait complete. 2024-04-20T11:24:59.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.471+0000 7f5228e82740 20 client.26065 trim_cache size 0 max 16384 2024-04-20T11:24:59.475 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:59.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f52040a3d20 msgr2=0x7f52040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f52040a3d20 0x7f52040a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x564c08921870 tx=0x7f52080096c0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 msgr2=0x7f522019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 0x7f522019ae90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f521800eaa0 tx=0x7f521800ef60 comp rx=0 tx=0).stop 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 shutdown_connections 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f522019a9e0 0x7f522019ae90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f52040a3d20 0x7f52040a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52201094e0 0x7f52201a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 --2- 172.21.15.44:0/3190554088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5220104410 0x7f52201a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 >> 172.21.15.44:0/3190554088 conn(0x7f52200ffcf0 msgr2=0x7f5220102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 shutdown_connections 2024-04-20T11:24:59.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.472+0000 7f5226200640 1 -- 172.21.15.44:0/3190554088 wait complete. 2024-04-20T11:24:59.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:24:59.551 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:24:59 smithi044 ceph-mon[16855]: pgmap v74: 97 pgs: 97 active+clean; 121 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 -- 172.21.15.44:0/3998301107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c281016d0 msgr2=0x7f2c28105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 --2- 172.21.15.44:0/3998301107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c281016d0 0x7f2c28105b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009900 tx=0x7f2c1002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 -- 172.21.15.44:0/3998301107 shutdown_connections 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 --2- 172.21.15.44:0/3998301107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c281061b0 0x7f2c28106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 --2- 172.21.15.44:0/3998301107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c281016d0 0x7f2c28105b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 --2- 172.21.15.44:0/3998301107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c28100db0 0x7f2c28101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 -- 172.21.15.44:0/3998301107 >> 172.21.15.44:0/3998301107 conn(0x7f2c280fc7e0 msgr2=0x7f2c280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 -- 172.21.15.44:0/3998301107 shutdown_connections 2024-04-20T11:24:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.698+0000 7f2c27200640 1 -- 172.21.15.44:0/3998301107 wait complete. 2024-04-20T11:24:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.699+0000 7f2c27200640 1 Processor -- start 2024-04-20T11:24:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.699+0000 7f2c27200640 1 -- start start 2024-04-20T11:24:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.699+0000 7f2c27200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 0x7f2c2819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.699+0000 7f2c27200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c281016d0 0x7f2c2819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.699+0000 7f2c27200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c281061b0 0x7f2c281a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c27200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281188a0 con 0x7f2c281016d0 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c27200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28118720 con 0x7f2c28100db0 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c27200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28118a20 con 0x7f2c281061b0 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 0x7f2c2819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 0x7f2c2819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39694/0 (socket says 172.21.15.44:39694) 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 -- 172.21.15.44:0/1404155998 learned_addr learned my addr 172.21.15.44:0/1404155998 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c281016d0 0x7f2c2819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c26800640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c281061b0 0x7f2c281a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 -- 172.21.15.44:0/1404155998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c281061b0 msgr2=0x7f2c281a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c281061b0 0x7f2c281a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 -- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c281016d0 msgr2=0x7f2c2819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c281016d0 0x7f2c2819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 -- 172.21.15.44:0/1404155998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c281a3d70 con 0x7f2c28100db0 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c26800640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c281061b0 0x7f2c281a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.700+0000 7f2c25e00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 0x7f2c2819ed80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800e9f0 tx=0x7f2c1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.701+0000 7f2c1ea00640 1 -- 172.21.15.44:0/1404155998 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800cd80 con 0x7f2c28100db0 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.701+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c281a4060 con 0x7f2c28100db0 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.701+0000 7f2c1ea00640 1 -- 172.21.15.44:0/1404155998 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c18010930 con 0x7f2c28100db0 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.701+0000 7f2c1ea00640 1 -- 172.21.15.44:0/1404155998 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18018b00 con 0x7f2c28100db0 2024-04-20T11:24:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.701+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281ab8f0 con 0x7f2c28100db0 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 Processor -- start 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 -- start start 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c1ea00640 1 -- 172.21.15.44:0/1404155998 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2c18018ca0 con 0x7f2c28100db0 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 0x558689e4c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e57a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558689db1050 con 0x558689e55670 2024-04-20T11:24:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558689d57490 con 0x558689e4c170 2024-04-20T11:24:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.704+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558689dad010 con 0x558689e4ca90 2024-04-20T11:24:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 0x558689e4c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c25e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 0x558689e4c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39704/0 (socket says 172.21.15.44:39704) 2024-04-20T11:24:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 -- 172.21.15.44:0/3451266825 learned_addr learned my addr 172.21.15.44:0/3451266825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 -- 172.21.15.44:0/3451266825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 msgr2=0x558689e55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e55130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 -- 172.21.15.44:0/3451266825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 msgr2=0x558689e57a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e57a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c26800640 1 -- 172.21.15.44:0/3451266825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558689d5f070 con 0x558689e4c170 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.705+0000 7f2c25e00640 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c26800640 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 0x558689e4c550 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009fd0 tx=0x7f2c20010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb20050979ff5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c1ea00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c0c0a3da0 0x7f2c0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c17400640 1 -- 172.21.15.44:0/3451266825 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011700 con 0x558689e4c170 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c17400640 1 -- 172.21.15.44:0/3451266825 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c20011da0 con 0x558689e4c170 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c17400640 1 -- 172.21.15.44:0/3451266825 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000e620 con 0x558689e4c170 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c1ea00640 1 -- 172.21.15.44:0/1404155998 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2c180d0ba0 con 0x7f2c28100db0 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.706+0000 7f2c25e00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c0c0a3da0 0x7f2c0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3451266825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 msgr2=0x558689e4c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 0x558689e4c550 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009fd0 tx=0x7f2c20010860 comp rx=0 tx=0).stop 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c25e00640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c0c0a3da0 0x7f2c0c0a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x558689e4bc40 tx=0x7f2c180d4650 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3451266825 shutdown_connections 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e57a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3451266825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4c170 0x558689e4c550 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3451266825 >> 172.21.15.44:0/3451266825 conn(0x558689e468a0 msgr2=0x558689e46c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3451266825 shutdown_connections 2024-04-20T11:24:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.707+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3451266825 wait complete. 2024-04-20T11:24:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.708+0000 7f2c2fbb3740 1 -- start start 2024-04-20T11:24:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.708+0000 7f2c2fbb3740 1 Processor -- start 2024-04-20T11:24:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e52700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e52c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e53180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558689e58460 con 0x558689e55670 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558689d6a860 con 0x558689e4ca90 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558689e585e0 con 0x558689e4c170 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c2fbb3740 10 client.0 _ll_register_callbacks cb 0x558689d38fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e52c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c26800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e52c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39718/0 (socket says 172.21.15.44:39718) 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c1fe00640 1 -- 172.21.15.44:0/2854471287 learned_addr learned my addr 172.21.15.44:0/2854471287 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c26800640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e52700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c1fe00640 1 -- 172.21.15.44:0/2854471287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 msgr2=0x558689e52700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.709+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e52700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c1fe00640 1 -- 172.21.15.44:0/2854471287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 msgr2=0x558689e53180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e53180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c1fe00640 1 -- 172.21.15.44:0/2854471287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558689d6e950 con 0x558689e4ca90 2024-04-20T11:24:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c26800640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e53180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e52c40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c10036040 tx=0x7f2c10009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97998c86e54e1b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c0be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c0be00640 1 -- 172.21.15.44:0/2854471287 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10045020 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c20011da0 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c0be00640 1 -- 172.21.15.44:0/2854471287 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2c10004040 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c0be00640 1 -- 172.21.15.44:0/2854471287 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2c10050050 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.710+0000 7f2c2fbb3740 10 client.26063 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558689d3c820 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c0be00640 1 -- 172.21.15.44:0/2854471287 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c20011da0 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c0be00640 1 -- 172.21.15.44:0/2854471287 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f2c10004b10 con 0x558689e4ca90 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 10 client.26063 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 20 client.26063 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689e65720 0x558689e67b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 4 client.26063 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:24:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.711+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558689dbea70 con 0x558689e65720 2024-04-20T11:24:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.712+0000 7f2c26800640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689e65720 0x558689e67b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.712+0000 7f2c26800640 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689e65720 0x558689e67b40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996a7d8857078bde server_cookie=560f955659edcd61 in_seq=0 out_seq=0 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.712+0000 7f2c0be00640 10 client.26063 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.713+0000 7f2c0be00640 1 -- 172.21.15.44:0/2854471287 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558689dbea70 con 0x558689e65720 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.713+0000 7f2c0be00640 10 client.26063 handle_command_reply: tid=0 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.713+0000 7f2c2fbb3740 1 client.26063 shutdown 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 msgr2=0x558689e52c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e52c40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c10036040 tx=0x7f2c10009680 comp rx=0 tx=0).stop 2024-04-20T11:24:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 shutdown_connections 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689e65720 0x558689e67b40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e53180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e52c40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/2854471287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e52700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 >> 172.21.15.44:0/2854471287 conn(0x558689e468a0 msgr2=0x558689e485f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c0be00640 0 client.26063 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 shutdown_connections 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/2854471287 wait complete. 2024-04-20T11:24:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.714+0000 7f2c2fbb3740 20 client.26063 trim_cache size 0 max 16384 2024-04-20T11:24:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.731+0000 7f2c2fbb3740 1 Processor -- start 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.731+0000 7f2c2fbb3740 1 -- start start 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.731+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 0x558689e54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.731+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.731+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e55670 0x558689e61320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b4070 con 0x558689e4c170 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b5c70 con 0x558689e55670 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b4740 con 0x558689e4ca90 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c26800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 0x558689e54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c26800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 0x558689e54e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60686/0 (socket says 172.21.15.44:60686) 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c26800640 1 -- 172.21.15.44:0/3697406615 learned_addr learned my addr 172.21.15.44:0/3697406615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.732+0000 7f2c26800640 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e55670 0x558689e61320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c1fe00640 1 -- 172.21.15.44:0/3697406615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e55670 msgr2=0x558689e61320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e55670 0x558689e61320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c1fe00640 1 -- 172.21.15.44:0/3697406615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 msgr2=0x558689e54e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 0x558689e54e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c1fe00640 1 -- 172.21.15.44:0/3697406615 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558689e4c8c0 con 0x558689e4ca90 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c26800640 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 0x558689e54e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e4de50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009900 tx=0x7f2c1002bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6193ede8e7acfcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c16a00640 1 -- 172.21.15.44:0/3697406615 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10045040 con 0x558689e4ca90 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c16a00640 1 -- 172.21.15.44:0/3697406615 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c10005850 con 0x558689e4ca90 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.733+0000 7f2c16a00640 1 -- 172.21.15.44:0/3697406615 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10003d20 con 0x558689e4ca90 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3697406615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 msgr2=0x558689e4de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e4de50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009900 tx=0x7f2c1002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3697406615 shutdown_connections 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e55670 0x558689e61320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4ca90 0x558689e4de50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/3697406615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e4c170 0x558689e54e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3697406615 >> 172.21.15.44:0/3697406615 conn(0x558689e468a0 msgr2=0x558689e450d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3697406615 shutdown_connections 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.734+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/3697406615 wait complete. 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 -- start start 2024-04-20T11:24:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 Processor -- start 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e59160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e59be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b54e0 con 0x558689e55670 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b4e10 con 0x558689e4ca90 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b1fb0 con 0x558689e4c170 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.735+0000 7f2c2fbb3740 10 client.0 _ll_register_callbacks cb 0x7f2c10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c26800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39748/0 (socket says 172.21.15.44:39748) 2024-04-20T11:24:59.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 -- 172.21.15.44:0/4019061597 learned_addr learned my addr 172.21.15.44:0/4019061597 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c26800640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e59be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 -- 172.21.15.44:0/4019061597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 msgr2=0x558689e59160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e59160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 -- 172.21.15.44:0/4019061597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 msgr2=0x558689e59be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e59be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 -- 172.21.15.44:0/4019061597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558689ac33f0 con 0x558689e4ca90 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c26800640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e59be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c1fe00640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e596a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2c1002c210 tx=0x7f2c10004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6762430c2ab837c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c0be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.736+0000 7f2c0be00640 1 -- 172.21.15.44:0/4019061597 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100051e0 con 0x558689e4ca90 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c10005850 con 0x558689e4ca90 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c0be00640 1 -- 172.21.15.44:0/4019061597 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2c10005380 con 0x558689e4ca90 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c0be00640 1 -- 172.21.15.44:0/4019061597 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2c10005580 con 0x558689e4ca90 2024-04-20T11:24:59.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 10 client.26069 fetch_fsmap learned FSMap version 31 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558689b8d7d0 con 0x558689e4ca90 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c0be00640 1 -- 172.21.15.44:0/4019061597 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c10005850 con 0x558689e4ca90 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c0be00640 1 -- 172.21.15.44:0/4019061597 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f2c10065020 con 0x558689e4ca90 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 10 client.26069 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 20 client.26069 populate_metadata read hostname 'smithi044' 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689ecc250 0x558689ece670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 4 client.26069 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.737+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558689db88e0 con 0x558689ecc250 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c26800640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689ecc250 0x558689ece670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c26800640 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689ecc250 0x558689ece670 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f139e8d6605f2a server_cookie=9d764f1c0eb0e0cd in_seq=0 out_seq=0 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c0be00640 10 client.26069 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c0be00640 1 -- 172.21.15.44:0/4019061597 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558689db88e0 con 0x558689ecc250 2024-04-20T11:24:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c0be00640 10 client.26069 handle_command_reply: tid=0 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c2fbb3740 1 client.26069 shutdown 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 msgr2=0x558689e596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e596a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2c1002c210 tx=0x7f2c10004c80 comp rx=0 tx=0).stop 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.738+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 shutdown_connections 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558689ecc250 0x558689ece670 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558689e55670 0x558689e59be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c0be00640 0 client.26069 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558689e4ca90 0x558689e596a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 --2- 172.21.15.44:0/4019061597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558689e4c170 0x558689e59160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 >> 172.21.15.44:0/4019061597 conn(0x558689e468a0 msgr2=0x558689e52f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 shutdown_connections 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 1 -- 172.21.15.44:0/4019061597 wait complete. 2024-04-20T11:24:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c2fbb3740 20 client.26069 trim_cache size 0 max 16384 2024-04-20T11:24:59.744 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:24:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c0c0a3da0 msgr2=0x7f2c0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c0c0a3da0 0x7f2c0c0a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x558689e4bc40 tx=0x7f2c180d4650 comp rx=0 tx=0).stop 2024-04-20T11:24:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 msgr2=0x7f2c2819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 0x7f2c2819ed80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800e9f0 tx=0x7f2c1800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 shutdown_connections 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c281061b0 0x7f2c281a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2c0c0a3da0 0x7f2c0c0a6260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c281016d0 0x7f2c2819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 --2- 172.21.15.44:0/1404155998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c28100db0 0x7f2c2819ed80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.739+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 >> 172.21.15.44:0/1404155998 conn(0x7f2c280fc7e0 msgr2=0x7f2c280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.740+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 shutdown_connections 2024-04-20T11:24:59.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:24:59.740+0000 7f2c27200640 1 -- 172.21.15.44:0/1404155998 wait complete. 2024-04-20T11:24:59.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:24:59 smithi078 ceph-mon[21464]: pgmap v74: 97 pgs: 97 active+clean; 121 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-04-20T11:24:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:24:59 smithi165 ceph-mon[21975]: pgmap v74: 97 pgs: 97 active+clean; 121 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-04-20T11:25:00.757 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:25:00.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:25:00.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 -- 172.21.15.44:0/2631894536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e01021a0 msgr2=0x7f03e0102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 --2- 172.21.15.44:0/2631894536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e01021a0 0x7f03e0102580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009900 tx=0x7f03d402bc40 comp rx=0 tx=0).stop 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 -- 172.21.15.44:0/2631894536 shutdown_connections 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 --2- 172.21.15.44:0/2631894536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0107270 0x7f03e010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 --2- 172.21.15.44:0/2631894536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03e0102b50 0x7f03e0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 --2- 172.21.15.44:0/2631894536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e01021a0 0x7f03e0102580 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 -- 172.21.15.44:0/2631894536 >> 172.21.15.44:0/2631894536 conn(0x7f03e00793b0 msgr2=0x7f03e00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 -- 172.21.15.44:0/2631894536 shutdown_connections 2024-04-20T11:25:00.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.964+0000 7f03e8600640 1 -- 172.21.15.44:0/2631894536 wait complete. 2024-04-20T11:25:00.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.965+0000 7f03e8600640 1 Processor -- start 2024-04-20T11:25:00.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.965+0000 7f03e8600640 1 -- start start 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e8600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03e01021a0 0x7f03e01a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e8600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 0x7f03e01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e8600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e019a9a0 0x7f03e019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e8600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03e0112460 con 0x7f03e019a9a0 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e8600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01122e0 con 0x7f03e0107270 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e8600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01125e0 con 0x7f03e01021a0 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e5200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 0x7f03e01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e5200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 0x7f03e01a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39776/0 (socket says 172.21.15.44:39776) 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e5200640 1 -- 172.21.15.44:0/743924493 learned_addr learned my addr 172.21.15.44:0/743924493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e5c00640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03e01021a0 0x7f03e01a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.966+0000 7f03e6600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e019a9a0 0x7f03e019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e5200640 1 -- 172.21.15.44:0/743924493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03e01021a0 msgr2=0x7f03e01a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e5200640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03e01021a0 0x7f03e01a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e5200640 1 -- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e019a9a0 msgr2=0x7f03e019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e5200640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e019a9a0 0x7f03e019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e5200640 1 -- 172.21.15.44:0/743924493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e019b4c0 con 0x7f03e0107270 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e6600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e019a9a0 0x7f03e019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e5200640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 0x7f03e01a0d30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9f0 tx=0x7f03cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03de000640 1 -- 172.21.15.44:0/743924493 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00ccc0 con 0x7f03e0107270 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03de000640 1 -- 172.21.15.44:0/743924493 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f03cc005520 con 0x7f03e0107270 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e019b7b0 con 0x7f03e0107270 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03de000640 1 -- 172.21.15.44:0/743924493 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc010690 con 0x7f03e0107270 2024-04-20T11:25:00.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.967+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a7a00 con 0x7f03e0107270 2024-04-20T11:25:00.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.970+0000 7f03de000640 1 -- 172.21.15.44:0/743924493 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f03cc004f70 con 0x7f03e0107270 2024-04-20T11:25:00.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.971+0000 7f03eb217740 1 Processor -- start 2024-04-20T11:25:00.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.971+0000 7f03eb217740 1 -- start start 2024-04-20T11:25:00.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.971+0000 7f03eb217740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 0x561d418d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.971+0000 7f03eb217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d5830 0x561d418dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03eb217740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418de410 0x561d418e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03eb217740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561d41839d20 con 0x561d418de410 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03eb217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d417e03e0 con 0x561d418d5830 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03eb217740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561d41835ce0 con 0x561d418d4f10 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 0x561d418d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d5830 0x561d418dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 0x561d418d52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40202/0 (socket says 172.21.15.44:40202) 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 -- 172.21.15.44:0/925888034 learned_addr learned my addr 172.21.15.44:0/925888034 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e5200640 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418de410 0x561d418e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 -- 172.21.15.44:0/925888034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d5830 msgr2=0x561d418dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d5830 0x561d418dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 -- 172.21.15.44:0/925888034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418de410 msgr2=0x561d418e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:00.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418de410 0x561d418e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03e6600640 1 -- 172.21.15.44:0/925888034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d417ff980 con 0x561d418d4f10 2024-04-20T11:25:00.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.972+0000 7f03de000640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f03c80a3da0 0x7f03c80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.973+0000 7f03e6600640 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 0x561d418d52f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009fd0 tx=0x7f03d8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a804132da7008d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.973+0000 7f03de000640 1 -- 172.21.15.44:0/743924493 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f03cc0d0b60 con 0x7f03e0107270 2024-04-20T11:25:00.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.973+0000 7f03e5c00640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f03c80a3da0 0x7f03c80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.973+0000 7f03d2a00640 1 -- 172.21.15.44:0/925888034 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8011700 con 0x561d418d4f10 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.973+0000 7f03d2a00640 1 -- 172.21.15.44:0/925888034 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f03d8011da0 con 0x561d418d4f10 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.973+0000 7f03d2a00640 1 -- 172.21.15.44:0/925888034 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d800e3c0 con 0x561d418d4f10 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03e5c00640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f03c80a3da0 0x7f03c80a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f03d4005ab0 tx=0x7f03d4005a20 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 -- 172.21.15.44:0/925888034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 msgr2=0x561d418d52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 0x561d418d52f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009fd0 tx=0x7f03d8010860 comp rx=0 tx=0).stop 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 -- 172.21.15.44:0/925888034 shutdown_connections 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418de410 0x561d418e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d5830 0x561d418dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 --2- 172.21.15.44:0/925888034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d4f10 0x561d418d52f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 -- 172.21.15.44:0/925888034 >> 172.21.15.44:0/925888034 conn(0x561d418cf640 msgr2=0x561d418cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 -- 172.21.15.44:0/925888034 shutdown_connections 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.974+0000 7f03eb217740 1 -- 172.21.15.44:0/925888034 wait complete. 2024-04-20T11:25:00.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.975+0000 7f03eb217740 1 -- start start 2024-04-20T11:25:00.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.975+0000 7f03eb217740 1 Processor -- start 2024-04-20T11:25:00.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d4f10 0x561d418db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d5830 0x561d418dba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 0x561d418dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561d418e1200 con 0x561d418d4f10 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d417f37a0 con 0x561d418de410 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561d418e1380 con 0x561d418d5830 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03eb217740 10 client.0 _ll_register_callbacks cb 0x561d417f5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e6600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 0x561d418dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d5830 0x561d418dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d4f10 0x561d418db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e6600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 0x561d418dbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39790/0 (socket says 172.21.15.44:39790) 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e6600640 1 -- 172.21.15.44:0/780241846 learned_addr learned my addr 172.21.15.44:0/780241846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:00.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5200640 1 -- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 msgr2=0x561d418dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5200640 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 0x561d418dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5200640 1 -- 172.21.15.44:0/780241846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d4f10 msgr2=0x561d418db4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5200640 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d4f10 0x561d418db4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e5200640 1 -- 172.21.15.44:0/780241846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d416147e0 con 0x561d418d5830 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.976+0000 7f03e6600640 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 0x561d418dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03e5200640 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d5830 0x561d418dba20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0ce2c0 tx=0x7f03cc098ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b14438092679841 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03c7400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03c7400640 1 -- 172.21.15.44:0/780241846 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00ce60 con 0x561d418d5830 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03c7400640 1 -- 172.21.15.44:0/780241846 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f03cc005260 con 0x561d418d5830 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d8011da0 con 0x561d418d5830 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03c7400640 1 -- 172.21.15.44:0/780241846 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f03cc09d210 con 0x561d418d5830 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03eb217740 10 client.26095 fetch_fsmap learned FSMap version 31 2024-04-20T11:25:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.977+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d41674f70 con 0x561d418d5830 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03c7400640 1 -- 172.21.15.44:0/780241846 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f03d8011da0 con 0x561d418d5830 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03c7400640 1 -- 172.21.15.44:0/780241846 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f03cc0ce540 con 0x561d418d5830 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03eb217740 10 client.26095 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03eb217740 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03eb217740 20 client.26095 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03eb217740 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d418ee8c0 0x561d418f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03eb217740 4 client.26095 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d41847740 con 0x561d418ee8c0 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.978+0000 7f03e5c00640 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d418ee8c0 0x561d418f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.979+0000 7f03e5c00640 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d418ee8c0 0x561d418f0ce0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d76f73ee4e91516 server_cookie=3d4d413e40b899bf in_seq=0 out_seq=0 2024-04-20T11:25:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.979+0000 7f03c7400640 10 client.26095 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:00.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03c7400640 1 -- 172.21.15.44:0/780241846 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d41847740 con 0x561d418ee8c0 2024-04-20T11:25:00.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03c7400640 10 client.26095 handle_command_reply: tid=0 2024-04-20T11:25:00.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 client.26095 shutdown 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d5830 msgr2=0x561d418dba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d5830 0x561d418dba20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0ce2c0 tx=0x7f03cc098ee0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 shutdown_connections 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d418ee8c0 0x561d418f0ce0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418de410 0x561d418dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418d5830 0x561d418dba20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 --2- 172.21.15.44:0/780241846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d4f10 0x561d418db4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 >> 172.21.15.44:0/780241846 conn(0x561d418cf640 msgr2=0x561d418d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03c7400640 0 client.26095 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 shutdown_connections 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 1 -- 172.21.15.44:0/780241846 wait complete. 2024-04-20T11:25:00.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:00.980+0000 7f03eb217740 20 client.26095 trim_cache size 0 max 16384 2024-04-20T11:25:01.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 Processor -- start 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 -- start start 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d418ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01aff70 con 0x561d418d5830 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b1b70 con 0x561d418d4f10 2024-04-20T11:25:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.000+0000 7f03eb217740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b0640 con 0x561d418de410 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418ddc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39794/0 (socket says 172.21.15.44:39794) 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 -- 172.21.15.44:0/306993501 learned_addr learned my addr 172.21.15.44:0/306993501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5200640 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e6600640 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d418ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 -- 172.21.15.44:0/306993501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 msgr2=0x561d418ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d418ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 -- 172.21.15.44:0/306993501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 msgr2=0x561d418ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 -- 172.21.15.44:0/306993501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4150c170 con 0x561d418d4f10 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.001+0000 7f03e5c00640 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418ddc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f03d402dd00 tx=0x7f03d4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176dd6a17a9508de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03d2000640 1 -- 172.21.15.44:0/306993501 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4057070 con 0x561d418d4f10 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03d2000640 1 -- 172.21.15.44:0/306993501 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f03d4046040 con 0x561d418d4f10 2024-04-20T11:25:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03d2000640 1 -- 172.21.15.44:0/306993501 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4051070 con 0x561d418d4f10 2024-04-20T11:25:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03eb217740 1 -- 172.21.15.44:0/306993501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 msgr2=0x561d418ddc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03eb217740 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418ddc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f03d402dd00 tx=0x7f03d4040000 comp rx=0 tx=0).stop 2024-04-20T11:25:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03eb217740 1 -- 172.21.15.44:0/306993501 shutdown_connections 2024-04-20T11:25:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03eb217740 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d418ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.002+0000 7f03eb217740 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.003+0000 7f03eb217740 1 --2- 172.21.15.44:0/306993501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418ddc10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.003+0000 7f03eb217740 1 -- 172.21.15.44:0/306993501 >> 172.21.15.44:0/306993501 conn(0x561d418cf640 msgr2=0x561d418cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.003+0000 7f03eb217740 1 -- 172.21.15.44:0/306993501 shutdown_connections 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.003+0000 7f03eb217740 1 -- 172.21.15.44:0/306993501 wait complete. 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.003+0000 7f03eb217740 1 -- start start 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.003+0000 7f03eb217740 1 Processor -- start 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d4194de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b13e0 con 0x561d418d5830 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01b0d10 con 0x561d418d4f10 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01adeb0 con 0x561d418de410 2024-04-20T11:25:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03eb217740 10 client.0 _ll_register_callbacks cb 0x7f03cc004b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418e2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39802/0 (socket says 172.21.15.44:39802) 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 -- 172.21.15.44:0/3176731625 learned_addr learned my addr 172.21.15.44:0/3176731625 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5200640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e6600640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d4194de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 -- 172.21.15.44:0/3176731625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 msgr2=0x561d4194de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d4194de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 -- 172.21.15.44:0/3176731625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 msgr2=0x561d418e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5c00640 1 -- 172.21.15.44:0/3176731625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4194e400 con 0x561d418d4f10 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e6600640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d4194de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.004+0000 7f03e5200640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03e5c00640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418e2880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006010 tx=0x7f03d4004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271239c81ed6b730 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03c7400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03c7400640 1 -- 172.21.15.44:0/3176731625 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4057070 con 0x561d418d4f10 2024-04-20T11:25:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03c7400640 1 -- 172.21.15.44:0/3176731625 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f03d4005040 con 0x561d418d4f10 2024-04-20T11:25:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d4046040 con 0x561d418d4f10 2024-04-20T11:25:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03c7400640 1 -- 172.21.15.44:0/3176731625 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f03d4005240 con 0x561d418d4f10 2024-04-20T11:25:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03eb217740 10 client.26087 fetch_fsmap learned FSMap version 31 2024-04-20T11:25:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.005+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d419557c0 con 0x561d418d4f10 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03c7400640 1 -- 172.21.15.44:0/3176731625 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f03d4046040 con 0x561d418d4f10 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03c7400640 1 -- 172.21.15.44:0/3176731625 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f03d404b040 con 0x561d418d4f10 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03eb217740 10 client.26087 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03eb217740 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03eb217740 20 client.26087 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03eb217740 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d41955f90 0x561d419583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03eb217740 4 client.26087 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d418415b0 con 0x561d41955f90 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03e5200640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d41955f90 0x561d419583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03c7400640 10 client.26087 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.006+0000 7f03e5200640 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d41955f90 0x561d419583b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d152fb21fad1bf server_cookie=10960fe19584623 in_seq=0 out_seq=0 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03c7400640 1 -- 172.21.15.44:0/3176731625 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d418415b0 con 0x561d41955f90 2024-04-20T11:25:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03c7400640 10 client.26087 handle_command_reply: tid=0 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 client.26087 shutdown 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 msgr2=0x561d418e2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418e2880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006010 tx=0x7f03d4004610 comp rx=0 tx=0).stop 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 shutdown_connections 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561d41955f90 0x561d419583b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d418de410 0x561d4194de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d418d5830 0x561d418e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 --2- 172.21.15.44:0/3176731625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d418d4f10 0x561d418e2880 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03c7400640 0 client.26087 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.008+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 >> 172.21.15.44:0/3176731625 conn(0x561d418cf640 msgr2=0x561d418ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 shutdown_connections 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03eb217740 1 -- 172.21.15.44:0/3176731625 wait complete. 2024-04-20T11:25:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03eb217740 20 client.26087 trim_cache size 0 max 16384 2024-04-20T11:25:01.012 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:25:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:25:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "d92dae26-04f9-4e44-8720-720415ee9845", 2024-04-20T11:25:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:25:01.012 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f03c80a3da0 msgr2=0x7f03c80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03e8600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f03c80a3da0 0x7f03c80a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f03d4005ab0 tx=0x7f03d4005a20 comp rx=0 tx=0).stop 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 msgr2=0x7f03e01a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.009+0000 7f03e8600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 0x7f03e01a0d30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9f0 tx=0x7f03cc00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 shutdown_connections 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f03c80a3da0 0x7f03c80a6260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e019a9a0 0x7f03e019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03e0107270 0x7f03e01a0d30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 --2- 172.21.15.44:0/743924493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03e01021a0 0x7f03e01a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 >> 172.21.15.44:0/743924493 conn(0x7f03e00793b0 msgr2=0x7f03e01000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 shutdown_connections 2024-04-20T11:25:01.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.010+0000 7f03e8600640 1 -- 172.21.15.44:0/743924493 wait complete. 2024-04-20T11:25:01.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:25:01.033 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:25:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 -- 172.21.15.44:0/3579072449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc4104410 msgr2=0x7f4bc41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3579072449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc4104410 0x7f4bc41047f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009900 tx=0x7f4bb802bc40 comp rx=0 tx=0).stop 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 -- 172.21.15.44:0/3579072449 shutdown_connections 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3579072449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bc41094e0 0x7f4bc41110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3579072449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bc4104dc0 0x7f4bc4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3579072449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc4104410 0x7f4bc41047f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.321+0000 7f4bcb600640 1 -- 172.21.15.44:0/3579072449 >> 172.21.15.44:0/3579072449 conn(0x7f4bc40ffcf0 msgr2=0x7f4bc4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.322+0000 7f4bcb600640 1 -- 172.21.15.44:0/3579072449 shutdown_connections 2024-04-20T11:25:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.322+0000 7f4bcb600640 1 -- 172.21.15.44:0/3579072449 wait complete. 2024-04-20T11:25:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.323+0000 7f4bcb600640 1 Processor -- start 2024-04-20T11:25:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.323+0000 7f4bcb600640 1 -- start start 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bcb600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bc4104410 0x7f4bc41a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bcb600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 0x7f4bc41a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bcb600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bc419a9d0 0x7f4bc419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bcb600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41146d0 con 0x7f4bc4104410 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bcb600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4114550 con 0x7f4bc41094e0 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bcb600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4114850 con 0x7f4bc419a9d0 2024-04-20T11:25:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 0x7f4bc41a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bc9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bc419a9d0 0x7f4bc419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 0x7f4bc41a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39816/0 (socket says 172.21.15.44:39816) 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3335976700 learned_addr learned my addr 172.21.15.44:0/3335976700 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.324+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bc4104410 0x7f4bc41a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3335976700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bc419a9d0 msgr2=0x7f4bc419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bc419a9d0 0x7f4bc419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bc4104410 msgr2=0x7f4bc41a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bc4104410 0x7f4bc41a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3335976700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc419b4f0 con 0x7f4bc41094e0 2024-04-20T11:25:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 0x7f4bc41a0d60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e9c0 tx=0x7f4bb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc1600640 1 -- 172.21.15.44:0/3335976700 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000cd80 con 0x7f4bc41094e0 2024-04-20T11:25:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc1600640 1 -- 172.21.15.44:0/3335976700 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4bb0005520 con 0x7f4bc41094e0 2024-04-20T11:25:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bc1600640 1 -- 172.21.15.44:0/3335976700 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0010690 con 0x7f4bc41094e0 2024-04-20T11:25:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.325+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc419b7e0 con 0x7f4bc41094e0 2024-04-20T11:25:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.326+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc41a79e0 con 0x7f4bc41094e0 2024-04-20T11:25:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.329+0000 7f4bce381740 1 Processor -- start 2024-04-20T11:25:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bc1600640 1 -- 172.21.15.44:0/3335976700 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4bb00046c0 con 0x7f4bc41094e0 2024-04-20T11:25:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 -- start start 2024-04-20T11:25:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 0x55784e9dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55784e938150 con 0x55784e9db6a0 2024-04-20T11:25:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55784e8de370 con 0x55784e9d2ac0 2024-04-20T11:25:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.330+0000 7f4bce381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55784e934110 con 0x55784e9d21a0 2024-04-20T11:25:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.331+0000 7f4bc9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.331+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 0x55784e9dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.331+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 0x55784e9dda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60784/0 (socket says 172.21.15.44:60784) 2024-04-20T11:25:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.331+0000 7f4bc3e00640 1 -- 172.21.15.44:0/843659766 learned_addr learned my addr 172.21.15.44:0/843659766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.331+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.331+0000 7f4bc9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40256/0 (socket says 172.21.15.44:40256) 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc3e00640 1 -- 172.21.15.44:0/843659766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 msgr2=0x55784e9d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc3e00640 1 -- 172.21.15.44:0/843659766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 msgr2=0x55784e9db160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9db160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc3e00640 1 -- 172.21.15.44:0/843659766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784e7147d0 con 0x55784e9db6a0 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 0x55784e9dda90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00ce1a0 tx=0x7f4bb00d4ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7757c384e86d561b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.332+0000 7f4bc1600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4bac0a3da0 0x7f4bac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.333+0000 7f4bc1600640 1 -- 172.21.15.44:0/3335976700 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4bb00d0b00 con 0x7f4bc41094e0 2024-04-20T11:25:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.333+0000 7f4bb5c00640 1 -- 172.21.15.44:0/843659766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00d8040 con 0x55784e9db6a0 2024-04-20T11:25:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.333+0000 7f4bb5c00640 1 -- 172.21.15.44:0/843659766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4bb00e2070 con 0x55784e9db6a0 2024-04-20T11:25:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.333+0000 7f4bb5c00640 1 -- 172.21.15.44:0/843659766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb009c5f0 con 0x55784e9db6a0 2024-04-20T11:25:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.333+0000 7f4bc9600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4bac0a3da0 0x7f4bac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 -- 172.21.15.44:0/843659766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 msgr2=0x55784e9dda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 0x55784e9dda90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00ce1a0 tx=0x7f4bb00d4ba0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bc9600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4bac0a3da0 0x7f4bac0a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005fd0 tx=0x7f4bbc009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 -- 172.21.15.44:0/843659766 shutdown_connections 2024-04-20T11:25:01.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9db6a0 0x55784e9dda90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 --2- 172.21.15.44:0/843659766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.334+0000 7f4bce381740 1 -- 172.21.15.44:0/843659766 >> 172.21.15.44:0/843659766 conn(0x55784e9cc8d0 msgr2=0x55784e9cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.335+0000 7f4bce381740 1 -- 172.21.15.44:0/843659766 shutdown_connections 2024-04-20T11:25:01.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.335+0000 7f4bce381740 1 -- 172.21.15.44:0/843659766 wait complete. 2024-04-20T11:25:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.335+0000 7f4bce381740 1 -- start start 2024-04-20T11:25:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 Processor -- start 2024-04-20T11:25:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 0x55784e9d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9d9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55784e9de430 con 0x55784e9d2ac0 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55784e8f1740 con 0x55784e9db6a0 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55784e9de5b0 con 0x55784e9d21a0 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.336+0000 7f4bce381740 10 client.0 _ll_register_callbacks cb 0x55784e8ecb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 0x55784e9d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 0x55784e9d8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60800/0 (socket says 172.21.15.44:60800) 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3545524942 learned_addr learned my addr 172.21.15.44:0/3545524942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc9600640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3545524942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 msgr2=0x55784e9d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 msgr2=0x55784e9d9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9d9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc3e00640 1 -- 172.21.15.44:0/3545524942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784e64a3f0 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.337+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.338+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 0x55784e9d8d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00a51f0 tx=0x7f4bb0004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2dc79f9c65637e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.338+0000 7f4baa000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.338+0000 7f4baa000640 1 -- 172.21.15.44:0/3545524942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb009c410 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.338+0000 7f4baa000640 1 -- 172.21.15.44:0/3545524942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4bb009adb0 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.338+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb00e2070 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4baa000640 1 -- 172.21.15.44:0/3545524942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4bb009c5f0 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 10 client.15978 fetch_fsmap learned FSMap version 31 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55784e60a0f0 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4baa000640 1 -- 172.21.15.44:0/3545524942 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4bb00e2070 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4baa000640 1 -- 172.21.15.44:0/3545524942 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f4bb00ce4e0 con 0x55784e9d2ac0 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 10 client.15978 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:25:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 20 client.15978 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784e9e2820 0x55784e9e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 4 client.15978 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.339+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55784e945b70 con 0x55784e9e2820 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.340+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784e9e2820 0x55784e9e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.340+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784e9e2820 0x55784e9e4c60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e127ef1284ea6cf server_cookie=74d43b44dfaf3107 in_seq=0 out_seq=0 2024-04-20T11:25:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.340+0000 7f4baa000640 10 client.15978 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.343+0000 7f4baa000640 1 -- 172.21.15.44:0/3545524942 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55784e945b70 con 0x55784e9e2820 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.343+0000 7f4baa000640 10 client.15978 handle_command_reply: tid=0 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.343+0000 7f4bce381740 1 client.15978 shutdown 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 msgr2=0x55784e9d8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 0x55784e9d8d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00a51f0 tx=0x7f4bb0004aa0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 shutdown_connections 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784e9e2820 0x55784e9e4c60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4baa000640 0 client.15978 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d2ac0 0x55784e9d8d40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 --2- 172.21.15.44:0/3545524942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d21a0 0x55784e9d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 >> 172.21.15.44:0/3545524942 conn(0x55784e9cc8d0 msgr2=0x55784e9ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 shutdown_connections 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 1 -- 172.21.15.44:0/3545524942 wait complete. 2024-04-20T11:25:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.344+0000 7f4bce381740 20 client.15978 trim_cache size 0 max 16384 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 Processor -- start 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 -- start start 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 0x55784e9cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9ef750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41abaf0 con 0x55784e9d21a0 2024-04-20T11:25:01.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41ad6f0 con 0x55784e9db6a0 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bce381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41ac1c0 con 0x55784e9d2ac0 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bc8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 0x55784e9cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 0x55784e9cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40264/0 (socket says 172.21.15.44:40264) 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bc3e00640 1 -- 172.21.15.44:0/2374244865 learned_addr learned my addr 172.21.15.44:0/2374244865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bc8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9d90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60814/0 (socket says 172.21.15.44:60814) 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.367+0000 7f4bc9600640 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bc3e00640 1 -- 172.21.15.44:0/2374244865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 msgr2=0x55784e9ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bc3e00640 1 -- 172.21.15.44:0/2374244865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 msgr2=0x55784e9d90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9d90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bc3e00640 1 -- 172.21.15.44:0/2374244865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784e9c6d10 con 0x55784e9d2ac0 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 0x55784e9cb8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4bb009c7b0 tx=0x7f4bb0004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d921d8511589b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.368+0000 7f4bb5200640 1 -- 172.21.15.44:0/2374244865 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb009c410 con 0x55784e9d2ac0 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.369+0000 7f4bb5200640 1 -- 172.21.15.44:0/2374244865 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4bb00e2030 con 0x55784e9d2ac0 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.369+0000 7f4bb5200640 1 -- 172.21.15.44:0/2374244865 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0098d50 con 0x55784e9d2ac0 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 -- 172.21.15.44:0/2374244865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 msgr2=0x55784e9cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 0x55784e9cb8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4bb009c7b0 tx=0x7f4bb0004b10 comp rx=0 tx=0).stop 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 -- 172.21.15.44:0/2374244865 shutdown_connections 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9db6a0 0x55784e9ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9d2ac0 0x55784e9cb8c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 --2- 172.21.15.44:0/2374244865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 -- 172.21.15.44:0/2374244865 >> 172.21.15.44:0/2374244865 conn(0x55784e9cc8d0 msgr2=0x55784e9d7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 -- 172.21.15.44:0/2374244865 shutdown_connections 2024-04-20T11:25:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 -- 172.21.15.44:0/2374244865 wait complete. 2024-04-20T11:25:01.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.370+0000 7f4bce381740 1 -- start start 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 Processor -- start 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9db6a0 0x55784e9d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41acf60 con 0x55784e9d21a0 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41ac890 con 0x55784e9d2ac0 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41a9a50 con 0x55784e9db6a0 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.371+0000 7f4bce381740 10 client.0 _ll_register_callbacks cb 0x7f4bbc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9e7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60822/0 (socket says 172.21.15.44:60822) 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc8c00640 1 -- 172.21.15.44:0/1008190395 learned_addr learned my addr 172.21.15.44:0/1008190395 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc9600640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9db6a0 0x55784e9d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 -- 172.21.15.44:0/1008190395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9db6a0 msgr2=0x55784e9d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9db6a0 0x55784e9d56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 -- 172.21.15.44:0/1008190395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 msgr2=0x55784e9e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9e7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 -- 172.21.15.44:0/1008190395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784e9d5e10 con 0x55784e9d2ac0 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4bc3e00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9d5190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55784e9d9cd0 tx=0x7f4bb0004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769fb82c056b82a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:01.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.372+0000 7f4baa000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.373+0000 7f4baa000640 1 -- 172.21.15.44:0/1008190395 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb009c880 con 0x55784e9d2ac0 2024-04-20T11:25:01.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.373+0000 7f4baa000640 1 -- 172.21.15.44:0/1008190395 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4bb00a5290 con 0x55784e9d2ac0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.373+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb00e2030 con 0x55784e9d2ac0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.373+0000 7f4baa000640 1 -- 172.21.15.44:0/1008190395 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4bb00a5770 con 0x55784e9d2ac0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.373+0000 7f4bce381740 10 client.26105 fetch_fsmap learned FSMap version 31 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.373+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55784e9d5e10 con 0x55784e9d2ac0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4baa000640 1 -- 172.21.15.44:0/1008190395 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4bb00e2030 con 0x55784e9d2ac0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4baa000640 1 -- 172.21.15.44:0/1008190395 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8387+0+0 (secure 0 0 0) 0x7f4bb009bf00 con 0x55784e9d2ac0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bce381740 10 client.26105 fetch_fsmap finished waiting for FSMap version 31 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bce381740 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bce381740 20 client.26105 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bce381740 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784ea62dc0 0x55784ea651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bce381740 4 client.26105 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55784e93f9e0 con 0x55784ea62dc0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.374+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784ea62dc0 0x55784ea651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:01.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.375+0000 7f4bc8c00640 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784ea62dc0 0x55784ea651e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44936d0f43a87b7 server_cookie=a81d9b3cbf018e9f in_seq=0 out_seq=0 2024-04-20T11:25:01.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.375+0000 7f4baa000640 10 client.26105 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4baa000640 1 -- 172.21.15.44:0/1008190395 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55784e93f9e0 con 0x55784ea62dc0 2024-04-20T11:25:01.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4baa000640 10 client.26105 handle_command_reply: tid=0 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 client.26105 shutdown 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 msgr2=0x55784e9d5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9d5190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55784e9d9cd0 tx=0x7f4bb0004aa0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 shutdown_connections 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55784ea62dc0 0x55784ea651e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55784e9db6a0 0x55784e9d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55784e9d2ac0 0x55784e9d5190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 --2- 172.21.15.44:0/1008190395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55784e9d21a0 0x55784e9e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4baa000640 0 client.26105 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 >> 172.21.15.44:0/1008190395 conn(0x55784e9cc8d0 msgr2=0x55784e9cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 shutdown_connections 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.383+0000 7f4bce381740 1 -- 172.21.15.44:0/1008190395 wait complete. 2024-04-20T11:25:01.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bce381740 20 client.26105 trim_cache size 0 max 16384 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (383 inodes in the stack)", 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: "d92dae26-04f9-4e44-8720-720415ee9845": { 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "d92dae26-04f9-4e44-8720-720415ee9845", 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:25:01.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:25:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:25:01.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4bac0a3da0 msgr2=0x7f4bac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4bac0a3da0 0x7f4bac0a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc005fd0 tx=0x7f4bbc009500 comp rx=0 tx=0).stop 2024-04-20T11:25:01.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 msgr2=0x7f4bc41a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:01.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 0x7f4bc41a0d60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e9c0 tx=0x7f4bb000ee80 comp rx=0 tx=0).stop 2024-04-20T11:25:01.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 shutdown_connections 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bc419a9d0 0x7f4bc419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc41094e0 0x7f4bc41a0d60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4bac0a3da0 0x7f4bac0a6260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 --2- 172.21.15.44:0/3335976700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bc4104410 0x7f4bc41a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.384+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 >> 172.21.15.44:0/3335976700 conn(0x7f4bc40ffcf0 msgr2=0x7f4bc4101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.385+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 shutdown_connections 2024-04-20T11:25:01.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:01.385+0000 7f4bcb600640 1 -- 172.21.15.44:0/3335976700 wait complete. 2024-04-20T11:25:01.405 INFO:tasks.cephfs.filesystem:scrub status for tag:d92dae26-04f9-4e44-8720-720415ee9845 - {'path': '/', 'tag': 'd92dae26-04f9-4e44-8720-720415ee9845', 'options': 'recursive,force'} 2024-04-20T11:25:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:01 smithi044 ceph-mon[16855]: pgmap v75: 97 pgs: 97 active+clean; 121 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-04-20T11:25:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:01 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:25:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:01 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:01 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:01 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:01 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:01 smithi078 ceph-mon[21464]: pgmap v75: 97 pgs: 97 active+clean; 121 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-04-20T11:25:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:01 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:25:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:01 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:01 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:01 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:01 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:01 smithi165 ceph-mon[21975]: pgmap v75: 97 pgs: 97 active+clean; 121 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-04-20T11:25:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:01 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:25:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:01 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:01 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:01 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:01 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:02.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: pgmap v76: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 720 op/s 2024-04-20T11:25:03.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.882 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:03 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: pgmap v76: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 720 op/s 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:03.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:03 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: pgmap v76: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 720 op/s 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:03.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:03 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:05 smithi078 ceph-mon[21464]: pgmap v77: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-04-20T11:25:05.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:05 smithi044 ceph-mon[16855]: pgmap v77: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-04-20T11:25:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:05 smithi165 ceph-mon[21975]: pgmap v77: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-04-20T11:25:06.499 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2024-04-20T11:25:07.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:07 smithi078 ceph-mon[21464]: pgmap v78: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-04-20T11:25:07.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:07 smithi044 ceph-mon[16855]: pgmap v78: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-04-20T11:25:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:07 smithi165 ceph-mon[21975]: pgmap v78: 97 pgs: 97 active+clean; 165 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-04-20T11:25:08.580 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar c linux-4.0.5 2024-04-20T11:25:09.317 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm -rf linux-4.0.5 2024-04-20T11:25:09.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:09 smithi078 ceph-mon[21464]: pgmap v79: 97 pgs: 97 active+clean; 190 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.2 MiB/s wr, 966 op/s 2024-04-20T11:25:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:09 smithi044 ceph-mon[16855]: pgmap v79: 97 pgs: 97 active+clean; 190 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.2 MiB/s wr, 966 op/s 2024-04-20T11:25:09.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:09 smithi165 ceph-mon[21975]: pgmap v79: 97 pgs: 97 active+clean; 190 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.2 MiB/s wr, 966 op/s 2024-04-20T11:25:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:11 smithi078 ceph-mon[21464]: pgmap v80: 97 pgs: 97 active+clean; 190 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 854 op/s 2024-04-20T11:25:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:11 smithi044 ceph-mon[16855]: pgmap v80: 97 pgs: 97 active+clean; 190 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 854 op/s 2024-04-20T11:25:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:11 smithi165 ceph-mon[21975]: pgmap v80: 97 pgs: 97 active+clean; 190 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 854 op/s 2024-04-20T11:25:12.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:12 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:12.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:12 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:12.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:12 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:12.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:12 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:12.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:12 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:12.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:12 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:13 smithi078 ceph-mon[21464]: pgmap v81: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 950 op/s 2024-04-20T11:25:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:13 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:13 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:13 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:13 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:13.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:13 smithi044 ceph-mon[16855]: pgmap v81: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 950 op/s 2024-04-20T11:25:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:13 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:13 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:13 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:13 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:13 smithi165 ceph-mon[21975]: pgmap v81: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 950 op/s 2024-04-20T11:25:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:13 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:13 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:13 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:13 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:15 smithi078 ceph-mon[21464]: pgmap v82: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-04-20T11:25:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:15 smithi044 ceph-mon[16855]: pgmap v82: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-04-20T11:25:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:15 smithi165 ceph-mon[21975]: pgmap v82: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-04-20T11:25:16.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:16 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:16.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:16 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:16 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:17 smithi078 ceph-mon[21464]: pgmap v83: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-04-20T11:25:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:17 smithi044 ceph-mon[16855]: pgmap v83: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-04-20T11:25:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:17 smithi165 ceph-mon[21975]: pgmap v83: 97 pgs: 97 active+clean; 203 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-04-20T11:25:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:19 smithi078 ceph-mon[21464]: pgmap v84: 97 pgs: 97 active+clean; 212 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 500 op/s 2024-04-20T11:25:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:19 smithi044 ceph-mon[16855]: pgmap v84: 97 pgs: 97 active+clean; 212 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 500 op/s 2024-04-20T11:25:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:19 smithi165 ceph-mon[21975]: pgmap v84: 97 pgs: 97 active+clean; 212 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 500 op/s 2024-04-20T11:25:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:21 smithi078 ceph-mon[21464]: pgmap v85: 97 pgs: 97 active+clean; 212 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-20T11:25:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:21 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:21 smithi044 ceph-mon[16855]: pgmap v85: 97 pgs: 97 active+clean; 212 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-20T11:25:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:21 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:21 smithi165 ceph-mon[21975]: pgmap v85: 97 pgs: 97 active+clean; 212 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-20T11:25:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:21 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:23.930 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar xf doc.tar 2024-04-20T11:25:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:23 smithi078 ceph-mon[21464]: pgmap v86: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-04-20T11:25:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:23 smithi044 ceph-mon[16855]: pgmap v86: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-04-20T11:25:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:23 smithi165 ceph-mon[21975]: pgmap v86: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-04-20T11:25:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:25 smithi078 ceph-mon[21464]: pgmap v87: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-20T11:25:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:25 smithi044 ceph-mon[16855]: pgmap v87: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-20T11:25:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:25 smithi165 ceph-mon[21975]: pgmap v87: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-20T11:25:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:27 smithi078 ceph-mon[21464]: pgmap v88: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-20T11:25:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:27 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:27 smithi044 ceph-mon[16855]: pgmap v88: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-20T11:25:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:27 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:27 smithi165 ceph-mon[21975]: pgmap v88: 97 pgs: 97 active+clean; 214 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-20T11:25:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:27 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:29 smithi078 ceph-mon[21464]: pgmap v89: 97 pgs: 97 active+clean; 216 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 352 op/s 2024-04-20T11:25:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:29 smithi044 ceph-mon[16855]: pgmap v89: 97 pgs: 97 active+clean; 216 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 352 op/s 2024-04-20T11:25:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:29 smithi165 ceph-mon[21975]: pgmap v89: 97 pgs: 97 active+clean; 216 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 352 op/s 2024-04-20T11:25:31.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:25:31.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:25:31.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.632+0000 7f8b97200640 1 -- 172.21.15.44:0/2343014808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98108500 msgr2=0x7f8b98108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.632+0000 7f8b97200640 1 --2- 172.21.15.44:0/2343014808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98108500 0x7f8b98108960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009900 tx=0x7f8b8402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 -- 172.21.15.44:0/2343014808 shutdown_connections 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 --2- 172.21.15.44:0/2343014808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 0x7f8b981152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 --2- 172.21.15.44:0/2343014808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98108500 0x7f8b98108960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 --2- 172.21.15.44:0/2343014808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9810d3d0 0x7f8b9810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 -- 172.21.15.44:0/2343014808 >> 172.21.15.44:0/2343014808 conn(0x7f8b98077d80 msgr2=0x7f8b98106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 -- 172.21.15.44:0/2343014808 shutdown_connections 2024-04-20T11:25:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.633+0000 7f8b97200640 1 -- 172.21.15.44:0/2343014808 wait complete. 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 Processor -- start 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 -- start start 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b98108500 0x7f8b9819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 0x7f8b9819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 0x7f8b981a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98118950 con 0x7f8b98108ea0 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981187d0 con 0x7f8b9810d3d0 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.634+0000 7f8b97200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98118ad0 con 0x7f8b98108500 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b95400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 0x7f8b9819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 0x7f8b981a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 0x7f8b981a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35510/0 (socket says 172.21.15.44:35510) 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 -- 172.21.15.44:0/1277571045 learned_addr learned my addr 172.21.15.44:0/1277571045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b95400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 0x7f8b9819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33132/0 (socket says 172.21.15.44:33132) 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b95e00640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b98108500 0x7f8b9819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 -- 172.21.15.44:0/1277571045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b98108500 msgr2=0x7f8b9819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b98108500 0x7f8b9819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 -- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 msgr2=0x7f8b9819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 0x7f8b9819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 -- 172.21.15.44:0/1277571045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981a3da0 con 0x7f8b9810d3d0 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.635+0000 7f8b96800640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 0x7f8b981a3670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ec70 tx=0x7f8b9000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.636+0000 7f8b8ea00640 1 -- 172.21.15.44:0/1277571045 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90003b70 con 0x7f8b9810d3d0 2024-04-20T11:25:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.636+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b981a4090 con 0x7f8b9810d3d0 2024-04-20T11:25:31.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.636+0000 7f8b8ea00640 1 -- 172.21.15.44:0/1277571045 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b9000ee80 con 0x7f8b9810d3d0 2024-04-20T11:25:31.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.636+0000 7f8b8ea00640 1 -- 172.21.15.44:0/1277571045 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90010940 con 0x7f8b9810d3d0 2024-04-20T11:25:31.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.636+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981ab920 con 0x7f8b9810d3d0 2024-04-20T11:25:31.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.639+0000 7f8b9f78d740 1 Processor -- start 2024-04-20T11:25:31.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.639+0000 7f8b8ea00640 1 -- 172.21.15.44:0/1277571045 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8b900045b0 con 0x7f8b9810d3d0 2024-04-20T11:25:31.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 -- start start 2024-04-20T11:25:31.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c15340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2b72150 con 0x55fcd2c0cca0 2024-04-20T11:25:31.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2b18370 con 0x55fcd2c15880 2024-04-20T11:25:31.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2b6e110 con 0x55fcd2c0c380 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c15340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b95400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.640+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c15340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33148/0 (socket says 172.21.15.44:33148) 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 -- 172.21.15.44:0/2522388073 learned_addr learned my addr 172.21.15.44:0/2522388073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95400640 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 -- 172.21.15.44:0/2522388073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 msgr2=0x55fcd2c0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 -- 172.21.15.44:0/2522388073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 msgr2=0x55fcd2c17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c17c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 -- 172.21.15.44:0/2522388073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd294e7d0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95400640 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:31.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b8ea00640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b7c0a3d20 0x7f8b7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b8ea00640 1 -- 172.21.15.44:0/1277571045 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8b900d0b60 con 0x7f8b9810d3d0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.641+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c15340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009c70 tx=0x7f8b88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb58014e1048578 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.642+0000 7f8b95400640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b7c0a3d20 0x7f8b7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.642+0000 7f8b83000640 1 -- 172.21.15.44:0/2522388073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88004230 con 0x55fcd2c0cca0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.642+0000 7f8b83000640 1 -- 172.21.15.44:0/2522388073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b880043d0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.642+0000 7f8b83000640 1 -- 172.21.15.44:0/2522388073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88019830 con 0x55fcd2c0cca0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b95400640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b7c0a3d20 0x7f8b7c0a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006010 tx=0x7f8b84005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2522388073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 msgr2=0x55fcd2c15340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c15340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009c70 tx=0x7f8b88011810 comp rx=0 tx=0).stop 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2522388073 shutdown_connections 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c15340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2522388073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2522388073 >> 172.21.15.44:0/2522388073 conn(0x55fcd2c06ab0 msgr2=0x55fcd2c06e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2522388073 shutdown_connections 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.643+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2522388073 wait complete. 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.644+0000 7f8b9f78d740 1 -- start start 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.644+0000 7f8b9f78d740 1 Processor -- start 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.644+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0c380 0x55fcd2c128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.644+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c12de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.644+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c15880 0x55fcd2c13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.644+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2c18670 con 0x55fcd2c15880 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2b2b740 con 0x55fcd2c0c380 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fcd2c187f0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b9f78d740 10 client.0 _ll_register_callbacks cb 0x55fcd2b26b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b96800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c15880 0x55fcd2c13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0c380 0x55fcd2c128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c12de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0c380 0x55fcd2c128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35522/0 (socket says 172.21.15.44:35522) 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95e00640 1 -- 172.21.15.44:0/2160245403 learned_addr learned my addr 172.21.15.44:0/2160245403 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95400640 1 -- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0c380 msgr2=0x55fcd2c128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95400640 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0c380 0x55fcd2c128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95400640 1 -- 172.21.15.44:0/2160245403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c15880 msgr2=0x55fcd2c13320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95400640 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c15880 0x55fcd2c13320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b95400640 1 -- 172.21.15.44:0/2160245403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd28843f0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.645+0000 7f8b96800640 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c15880 0x55fcd2c13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.646+0000 7f8b95400640 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c12de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009230 tx=0x7f8b84009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92483c843fdbad65 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.646+0000 7f8b77400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:25:31.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.646+0000 7f8b77400640 1 -- 172.21.15.44:0/2160245403 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84052020 con 0x55fcd2c0cca0 2024-04-20T11:25:31.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.646+0000 7f8b77400640 1 -- 172.21.15.44:0/2160245403 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8b84003980 con 0x55fcd2c0cca0 2024-04-20T11:25:31.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.646+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b880043d0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.651+0000 7f8b77400640 1 -- 172.21.15.44:0/2160245403 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b8405d050 con 0x55fcd2c0cca0 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.651+0000 7f8b9f78d740 10 client.26131 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.651+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fcd28440f0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.651+0000 7f8b77400640 1 -- 172.21.15.44:0/2160245403 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b880043d0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.651+0000 7f8b77400640 1 -- 172.21.15.44:0/2160245403 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f8b84034440 con 0x55fcd2c0cca0 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b9f78d740 10 client.26131 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b9f78d740 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b9f78d740 20 client.26131 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c25d00 0x55fcd2c28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b9f78d740 4 client.26131 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcd2b7fb70 con 0x55fcd2c25d00 2024-04-20T11:25:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.652+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c25d00 0x55fcd2c28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.653+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c25d00 0x55fcd2c28180 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e88706c8505698 server_cookie=8f8ae1720f84dfbc in_seq=0 out_seq=0 2024-04-20T11:25:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.653+0000 7f8b77400640 10 client.26131 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.653+0000 7f8b77400640 1 -- 172.21.15.44:0/2160245403 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcd2b7fb70 con 0x55fcd2c25d00 2024-04-20T11:25:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.653+0000 7f8b77400640 10 client.26131 handle_command_reply: tid=0 2024-04-20T11:25:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 client.26131 shutdown 2024-04-20T11:25:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0cca0 msgr2=0x55fcd2c12de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c12de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009230 tx=0x7f8b84009260 comp rx=0 tx=0).stop 2024-04-20T11:25:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 shutdown_connections 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c25d00 0x55fcd2c28180 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c15880 0x55fcd2c13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c12de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b77400640 0 client.26131 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2160245403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0c380 0x55fcd2c128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 >> 172.21.15.44:0/2160245403 conn(0x55fcd2c06ab0 msgr2=0x55fcd2c08800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 shutdown_connections 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2160245403 wait complete. 2024-04-20T11:25:31.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.654+0000 7f8b9f78d740 20 client.26131 trim_cache size 0 max 16384 2024-04-20T11:25:31.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.674+0000 7f8b9f78d740 1 Processor -- start 2024-04-20T11:25:31.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.674+0000 7f8b9f78d740 1 -- start start 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0c380 0x55fcd2c14fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c21560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 0x55fcd2c11a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981b3ff0 con 0x55fcd2c0c380 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981b5cb0 con 0x55fcd2c0cca0 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981b46c0 con 0x55fcd2c15880 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b96800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 0x55fcd2c11a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0c380 0x55fcd2c14fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b95400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c21560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b96800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 0x55fcd2c11a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45072/0 (socket says 172.21.15.44:45072) 2024-04-20T11:25:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b96800640 1 -- 172.21.15.44:0/473568390 learned_addr learned my addr 172.21.15.44:0/473568390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.675+0000 7f8b95400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c21560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35532/0 (socket says 172.21.15.44:35532) 2024-04-20T11:25:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b96800640 1 -- 172.21.15.44:0/473568390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0cca0 msgr2=0x55fcd2c21560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b96800640 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c21560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b96800640 1 -- 172.21.15.44:0/473568390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0c380 msgr2=0x55fcd2c14fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b96800640 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0c380 0x55fcd2c14fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b96800640 1 -- 172.21.15.44:0/473568390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd2c10df0 con 0x55fcd2c15880 2024-04-20T11:25:31.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b96800640 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 0x55fcd2c11a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b900ce320 tx=0x7f8b90098e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df17ab3f99856de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b82600640 1 -- 172.21.15.44:0/473568390 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900a5780 con 0x55fcd2c15880 2024-04-20T11:25:31.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b82600640 1 -- 172.21.15.44:0/473568390 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b900bd0b0 con 0x55fcd2c15880 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.676+0000 7f8b82600640 1 -- 172.21.15.44:0/473568390 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9009bf60 con 0x55fcd2c15880 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 -- 172.21.15.44:0/473568390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 msgr2=0x55fcd2c11a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 0x55fcd2c11a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8b900ce320 tx=0x7f8b90098e30 comp rx=0 tx=0).stop 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 -- 172.21.15.44:0/473568390 shutdown_connections 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c15880 0x55fcd2c11a00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c21560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/473568390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0c380 0x55fcd2c14fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 -- 172.21.15.44:0/473568390 >> 172.21.15.44:0/473568390 conn(0x55fcd2c06ab0 msgr2=0x55fcd2c07ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 -- 172.21.15.44:0/473568390 shutdown_connections 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 -- 172.21.15.44:0/473568390 wait complete. 2024-04-20T11:25:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.677+0000 7f8b9f78d740 1 -- start start 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 Processor -- start 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c1df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c19a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981b5520 con 0x55fcd2c0cca0 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981b4df0 con 0x55fcd2c15880 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981b1f30 con 0x55fcd2c0c380 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b9f78d740 10 client.0 _ll_register_callbacks cb 0x7f8b84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c1df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.678+0000 7f8b95400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b96800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c19a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c1df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45086/0 (socket says 172.21.15.44:45086) 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 -- 172.21.15.44:0/2998150261 learned_addr learned my addr 172.21.15.44:0/2998150261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 -- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 msgr2=0x55fcd2c19a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c19a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 -- 172.21.15.44:0/2998150261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 msgr2=0x55fcd2c19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c19300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 -- 172.21.15.44:0/2998150261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd2c8b2c0 con 0x55fcd2c0c380 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b95e00640 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c1df90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b88003d90 tx=0x7f8b88005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e8ecac4873221 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b77400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b77400640 1 -- 172.21.15.44:0/2998150261 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88009e20 con 0x55fcd2c0c380 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b77400640 1 -- 172.21.15.44:0/2998150261 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8b880050c0 con 0x55fcd2c0c380 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.679+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b900bd0b0 con 0x55fcd2c0c380 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b77400640 1 -- 172.21.15.44:0/2998150261 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b880052e0 con 0x55fcd2c0c380 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 10 client.26137 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fcd2c8c430 con 0x55fcd2c0c380 2024-04-20T11:25:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b77400640 1 -- 172.21.15.44:0/2998150261 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b900bd0b0 con 0x55fcd2c0c380 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b77400640 1 -- 172.21.15.44:0/2998150261 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f8b88036030 con 0x55fcd2c0c380 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 10 client.26137 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 20 client.26137 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c8ca90 0x55fcd2c8eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 4 client.26137 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.680+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fcd2b799e0 con 0x55fcd2c8ca90 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.681+0000 7f8b95400640 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c8ca90 0x55fcd2c8eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.681+0000 7f8b95400640 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c8ca90 0x55fcd2c8eeb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47b43637f20cfb4 server_cookie=55a884e3feb8745e in_seq=0 out_seq=0 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.681+0000 7f8b77400640 10 client.26137 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.681+0000 7f8b77400640 1 -- 172.21.15.44:0/2998150261 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fcd2b799e0 con 0x55fcd2c8ca90 2024-04-20T11:25:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.681+0000 7f8b77400640 10 client.26137 handle_command_reply: tid=0 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.681+0000 7f8b9f78d740 1 client.26137 shutdown 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 msgr2=0x55fcd2c1df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c1df90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b88003d90 tx=0x7f8b88005850 comp rx=0 tx=0).stop 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 shutdown_connections 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fcd2c8ca90 0x55fcd2c8eeb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcd2c15880 0x55fcd2c19a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b77400640 0 client.26137 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fcd2c0cca0 0x55fcd2c19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 --2- 172.21.15.44:0/2998150261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcd2c0c380 0x55fcd2c1df90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 >> 172.21.15.44:0/2998150261 conn(0x55fcd2c06ab0 msgr2=0x55fcd2c130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 shutdown_connections 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 1 -- 172.21.15.44:0/2998150261 wait complete. 2024-04-20T11:25:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.682+0000 7f8b9f78d740 20 client.26137 trim_cache size 0 max 16384 2024-04-20T11:25:31.687 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:25:31.687 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:25:31.687 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:25:31.687 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b7c0a3d20 msgr2=0x7f8b7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b7c0a3d20 0x7f8b7c0a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006010 tx=0x7f8b84005e40 comp rx=0 tx=0).stop 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 msgr2=0x7f8b981a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 0x7f8b981a3670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ec70 tx=0x7f8b9000c6a0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 shutdown_connections 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b9810d3d0 0x7f8b981a3670 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b7c0a3d20 0x7f8b7c0a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108ea0 0x7f8b9819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 --2- 172.21.15.44:0/1277571045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b98108500 0x7f8b9819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 >> 172.21.15.44:0/1277571045 conn(0x7f8b98077d80 msgr2=0x7f8b98078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 shutdown_connections 2024-04-20T11:25:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.683+0000 7f8b97200640 1 -- 172.21.15.44:0/1277571045 wait complete. 2024-04-20T11:25:31.701 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:25:31.701 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:25:31.767 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:31 smithi044 ceph-mon[16855]: pgmap v90: 97 pgs: 97 active+clean; 216 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-04-20T11:25:31.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 -- 172.21.15.44:0/1436941325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 msgr2=0x7f0d3c110d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 --2- 172.21.15.44:0/1436941325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 0x7f0d3c110d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009be0 tx=0x7f0d3402bef0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 -- 172.21.15.44:0/1436941325 shutdown_connections 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 --2- 172.21.15.44:0/1436941325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 0x7f0d3c110d10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 --2- 172.21.15.44:0/1436941325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c101690 0x7f0d3c10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 --2- 172.21.15.44:0/1436941325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d3c100d70 0x7f0d3c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 -- 172.21.15.44:0/1436941325 >> 172.21.15.44:0/1436941325 conn(0x7f0d3c077f60 msgr2=0x7f0d3c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 -- 172.21.15.44:0/1436941325 shutdown_connections 2024-04-20T11:25:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.905+0000 7f0d41a00640 1 -- 172.21.15.44:0/1436941325 wait complete. 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.906+0000 7f0d41a00640 1 Processor -- start 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.906+0000 7f0d41a00640 1 -- start start 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d41a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 0x7f0d3c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d41a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d3c101690 0x7f0d3c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d41a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 0x7f0d3c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d41a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1146c0 con 0x7f0d3c100d70 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d41a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c114540 con 0x7f0d3c10e920 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d41a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c114840 con 0x7f0d3c101690 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 0x7f0d3c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 0x7f0d3c1a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33204/0 (socket says 172.21.15.44:33204) 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 -- 172.21.15.44:0/731802977 learned_addr learned my addr 172.21.15.44:0/731802977 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3a000640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d3c101690 0x7f0d3c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3b400640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 0x7f0d3c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 -- 172.21.15.44:0/731802977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d3c101690 msgr2=0x7f0d3c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d3c101690 0x7f0d3c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 -- 172.21.15.44:0/731802977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 msgr2=0x7f0d3c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 0x7f0d3c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.907+0000 7f0d3aa00640 1 -- 172.21.15.44:0/731802977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19b0a0 con 0x7f0d3c100d70 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.908+0000 7f0d3aa00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 0x7f0d3c1a0720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d950 tx=0x7f0d2c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.908+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017030 con 0x7f0d3c100d70 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.908+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19b390 con 0x7f0d3c100d70 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.908+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d2c004020 con 0x7f0d3c100d70 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.908+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0047a0 con 0x7f0d3c100d70 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.908+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a75e0 con 0x7f0d3c100d70 2024-04-20T11:25:31.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.909+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c068d80 con 0x7f0d3c100d70 2024-04-20T11:25:31.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.911+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0d2c0041d0 con 0x7f0d3c100d70 2024-04-20T11:25:31.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.913+0000 7f0d33400640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d200a3da0 0x7f0d200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:31.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.913+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f0d2c0d0020 con 0x7f0d3c100d70 2024-04-20T11:25:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.913+0000 7f0d3a000640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d200a3da0 0x7f0d200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.914+0000 7f0d3a000640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d200a3da0 0x7f0d200a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c1007b0 tx=0x7f0d24009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:31.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:31.918+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0d2c09c6d0 con 0x7f0d3c100d70 2024-04-20T11:25:31.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:31 smithi078 ceph-mon[21464]: pgmap v90: 97 pgs: 97 active+clean; 216 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-04-20T11:25:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:31 smithi165 ceph-mon[21975]: pgmap v90: 97 pgs: 97 active+clean; 216 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-04-20T11:25:32.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.165+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d3c19be00 con 0x7f0d3c100d70 2024-04-20T11:25:32.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.168+0000 7f0d33400640 1 -- 172.21.15.44:0/731802977 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11962 (secure 0 0 0) 0x7f0d2c09bfd0 con 0x7f0d3c100d70 2024-04-20T11:25:32.167 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:25:32.168 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:25:25.635168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:25:32.168 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-04-20T11:25:32.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d200a3da0 msgr2=0x7f0d200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d200a3da0 0x7f0d200a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c1007b0 tx=0x7f0d24009260 comp rx=0 tx=0).stop 2024-04-20T11:25:32.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 msgr2=0x7f0d3c1a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 0x7f0d3c1a0720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d950 tx=0x7f0d2c00dd50 comp rx=0 tx=0).stop 2024-04-20T11:25:32.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 shutdown_connections 2024-04-20T11:25:32.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3c10e920 0x7f0d3c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d3c101690 0x7f0d3c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f0d200a3da0 0x7f0d200a6260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 --2- 172.21.15.44:0/731802977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d3c100d70 0x7f0d3c1a0720 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.170+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 >> 172.21.15.44:0/731802977 conn(0x7f0d3c077f60 msgr2=0x7f0d3c10fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.171+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 shutdown_connections 2024-04-20T11:25:32.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.171+0000 7f0d41a00640 1 -- 172.21.15.44:0/731802977 wait complete. 2024-04-20T11:25:32.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 -- 172.21.15.44:0/2887320144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f002410c450 msgr2=0x7f002410e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 --2- 172.21.15.44:0/2887320144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f002410c450 0x7f002410e8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f001c009ca0 tx=0x7f001c02c180 comp rx=0 tx=0).stop 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 -- 172.21.15.44:0/2887320144 shutdown_connections 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 --2- 172.21.15.44:0/2887320144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f002410c450 0x7f002410e8f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 --2- 172.21.15.44:0/2887320144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 0x7f002410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 --2- 172.21.15.44:0/2887320144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024103030 0x7f0024103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 -- 172.21.15.44:0/2887320144 >> 172.21.15.44:0/2887320144 conn(0x7f00240fc850 msgr2=0x7f00240fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 -- 172.21.15.44:0/2887320144 shutdown_connections 2024-04-20T11:25:32.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.412+0000 7f002a400640 1 -- 172.21.15.44:0/2887320144 wait complete. 2024-04-20T11:25:32.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.413+0000 7f002a400640 1 Processor -- start 2024-04-20T11:25:32.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.413+0000 7f002a400640 1 -- start start 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f002a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 0x7f002419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f002a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 0x7f002419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f002a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f002410c450 0x7f0024198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f002a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0024112370 con 0x7f0024103950 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f002a400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00241121f0 con 0x7f002410c450 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f002a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00241124f0 con 0x7f0024103030 2024-04-20T11:25:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f0023e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 0x7f002419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f0029000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 0x7f002419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f0029000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 0x7f002419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45116/0 (socket says 172.21.15.44:45116) 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.414+0000 7f0029000640 1 -- 172.21.15.44:0/2954231393 learned_addr learned my addr 172.21.15.44:0/2954231393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0029a00640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f002410c450 0x7f0024198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0023e00640 1 -- 172.21.15.44:0/2954231393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 msgr2=0x7f002419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0023e00640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 0x7f002419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0023e00640 1 -- 172.21.15.44:0/2954231393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f002410c450 msgr2=0x7f0024198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0023e00640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f002410c450 0x7f0024198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0023e00640 1 -- 172.21.15.44:0/2954231393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0024198f30 con 0x7f0024103950 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0029a00640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f002410c450 0x7f0024198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0029000640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 0x7f002419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.415+0000 7f0023e00640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 0x7f002419eaf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f001000e7a0 tx=0x7f001000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.416+0000 7f0021600640 1 -- 172.21.15.44:0/2954231393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001000ccc0 con 0x7f0024103950 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.416+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0024199220 con 0x7f0024103950 2024-04-20T11:25:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.416+0000 7f0021600640 1 -- 172.21.15.44:0/2954231393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0010009530 con 0x7f0024103950 2024-04-20T11:25:32.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.416+0000 7f0021600640 1 -- 172.21.15.44:0/2954231393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100188e0 con 0x7f0024103950 2024-04-20T11:25:32.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.416+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00241a5530 con 0x7f0024103950 2024-04-20T11:25:32.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.420+0000 7f002e764740 1 Processor -- start 2024-04-20T11:25:32.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.420+0000 7f002e764740 1 -- start start 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f002e764740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e41550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f002e764740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f002e764740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e4a670 0x55b085e4ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f002e764740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b085da6050 con 0x55b085e4a670 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f002e764740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b085d4c490 con 0x55b085e41170 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f002e764740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b085da2010 con 0x55b085e41a90 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e4a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45124/0 (socket says 172.21.15.44:45124) 2024-04-20T11:25:32.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 -- 172.21.15.44:0/4021157230 learned_addr learned my addr 172.21.15.44:0/4021157230 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e41550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 -- 172.21.15.44:0/4021157230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 msgr2=0x55b085e41550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e41550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 -- 172.21.15.44:0/4021157230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e4a670 msgr2=0x55b085e4ca60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.421+0000 7f0029000640 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e4a670 0x55b085e4ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.422+0000 7f0029000640 1 -- 172.21.15.44:0/4021157230 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b085d54070 con 0x55b085e41a90 2024-04-20T11:25:32.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.422+0000 7f0021600640 1 -- 172.21.15.44:0/2954231393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0010018a80 con 0x7f0024103950 2024-04-20T11:25:32.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.422+0000 7f0029000640 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e4a130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0018011630 tx=0x7f0018011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a940251abf96117 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.422+0000 7f0016000640 1 -- 172.21.15.44:0/4021157230 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001801e070 con 0x55b085e41a90 2024-04-20T11:25:32.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.422+0000 7f0016000640 1 -- 172.21.15.44:0/4021157230 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f001800fbf0 con 0x55b085e41a90 2024-04-20T11:25:32.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.422+0000 7f0016000640 1 -- 172.21.15.44:0/4021157230 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018019da0 con 0x55b085e41a90 2024-04-20T11:25:32.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 -- 172.21.15.44:0/4021157230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 msgr2=0x55b085e4a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e4a130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0018011630 tx=0x7f0018011af0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 -- 172.21.15.44:0/4021157230 shutdown_connections 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e4a670 0x55b085e4ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e4a130 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 --2- 172.21.15.44:0/4021157230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e41550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.423+0000 7f002e764740 1 -- 172.21.15.44:0/4021157230 >> 172.21.15.44:0/4021157230 conn(0x55b085e3b8a0 msgr2=0x55b085e3bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.424+0000 7f002e764740 1 -- 172.21.15.44:0/4021157230 shutdown_connections 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.424+0000 7f0021600640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f000c0a3cd0 0x7f000c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.424+0000 7f002e764740 1 -- 172.21.15.44:0/4021157230 wait complete. 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.424+0000 7f0021600640 1 -- 172.21.15.44:0/2954231393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f00100d0890 con 0x7f0024103950 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.424+0000 7f002e764740 1 -- start start 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 Processor -- start 2024-04-20T11:25:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f0029000640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f000c0a3cd0 0x7f000c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e43bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41a90 0x55b085e44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 0x55b085e481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b085e4d270 con 0x55b085e41a90 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b085d5f860 con 0x55b085e41170 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.425+0000 7f002e764740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b085e4d3f0 con 0x55b085e4a670 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f002e764740 10 client.0 _ll_register_callbacks cb 0x55b085d2dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 0x55b085e481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 0x55b085e481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45134/0 (socket says 172.21.15.44:45134) 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 -- 172.21.15.44:0/2751682851 learned_addr learned my addr 172.21.15.44:0/2751682851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029000640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f000c0a3cd0 0x7f000c0a6190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f00180198e0 tx=0x7f0018019940 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029000640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e43bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0023e00640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41a90 0x55b085e44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 -- 172.21.15.44:0/2751682851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 msgr2=0x55b085e43bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e43bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 -- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41a90 msgr2=0x55b085e44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41a90 0x55b085e44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.426+0000 7f0029a00640 1 -- 172.21.15.44:0/2751682851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b085d63950 con 0x55b085e4a670 2024-04-20T11:25:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f0029a00640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 0x55b085e481a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f001c009ca0 tx=0x7f001c003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6d9571df26b479a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f000aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f000aa00640 1 -- 172.21.15.44:0/2751682851 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c044070 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f000aa00640 1 -- 172.21.15.44:0/2751682851 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f001c032b30 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f001800fbf0 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f000aa00640 1 -- 172.21.15.44:0/2751682851 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f001c039040 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f002e764740 10 client.26155 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.427+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b085d31820 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f000aa00640 1 -- 172.21.15.44:0/2751682851 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f001800fbf0 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f000aa00640 1 -- 172.21.15.44:0/2751682851 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f001c050020 con 0x55b085e4a670 2024-04-20T11:25:32.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f002e764740 10 client.26155 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f002e764740 10 client.26155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f002e764740 20 client.26155 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f002e764740 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085e62ee0 0x55b085e65300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f002e764740 4 client.26155 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b085db3a70 con 0x55b085e62ee0 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.428+0000 7f0029000640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085e62ee0 0x55b085e65300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.429+0000 7f000aa00640 10 client.26155 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:25:32.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.429+0000 7f0029000640 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085e62ee0 0x55b085e65300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc6202621b90ecfe server_cookie=f6fd6e2a94a56745 in_seq=0 out_seq=0 2024-04-20T11:25:32.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f000aa00640 1 -- 172.21.15.44:0/2751682851 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b085db3a70 con 0x55b085e62ee0 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f000aa00640 10 client.26155 handle_command_reply: tid=0 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 client.26155 shutdown 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 msgr2=0x55b085e481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 0x55b085e481a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f001c009ca0 tx=0x7f001c003f80 comp rx=0 tx=0).stop 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 shutdown_connections 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e4a670 0x55b085e481a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41a90 0x55b085e44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085e62ee0 0x55b085e65300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 --2- 172.21.15.44:0/2751682851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e41170 0x55b085e43bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 >> 172.21.15.44:0/2751682851 conn(0x55b085e3b8a0 msgr2=0x55b085e3d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f000aa00640 0 client.26155 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:25:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.430+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 shutdown_connections 2024-04-20T11:25:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.431+0000 7f002e764740 1 -- 172.21.15.44:0/2751682851 wait complete. 2024-04-20T11:25:32.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.431+0000 7f002e764740 20 client.26155 trim_cache size 0 max 16384 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 Processor -- start 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 -- start start 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00241ab890 con 0x55b085e41170 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00241ad550 con 0x55b085e4a670 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f002e764740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00241abf60 con 0x55b085e41a90 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f0029000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f0029000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e6abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33242/0 (socket says 172.21.15.44:33242) 2024-04-20T11:25:32.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f0029000640 1 -- 172.21.15.44:0/3596542640 learned_addr learned my addr 172.21.15.44:0/3596542640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f0029a00640 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.446+0000 7f0023e00640 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029000640 1 -- 172.21.15.44:0/3596542640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 msgr2=0x55b085e6b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029000640 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e6b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029000640 1 -- 172.21.15.44:0/3596542640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 msgr2=0x55b085e48120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029000640 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e48120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029000640 1 -- 172.21.15.44:0/3596542640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b085e418c0 con 0x55b085e41170 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029a00640 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0029000640 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e6abb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0018019910 tx=0x7f0018020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485165be0e98f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0015600640 1 -- 172.21.15.44:0/3596542640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001801e040 con 0x55b085e41170 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0015600640 1 -- 172.21.15.44:0/3596542640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0018019ca0 con 0x55b085e41170 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.447+0000 7f0015600640 1 -- 172.21.15.44:0/3596542640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00180048e0 con 0x55b085e41170 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 -- 172.21.15.44:0/3596542640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 msgr2=0x55b085e6abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e6abb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0018019910 tx=0x7f0018020000 comp rx=0 tx=0).stop 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 -- 172.21.15.44:0/3596542640 shutdown_connections 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e48120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e6b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 --2- 172.21.15.44:0/3596542640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e6abb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 -- 172.21.15.44:0/3596542640 >> 172.21.15.44:0/3596542640 conn(0x55b085e3b8a0 msgr2=0x55b085e3d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 -- 172.21.15.44:0/3596542640 shutdown_connections 2024-04-20T11:25:32.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.448+0000 7f002e764740 1 -- 172.21.15.44:0/3596542640 wait complete. 2024-04-20T11:25:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.449+0000 7f002e764740 1 -- start start 2024-04-20T11:25:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.449+0000 7f002e764740 1 Processor -- start 2024-04-20T11:25:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.449+0000 7f002e764740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e5f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.449+0000 7f002e764740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e5c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f002e764740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e5cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f002e764740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00241acdc0 con 0x55b085e41170 2024-04-20T11:25:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f002e764740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00241ac690 con 0x55b085e4a670 2024-04-20T11:25:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f002e764740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00241a97f0 con 0x55b085e41a90 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f002e764740 10 client.0 _ll_register_callbacks cb 0x7f001c003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e5f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e5f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33248/0 (socket says 172.21.15.44:33248) 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 -- 172.21.15.44:0/3809989811 learned_addr learned my addr 172.21.15.44:0/3809989811 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029a00640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e5cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0023e00640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e5c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 -- 172.21.15.44:0/3809989811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 msgr2=0x55b085e5c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e5c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 -- 172.21.15.44:0/3809989811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 msgr2=0x55b085e5cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e5cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0029000640 1 -- 172.21.15.44:0/3809989811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b085ab83f0 con 0x55b085e41170 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.450+0000 7f0023e00640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e5c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f0029000640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e5f650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f001801c040 tx=0x7f0018009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292abae4c3fee96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f000aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f000aa00640 1 -- 172.21.15.44:0/3809989811 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018041060 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f000aa00640 1 -- 172.21.15.44:0/3809989811 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f0018016050 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0018019ca0 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f000aa00640 1 -- 172.21.15.44:0/3809989811 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f001801e040 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f002e764740 10 client.16023 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b085b827d0 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.451+0000 7f000aa00640 1 -- 172.21.15.44:0/3809989811 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0018019ca0 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f000aa00640 1 -- 172.21.15.44:0/3809989811 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f0018044020 con 0x55b085e41170 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f002e764740 10 client.16023 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f002e764740 10 client.16023 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f002e764740 20 client.16023 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f002e764740 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085ec9330 0x55b085ecb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f002e764740 4 client.16023 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b085dad8e0 con 0x55b085ec9330 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.452+0000 7f0023e00640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085ec9330 0x55b085ecb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f0023e00640 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085ec9330 0x55b085ecb750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16c3d387c9da1651 server_cookie=e1cb252938c07298 in_seq=0 out_seq=0 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f000aa00640 10 client.16023 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f000aa00640 1 -- 172.21.15.44:0/3809989811 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b085dad8e0 con 0x55b085ec9330 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f000aa00640 10 client.16023 handle_command_reply: tid=0 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 client.16023 shutdown 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 msgr2=0x55b085e5f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e5f650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f001801c040 tx=0x7f0018009330 comp rx=0 tx=0).stop 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 shutdown_connections 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b085ec9330 0x55b085ecb750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b085e4a670 0x55b085e5cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b085e41a90 0x55b085e5c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 --2- 172.21.15.44:0/3809989811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b085e41170 0x55b085e5f650 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 >> 172.21.15.44:0/3809989811 conn(0x55b085e3b8a0 msgr2=0x55b085e3a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f000aa00640 0 client.16023 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:25:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.453+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 shutdown_connections 2024-04-20T11:25:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002e764740 1 -- 172.21.15.44:0/3809989811 wait complete. 2024-04-20T11:25:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002e764740 20 client.16023 trim_cache size 0 max 16384 2024-04-20T11:25:32.458 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f000c0a3cd0 msgr2=0x7f000c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f000c0a3cd0 0x7f000c0a6190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f00180198e0 tx=0x7f0018019940 comp rx=0 tx=0).stop 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 msgr2=0x7f002419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 0x7f002419eaf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f001000e7a0 tx=0x7f001000ec60 comp rx=0 tx=0).stop 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 shutdown_connections 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f002410c450 0x7f0024198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f000c0a3cd0 0x7f000c0a6190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0024103950 0x7f002419eaf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 --2- 172.21.15.44:0/2954231393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0024103030 0x7f002419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.454+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 >> 172.21.15.44:0/2954231393 conn(0x7f00240fc850 msgr2=0x7f00240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.455+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 shutdown_connections 2024-04-20T11:25:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.455+0000 7f002a400640 1 -- 172.21.15.44:0/2954231393 wait complete. 2024-04-20T11:25:32.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 -- 172.21.15.44:0/1105650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f641410d3d0 msgr2=0x7f641410d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 --2- 172.21.15.44:0/1105650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f641410d3d0 0x7f641410d7b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6408009900 tx=0x7f640802bcd0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 -- 172.21.15.44:0/1105650101 shutdown_connections 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 --2- 172.21.15.44:0/1105650101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6414108ea0 0x7f64141152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 --2- 172.21.15.44:0/1105650101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6414108500 0x7f6414108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 --2- 172.21.15.44:0/1105650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f641410d3d0 0x7f641410d7b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 -- 172.21.15.44:0/1105650101 >> 172.21.15.44:0/1105650101 conn(0x7f6414077d80 msgr2=0x7f6414106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 -- 172.21.15.44:0/1105650101 shutdown_connections 2024-04-20T11:25:32.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.686+0000 7f641ac00640 1 -- 172.21.15.44:0/1105650101 wait complete. 2024-04-20T11:25:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.687+0000 7f641ac00640 1 Processor -- start 2024-04-20T11:25:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.687+0000 7f641ac00640 1 -- start start 2024-04-20T11:25:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641ac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 0x7f641419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6414108ea0 0x7f641419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f641410d3d0 0x7f64141a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6414118980 con 0x7f641410d3d0 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641ac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6414118800 con 0x7f6414108500 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6414118b00 con 0x7f6414108ea0 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f6419800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 0x7f641419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f6419800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 0x7f641419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35626/0 (socket says 172.21.15.44:35626) 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f6419800640 1 -- 172.21.15.44:0/408923180 learned_addr learned my addr 172.21.15.44:0/408923180 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f641a200640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f641410d3d0 0x7f64141a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.688+0000 7f6418e00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6414108ea0 0x7f641419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6419800640 1 -- 172.21.15.44:0/408923180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6414108ea0 msgr2=0x7f641419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6419800640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6414108ea0 0x7f641419f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6419800640 1 -- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f641410d3d0 msgr2=0x7f64141a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6419800640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f641410d3d0 0x7f64141a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6419800640 1 -- 172.21.15.44:0/408923180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64141a3d90 con 0x7f6414108500 2024-04-20T11:25:32.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6419800640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 0x7f641419eda0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6408000c00 tx=0x7f6408004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6412000640 1 -- 172.21.15.44:0/408923180 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408043020 con 0x7f6414108500 2024-04-20T11:25:32.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6412000640 1 -- 172.21.15.44:0/408923180 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6408004310 con 0x7f6414108500 2024-04-20T11:25:32.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f6412000640 1 -- 172.21.15.44:0/408923180 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408003bc0 con 0x7f6414108500 2024-04-20T11:25:32.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.689+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64141a4020 con 0x7f6414108500 2024-04-20T11:25:32.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.690+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64141ab910 con 0x7f6414108500 2024-04-20T11:25:32.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.693+0000 7f6412000640 1 -- 172.21.15.44:0/408923180 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6408046020 con 0x7f6414108500 2024-04-20T11:25:32.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.693+0000 7f641eb22740 1 Processor -- start 2024-04-20T11:25:32.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.693+0000 7f641eb22740 1 -- start start 2024-04-20T11:25:32.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641eb22740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641eb22740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5ba90 0x559fcab64130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641eb22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641eb22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559fcaac0050 con 0x559fcab64670 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641eb22740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559fcaa66490 con 0x559fcab5b170 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641eb22740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559fcaabc010 con 0x559fcab5ba90 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab5b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35638/0 (socket says 172.21.15.44:35638) 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f641a200640 1 -- 172.21.15.44:0/2772718137 learned_addr learned my addr 172.21.15.44:0/2772718137 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f6418e00640 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f6419800640 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5ba90 0x559fcab64130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.694+0000 7f6412000640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f63fc0a3da0 0x7f63fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 -- 172.21.15.44:0/2772718137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5ba90 msgr2=0x559fcab64130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5ba90 0x559fcab64130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 -- 172.21.15.44:0/2772718137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 msgr2=0x559fcab66a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab66a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 -- 172.21.15.44:0/2772718137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcaa6e070 con 0x559fcab5b170 2024-04-20T11:25:32.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f6419800640 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5ba90 0x559fcab64130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:25:32.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f6412000640 1 -- 172.21.15.44:0/408923180 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f64080f2610 con 0x7f6414108500 2024-04-20T11:25:32.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab5b550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f640c009f40 tx=0x7f640c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d37eb559f970e372 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f641a200640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f63fc0a3da0 0x7f63fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.695+0000 7f6406a00640 1 -- 172.21.15.44:0/2772718137 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c004250 con 0x559fcab5b170 2024-04-20T11:25:32.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f6406a00640 1 -- 172.21.15.44:0/2772718137 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f640c00f480 con 0x559fcab5b170 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f6406a00640 1 -- 172.21.15.44:0/2772718137 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c01b690 con 0x559fcab5b170 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 -- 172.21.15.44:0/2772718137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 msgr2=0x559fcab5b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab5b550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f640c009f40 tx=0x7f640c0118d0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641a200640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f63fc0a3da0 0x7f63fc0a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f640c009fd0 tx=0x7f640c003f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 -- 172.21.15.44:0/2772718137 shutdown_connections 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab66a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5ba90 0x559fcab64130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 --2- 172.21.15.44:0/2772718137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab5b550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.696+0000 7f641eb22740 1 -- 172.21.15.44:0/2772718137 >> 172.21.15.44:0/2772718137 conn(0x559fcab558a0 msgr2=0x559fcab55c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.697+0000 7f641eb22740 1 -- 172.21.15.44:0/2772718137 shutdown_connections 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.697+0000 7f641eb22740 1 -- 172.21.15.44:0/2772718137 wait complete. 2024-04-20T11:25:32.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.697+0000 7f641eb22740 1 -- start start 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.697+0000 7f641eb22740 1 Processor -- start 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559fcab67460 con 0x559fcab64670 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559fcaa79860 con 0x559fcab5ba90 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559fcab675e0 con 0x559fcab5b170 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641eb22740 10 client.0 _ll_register_callbacks cb 0x559fcaa47fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab61be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35646/0 (socket says 172.21.15.44:35646) 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 -- 172.21.15.44:0/2569750703 learned_addr learned my addr 172.21.15.44:0/2569750703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641a200640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6419800640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 -- 172.21.15.44:0/2569750703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 msgr2=0x559fcab616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 -- 172.21.15.44:0/2569750703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 msgr2=0x559fcab62120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab62120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6418e00640 1 -- 172.21.15.44:0/2569750703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcaa7d950 con 0x559fcab5ba90 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f6419800640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.698+0000 7f641a200640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f6418e00640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab61be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f640000b9f0 tx=0x7f640000beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e423c2a7bef5b711 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f63fb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f63fb400640 1 -- 172.21.15.44:0/2569750703 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640001b640 con 0x559fcab5ba90 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f63fb400640 1 -- 172.21.15.44:0/2569750703 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f640001b7e0 con 0x559fcab5ba90 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c00f480 con 0x559fcab5ba90 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f63fb400640 1 -- 172.21.15.44:0/2569750703 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f640001b9e0 con 0x559fcab5ba90 2024-04-20T11:25:32.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f641eb22740 10 client.26159 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.699+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559fcaa4b820 con 0x559fcab5ba90 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f63fb400640 1 -- 172.21.15.44:0/2569750703 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f640c00f480 con 0x559fcab5ba90 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f63fb400640 1 -- 172.21.15.44:0/2569750703 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f6400023020 con 0x559fcab5ba90 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f641eb22740 10 client.26159 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f641eb22740 10 client.26159 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f641eb22740 20 client.26159 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f641eb22740 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcab74670 0x559fcab76ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f641eb22740 4 client.26159 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcaacda70 con 0x559fcab74670 2024-04-20T11:25:32.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.700+0000 7f6419800640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcab74670 0x559fcab76ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.701+0000 7f6419800640 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcab74670 0x559fcab76ad0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83f5b486606913a2 server_cookie=d1d526f4ba90098c in_seq=0 out_seq=0 2024-04-20T11:25:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.701+0000 7f63fb400640 10 client.26159 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:25:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.701+0000 7f63fb400640 1 -- 172.21.15.44:0/2569750703 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fcaacda70 con 0x559fcab74670 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.701+0000 7f63fb400640 10 client.26159 handle_command_reply: tid=0 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.701+0000 7f641eb22740 1 client.26159 shutdown 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 msgr2=0x559fcab61be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab61be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f640000b9f0 tx=0x7f640000beb0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 shutdown_connections 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcab74670 0x559fcab76ad0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab61be0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 --2- 172.21.15.44:0/2569750703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 >> 172.21.15.44:0/2569750703 conn(0x559fcab558a0 msgr2=0x559fcab575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f63fb400640 0 client.26159 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 shutdown_connections 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 1 -- 172.21.15.44:0/2569750703 wait complete. 2024-04-20T11:25:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.702+0000 7f641eb22740 20 client.26159 trim_cache size 0 max 16384 2024-04-20T11:25:32.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 Processor -- start 2024-04-20T11:25:32.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 -- start start 2024-04-20T11:25:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab63dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab7c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64141b4100 con 0x559fcab64670 2024-04-20T11:25:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64141b5dc0 con 0x559fcab5ba90 2024-04-20T11:25:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.717+0000 7f641eb22740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64141b47d0 con 0x559fcab5b170 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f641a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab7c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f641a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab7c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33278/0 (socket says 172.21.15.44:33278) 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f641a200640 1 -- 172.21.15.44:0/1217941725 learned_addr learned my addr 172.21.15.44:0/1217941725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6418e00640 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6419800640 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab63dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6418e00640 1 -- 172.21.15.44:0/1217941725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 msgr2=0x559fcab63dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6418e00640 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab63dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6418e00640 1 -- 172.21.15.44:0/1217941725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 msgr2=0x559fcab7c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6418e00640 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab7c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6418e00640 1 -- 172.21.15.44:0/1217941725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcab5b8c0 con 0x559fcab5ba90 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.718+0000 7f6419800640 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab63dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.719+0000 7f6418e00640 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab70350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559fcab62c70 tx=0x7f6400004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db97357fda3ccc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.719+0000 7f6406000640 1 -- 172.21.15.44:0/1217941725 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640002b070 con 0x559fcab5ba90 2024-04-20T11:25:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.719+0000 7f6406000640 1 -- 172.21.15.44:0/1217941725 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6400011690 con 0x559fcab5ba90 2024-04-20T11:25:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.719+0000 7f6406000640 1 -- 172.21.15.44:0/1217941725 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64000053e0 con 0x559fcab5ba90 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 -- 172.21.15.44:0/1217941725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 msgr2=0x559fcab70350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab70350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559fcab62c70 tx=0x7f6400004270 comp rx=0 tx=0).stop 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 -- 172.21.15.44:0/1217941725 shutdown_connections 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab64670 0x559fcab7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5ba90 0x559fcab70350 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 --2- 172.21.15.44:0/1217941725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab5b170 0x559fcab63dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 -- 172.21.15.44:0/1217941725 >> 172.21.15.44:0/1217941725 conn(0x559fcab558a0 msgr2=0x559fcab57560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 -- 172.21.15.44:0/1217941725 shutdown_connections 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 -- 172.21.15.44:0/1217941725 wait complete. 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.720+0000 7f641eb22740 1 -- start start 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 Processor -- start 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab5ba90 0x559fcab687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab64670 0x559fcab68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64141b5630 con 0x559fcab5ba90 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64141b4f00 con 0x559fcab5b170 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64141b2040 con 0x559fcab64670 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641eb22740 10 client.0 _ll_register_callbacks cb 0x7f6400004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f6419800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f6419800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35664/0 (socket says 172.21.15.44:35664) 2024-04-20T11:25:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f6419800640 1 -- 172.21.15.44:0/1525852354 learned_addr learned my addr 172.21.15.44:0/1525852354 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f6418e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab5ba90 0x559fcab687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.721+0000 7f641a200640 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab64670 0x559fcab68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f6419800640 1 -- 172.21.15.44:0/1525852354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab64670 msgr2=0x559fcab68d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f6419800640 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab64670 0x559fcab68d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f6419800640 1 -- 172.21.15.44:0/1525852354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab5ba90 msgr2=0x559fcab687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f6419800640 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab5ba90 0x559fcab687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f6419800640 1 -- 172.21.15.44:0/1525852354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fca7d23f0 con 0x559fcab5b170 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f6419800640 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab682a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6408005c20 tx=0x7f6408009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1109f0f8032ee2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f63fb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f63fb400640 1 -- 172.21.15.44:0/1525852354 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64080d4020 con 0x559fcab5b170 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f63fb400640 1 -- 172.21.15.44:0/1525852354 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f64080054c0 con 0x559fcab5b170 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6400011690 con 0x559fcab5b170 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f63fb400640 1 -- 172.21.15.44:0/1525852354 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64080a90b0 con 0x559fcab5b170 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f641eb22740 10 client.26165 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.722+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559fca89c7d0 con 0x559fcab5b170 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f63fb400640 1 -- 172.21.15.44:0/1525852354 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6400011690 con 0x559fcab5b170 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f63fb400640 1 -- 172.21.15.44:0/1525852354 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f64080bad40 con 0x559fcab5b170 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f641eb22740 10 client.26165 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f641eb22740 10 client.26165 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f641eb22740 20 client.26165 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f641eb22740 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcabdb3a0 0x559fcabdd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f641eb22740 4 client.26165 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fcaac78e0 con 0x559fcabdb3a0 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.723+0000 7f6418e00640 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcabdb3a0 0x559fcabdd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f6418e00640 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcabdb3a0 0x559fcabdd7c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9bfaffdf077f7f00 server_cookie=8f20f93a3f13171f in_seq=0 out_seq=0 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f63fb400640 10 client.26165 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f63fb400640 1 -- 172.21.15.44:0/1525852354 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fcaac78e0 con 0x559fcabdb3a0 2024-04-20T11:25:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f63fb400640 10 client.26165 handle_command_reply: tid=0 2024-04-20T11:25:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 client.26165 shutdown 2024-04-20T11:25:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 msgr2=0x559fcab682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab682a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6408005c20 tx=0x7f6408009260 comp rx=0 tx=0).stop 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 shutdown_connections 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fcab64670 0x559fcab68d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559fcabdb3a0 0x559fcabdd7c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559fcab5ba90 0x559fcab687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 --2- 172.21.15.44:0/1525852354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fcab5b170 0x559fcab682a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 >> 172.21.15.44:0/1525852354 conn(0x559fcab558a0 msgr2=0x559fcab61640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f63fb400640 0 client.26165 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 shutdown_connections 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 1 -- 172.21.15.44:0/1525852354 wait complete. 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.724+0000 7f641eb22740 20 client.26165 trim_cache size 0 max 16384 2024-04-20T11:25:32.728 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:25:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f63fc0a3da0 msgr2=0x7f63fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f63fc0a3da0 0x7f63fc0a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f640c009fd0 tx=0x7f640c003f90 comp rx=0 tx=0).stop 2024-04-20T11:25:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 msgr2=0x7f641419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 0x7f641419eda0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6408000c00 tx=0x7f6408004a00 comp rx=0 tx=0).stop 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 shutdown_connections 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f63fc0a3da0 0x7f63fc0a6260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f641410d3d0 0x7f64141a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6414108ea0 0x7f641419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 --2- 172.21.15.44:0/408923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6414108500 0x7f641419eda0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 >> 172.21.15.44:0/408923180 conn(0x7f6414077d80 msgr2=0x7f6414078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 shutdown_connections 2024-04-20T11:25:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.725+0000 7f641ac00640 1 -- 172.21.15.44:0/408923180 wait complete. 2024-04-20T11:25:32.741 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:32 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/731802977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:25:32.741 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:32 smithi044 ceph-mon[16855]: pgmap v91: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2024-04-20T11:25:32.745 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:25:32.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:32 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/731802977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:25:32.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:32 smithi078 ceph-mon[21464]: pgmap v91: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2024-04-20T11:25:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 -- 172.21.15.44:0/1017100350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8112810 msgr2=0x7f4dd8112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 --2- 172.21.15.44:0/1017100350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8112810 0x7f4dd8112bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009900 tx=0x7f4dc802bc40 comp rx=0 tx=0).stop 2024-04-20T11:25:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 -- 172.21.15.44:0/1017100350 shutdown_connections 2024-04-20T11:25:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 --2- 172.21.15.44:0/1017100350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 --2- 172.21.15.44:0/1017100350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd810c800 0x7f4dd810cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 --2- 172.21.15.44:0/1017100350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8112810 0x7f4dd8112bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 -- 172.21.15.44:0/1017100350 >> 172.21.15.44:0/1017100350 conn(0x7f4dd806cde0 msgr2=0x7f4dd810af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 -- 172.21.15.44:0/1017100350 shutdown_connections 2024-04-20T11:25:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.956+0000 7f4ddce00640 1 -- 172.21.15.44:0/1017100350 wait complete. 2024-04-20T11:25:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.958+0000 7f4ddce00640 1 Processor -- start 2024-04-20T11:25:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.958+0000 7f4ddce00640 1 -- start start 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd810c800 0x7f4dd81a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 0x7f4dd81a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4ddce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd811ccb0 con 0x7f4dd810d1c0 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4ddce00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd811cb30 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4ddce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd811ce30 con 0x7f4dd810c800 2024-04-20T11:25:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 0x7f4dd81a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 0x7f4dd81a8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35680/0 (socket says 172.21.15.44:35680) 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33306/0 (socket says 172.21.15.44:33306) 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd7e00640 1 -- 172.21.15.44:0/2480816974 learned_addr learned my addr 172.21.15.44:0/2480816974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd7400640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd810c800 0x7f4dd81a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd7e00640 1 -- 172.21.15.44:0/2480816974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd810c800 msgr2=0x7f4dd81a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.959+0000 7f4dd7e00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd810c800 0x7f4dd81a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dd7e00640 1 -- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 msgr2=0x7f4dd81a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dd7e00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dd7e00640 1 -- 172.21.15.44:0/2480816974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd81a87b0 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dd7e00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 0x7f4dd81a8170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000eb10 tx=0x7f4dd000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dcfe00640 1 -- 172.21.15.44:0/2480816974 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0004b20 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dcfe00640 1 -- 172.21.15.44:0/2480816974 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4dd0004cc0 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4dcfe00640 1 -- 172.21.15.44:0/2480816974 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00107f0 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd81a8aa0 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.960+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd81b0240 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.963+0000 7f4dcfe00640 1 -- 172.21.15.44:0/2480816974 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4dd0005020 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.964+0000 7f4de1185740 1 Processor -- start 2024-04-20T11:25:32.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.964+0000 7f4de1185740 1 -- start start 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4de1185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4de1185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad23130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4de1185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad23670 0x558f6ad25a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4de1185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f6ac7f050 con 0x558f6ad1a170 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4de1185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f6ac25490 con 0x558f6ad1aa90 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4de1185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f6ac7b010 con 0x558f6ad23670 2024-04-20T11:25:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dcfe00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4dbc0a3d20 0x7f4dbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad23130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad23130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35694/0 (socket says 172.21.15.44:35694) 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dd7400640 1 -- 172.21.15.44:0/132580003 learned_addr learned my addr 172.21.15.44:0/132580003 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dcfe00640 1 -- 172.21.15.44:0/2480816974 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4dd00d0cf0 con 0x7f4dd81a7cc0 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.965+0000 7f4dd7e00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4dbc0a3d20 0x7f4dbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad23670 0x558f6ad25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7400640 1 -- 172.21.15.44:0/132580003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad23670 msgr2=0x558f6ad25a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7400640 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad23670 0x558f6ad25a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7400640 1 -- 172.21.15.44:0/132580003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 msgr2=0x558f6ad1a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7400640 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad1a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7400640 1 -- 172.21.15.44:0/132580003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ac2d070 con 0x558f6ad1aa90 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad23670 0x558f6ad25a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7400640 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad23130 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4dd810c730 tx=0x7f4dc8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d34a7f843621bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dd7e00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4dbc0a3d20 0x7f4dbc0a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00a53e0 tx=0x7f4dd0009200 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.966+0000 7f4dccc00640 1 -- 172.21.15.44:0/132580003 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc802c760 con 0x558f6ad1aa90 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4dccc00640 1 -- 172.21.15.44:0/132580003 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4dc8004040 con 0x558f6ad1aa90 2024-04-20T11:25:32.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4dccc00640 1 -- 172.21.15.44:0/132580003 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80046c0 con 0x558f6ad1aa90 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 -- 172.21.15.44:0/132580003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 msgr2=0x558f6ad23130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad23130 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4dd810c730 tx=0x7f4dc8000bc0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 -- 172.21.15.44:0/132580003 shutdown_connections 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad23670 0x558f6ad25a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad23130 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 --2- 172.21.15.44:0/132580003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad1a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.967+0000 7f4de1185740 1 -- 172.21.15.44:0/132580003 >> 172.21.15.44:0/132580003 conn(0x558f6ad148a0 msgr2=0x558f6ad14c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.968+0000 7f4de1185740 1 -- 172.21.15.44:0/132580003 shutdown_connections 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.968+0000 7f4de1185740 1 -- 172.21.15.44:0/132580003 wait complete. 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.968+0000 7f4de1185740 1 -- start start 2024-04-20T11:25:32.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.968+0000 7f4de1185740 1 Processor -- start 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad20620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad20b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f6ad26400 con 0x558f6ad23670 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f6ac38860 con 0x558f6ad1aa90 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f6ad26580 con 0x558f6ad1a170 2024-04-20T11:25:32.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4de1185740 10 client.0 _ll_register_callbacks cb 0x558f6ac06fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad20b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad20b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35708/0 (socket says 172.21.15.44:35708) 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 -- 172.21.15.44:0/3651207089 learned_addr learned my addr 172.21.15.44:0/3651207089 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad20620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 -- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 msgr2=0x558f6ad20620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad20620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 -- 172.21.15.44:0/3651207089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 msgr2=0x558f6ad210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd7400640 1 -- 172.21.15.44:0/3651207089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6ac3c950 con 0x558f6ad1aa90 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.969+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:25:32.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4dd7400640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad20b60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8036040 tx=0x7f4dc8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abacf1f51ff67dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4dc1600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4dc1600640 1 -- 172.21.15.44:0/3651207089 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc802c9d0 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4dc1600640 1 -- 172.21.15.44:0/3651207089 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4dc802cb70 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc8004040 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4dc1600640 1 -- 172.21.15.44:0/3651207089 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4dc8056050 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4de1185740 10 client.26177 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.970+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558f6ac0a820 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4dc1600640 1 -- 172.21.15.44:0/3651207089 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4dc8004040 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4dc1600640 1 -- 172.21.15.44:0/3651207089 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f4dc8004a20 con 0x558f6ad1aa90 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4de1185740 10 client.26177 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4de1185740 10 client.26177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4de1185740 20 client.26177 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4de1185740 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad333b0 0x558f6ad357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4de1185740 4 client.26177 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:32.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f6ac8ca70 con 0x558f6ad333b0 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.971+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad333b0 0x558f6ad357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.972+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad333b0 0x558f6ad357a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6ed5504d31ff1de server_cookie=c96bedd9f3ecae1d in_seq=0 out_seq=0 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.972+0000 7f4dc1600640 10 client.26177 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4dc1600640 1 -- 172.21.15.44:0/3651207089 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f6ac8ca70 con 0x558f6ad333b0 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4dc1600640 10 client.26177 handle_command_reply: tid=0 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 client.26177 shutdown 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 msgr2=0x558f6ad20b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad20b60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8036040 tx=0x7f4dc8005850 comp rx=0 tx=0).stop 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 shutdown_connections 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad333b0 0x558f6ad357a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad20b60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4dc1600640 0 client.26177 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:25:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 --2- 172.21.15.44:0/3651207089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad20620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 >> 172.21.15.44:0/3651207089 conn(0x558f6ad148a0 msgr2=0x558f6ad165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 shutdown_connections 2024-04-20T11:25:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 1 -- 172.21.15.44:0/3651207089 wait complete. 2024-04-20T11:25:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.973+0000 7f4de1185740 20 client.26177 trim_cache size 0 max 16384 2024-04-20T11:25:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:32 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/731802977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:25:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:32 smithi165 ceph-mon[21975]: pgmap v91: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2024-04-20T11:25:32.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.988+0000 7f4de1185740 1 Processor -- start 2024-04-20T11:25:32.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.988+0000 7f4de1185740 1 -- start start 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4de1185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad22d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4de1185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1aa90 0x558f6ad2f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4de1185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad23670 0x558f6ad3aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4de1185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b8950 con 0x558f6ad1a170 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4de1185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81ba610 con 0x558f6ad23670 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4de1185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b9020 con 0x558f6ad1aa90 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad22d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad22d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33330/0 (socket says 172.21.15.44:33330) 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 -- 172.21.15.44:0/1851588484 learned_addr learned my addr 172.21.15.44:0/1851588484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad23670 0x558f6ad3aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd7400640 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1aa90 0x558f6ad2f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 -- 172.21.15.44:0/1851588484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1aa90 msgr2=0x558f6ad2f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1aa90 0x558f6ad2f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 -- 172.21.15.44:0/1851588484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad23670 msgr2=0x558f6ad3aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.989+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad23670 0x558f6ad3aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.990+0000 7f4dd6a00640 1 -- 172.21.15.44:0/1851588484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6ad1a8c0 con 0x558f6ad1a170 2024-04-20T11:25:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.990+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad22d50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b3d0 tx=0x7f4dc400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6e461d11c27ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.990+0000 7f4dc3e00640 1 -- 172.21.15.44:0/1851588484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4004220 con 0x558f6ad1a170 2024-04-20T11:25:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.990+0000 7f4dc3e00640 1 -- 172.21.15.44:0/1851588484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4dc40043c0 con 0x558f6ad1a170 2024-04-20T11:25:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.990+0000 7f4dc3e00640 1 -- 172.21.15.44:0/1851588484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4004e40 con 0x558f6ad1a170 2024-04-20T11:25:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 -- 172.21.15.44:0/1851588484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 msgr2=0x558f6ad22d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad22d50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b3d0 tx=0x7f4dc400bd50 comp rx=0 tx=0).stop 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 -- 172.21.15.44:0/1851588484 shutdown_connections 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad23670 0x558f6ad3aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1aa90 0x558f6ad2f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 --2- 172.21.15.44:0/1851588484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad1a170 0x558f6ad22d50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 -- 172.21.15.44:0/1851588484 >> 172.21.15.44:0/1851588484 conn(0x558f6ad148a0 msgr2=0x558f6ad1b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 -- 172.21.15.44:0/1851588484 shutdown_connections 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 -- 172.21.15.44:0/1851588484 wait complete. 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.991+0000 7f4de1185740 1 -- start start 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 Processor -- start 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad1d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad1dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b9e80 con 0x558f6ad23670 2024-04-20T11:25:32.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b9750 con 0x558f6ad1aa90 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b68d0 con 0x558f6ad1a170 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4de1185740 10 client.0 _ll_register_callbacks cb 0x7f4dc80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad1dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.992+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad1d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad1d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35732/0 (socket says 172.21.15.44:35732) 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 -- 172.21.15.44:0/3107319965 learned_addr learned my addr 172.21.15.44:0/3107319965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad1dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33332/0 (socket says 172.21.15.44:33332) 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 -- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 msgr2=0x558f6ad1d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad1d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 -- 172.21.15.44:0/3107319965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 msgr2=0x558f6ad1dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad1dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 -- 172.21.15.44:0/3107319965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6a9913f0 con 0x558f6ad1aa90 2024-04-20T11:25:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dd7400640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad1d750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8004e90 tx=0x7f4dc802c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14401e1ba9f11cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dc1600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.993+0000 7f4dc1600640 1 -- 172.21.15.44:0/3107319965 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80051c0 con 0x558f6ad1aa90 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4dc1600640 1 -- 172.21.15.44:0/3107319965 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4dc802cbe0 con 0x558f6ad1aa90 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc40043c0 con 0x558f6ad1aa90 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4dc1600640 1 -- 172.21.15.44:0/3107319965 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4dc8069050 con 0x558f6ad1aa90 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 10 client.26183 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558f6aa5b7d0 con 0x558f6ad1aa90 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4dc1600640 1 -- 172.21.15.44:0/3107319965 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4dc40043c0 con 0x558f6ad1aa90 2024-04-20T11:25:32.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4dc1600640 1 -- 172.21.15.44:0/3107319965 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f4dc80039c0 con 0x558f6ad1aa90 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 10 client.26183 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 10 client.26183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 20 client.26183 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad99e80 0x558f6ad9c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 4 client.26183 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.994+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f6ac868e0 con 0x558f6ad99e80 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.995+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad99e80 0x558f6ad9c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.995+0000 7f4dd6a00640 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad99e80 0x558f6ad9c2a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7aff9134552f9d16 server_cookie=ded7c6ac550d286f in_seq=0 out_seq=0 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.995+0000 7f4dc1600640 10 client.26183 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.995+0000 7f4dc1600640 1 -- 172.21.15.44:0/3107319965 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f6ac868e0 con 0x558f6ad99e80 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.995+0000 7f4dc1600640 10 client.26183 handle_command_reply: tid=0 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.995+0000 7f4de1185740 1 client.26183 shutdown 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 msgr2=0x558f6ad1d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad1d750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8004e90 tx=0x7f4dc802c560 comp rx=0 tx=0).stop 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 shutdown_connections 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558f6ad99e80 0x558f6ad9c2a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4dc1600640 0 client.26183 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f6ad23670 0x558f6ad1dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f6ad1aa90 0x558f6ad1d750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 --2- 172.21.15.44:0/3107319965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f6ad1a170 0x558f6ad1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 >> 172.21.15.44:0/3107319965 conn(0x558f6ad148a0 msgr2=0x558f6ad130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 shutdown_connections 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 1 -- 172.21.15.44:0/3107319965 wait complete. 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.996+0000 7f4de1185740 20 client.26183 trim_cache size 0 max 16384 2024-04-20T11:25:32.999 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:25:33.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4dbc0a3d20 msgr2=0x7f4dbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4dbc0a3d20 0x7f4dbc0a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00a53e0 tx=0x7f4dd0009200 comp rx=0 tx=0).stop 2024-04-20T11:25:33.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 msgr2=0x7f4dd81a8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 0x7f4dd81a8170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000eb10 tx=0x7f4dd000c2d0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 shutdown_connections 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd81a7cc0 0x7f4dd81a8170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4dbc0a3d20 0x7f4dbc0a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd810d1c0 0x7f4dd81a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 --2- 172.21.15.44:0/2480816974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dd810c800 0x7f4dd81a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 >> 172.21.15.44:0/2480816974 conn(0x7f4dd806cde0 msgr2=0x7f4dd810f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 shutdown_connections 2024-04-20T11:25:33.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:32.997+0000 7f4ddce00640 1 -- 172.21.15.44:0/2480816974 wait complete. 2024-04-20T11:25:33.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 -- 172.21.15.44:0/3287422325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4688100db0 msgr2=0x7f4688101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 --2- 172.21.15.44:0/3287422325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4688100db0 0x7f4688101190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4678009860 tx=0x7f467802bc60 comp rx=0 tx=0).stop 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 -- 172.21.15.44:0/3287422325 shutdown_connections 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 --2- 172.21.15.44:0/3287422325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46881061b0 0x7f4688106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 --2- 172.21.15.44:0/3287422325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46881016d0 0x7f4688105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 --2- 172.21.15.44:0/3287422325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4688100db0 0x7f4688101190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 -- 172.21.15.44:0/3287422325 >> 172.21.15.44:0/3287422325 conn(0x7f46880fc7e0 msgr2=0x7f46880fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 -- 172.21.15.44:0/3287422325 shutdown_connections 2024-04-20T11:25:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.227+0000 7f4687e00640 1 -- 172.21.15.44:0/3287422325 wait complete. 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.228+0000 7f4687e00640 1 Processor -- start 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.228+0000 7f4687e00640 1 -- start start 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 0x7f468819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46881016d0 0x7f468819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46881061b0 0x7f46881a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4688118910 con 0x7f46881061b0 2024-04-20T11:25:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4688118790 con 0x7f4688100db0 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4688118a90 con 0x7f46881016d0 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 0x7f468819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 0x7f468819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35736/0 (socket says 172.21.15.44:35736) 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 -- 172.21.15.44:0/1682624276 learned_addr learned my addr 172.21.15.44:0/1682624276 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4687400640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46881061b0 0x7f46881a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686000640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46881016d0 0x7f468819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 -- 172.21.15.44:0/1682624276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46881016d0 msgr2=0x7f468819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46881016d0 0x7f468819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 -- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46881061b0 msgr2=0x7f46881a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46881061b0 0x7f46881a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.229+0000 7f4686a00640 1 -- 172.21.15.44:0/1682624276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46881a3da0 con 0x7f4688100db0 2024-04-20T11:25:33.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.230+0000 7f4686a00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 0x7f468819edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4678037040 tx=0x7f467802c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.230+0000 7f468ca00640 1 -- 172.21.15.44:0/1682624276 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678041020 con 0x7f4688100db0 2024-04-20T11:25:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.230+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46881a4030 con 0x7f4688100db0 2024-04-20T11:25:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.230+0000 7f468ca00640 1 -- 172.21.15.44:0/1682624276 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f467802ce50 con 0x7f4688100db0 2024-04-20T11:25:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.230+0000 7f468ca00640 1 -- 172.21.15.44:0/1682624276 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678039040 con 0x7f4688100db0 2024-04-20T11:25:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.230+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46881ab920 con 0x7f4688100db0 2024-04-20T11:25:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.233+0000 7f468ca00640 1 -- 172.21.15.44:0/1682624276 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4678044020 con 0x7f4688100db0 2024-04-20T11:25:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 Processor -- start 2024-04-20T11:25:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 -- start start 2024-04-20T11:25:33.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a2664130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a2666a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d4a25c0050 con 0x55d4a2664670 2024-04-20T11:25:33.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4a2566490 con 0x55d4a265b170 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.234+0000 7f4690beb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d4a25bc010 con 0x55d4a265ba90 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a2664130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4687400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a2664130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45248/0 (socket says 172.21.15.44:45248) 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 -- 172.21.15.44:0/4081709874 learned_addr learned my addr 172.21.15.44:0/4081709874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686000640 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a2666a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f468ca00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f466c0a3da0 0x7f466c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 -- 172.21.15.44:0/4081709874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 msgr2=0x55d4a265b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 -- 172.21.15.44:0/4081709874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 msgr2=0x55d4a2666a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a2666a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f4686a00640 1 -- 172.21.15.44:0/4081709874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4a256e070 con 0x55d4a265ba90 2024-04-20T11:25:33.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.235+0000 7f468ca00640 1 -- 172.21.15.44:0/1682624276 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f46780f1cc0 con 0x7f4688100db0 2024-04-20T11:25:33.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.236+0000 7f4687400640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f466c0a3da0 0x7f466c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.236+0000 7f4686a00640 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a2664130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46780ef470 tx=0x7f4678088430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4867d924125ae56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.236+0000 7f4673e00640 1 -- 172.21.15.44:0/4081709874 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780bd8c0 con 0x55d4a265ba90 2024-04-20T11:25:33.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.236+0000 7f4673e00640 1 -- 172.21.15.44:0/4081709874 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4678003e50 con 0x55d4a265ba90 2024-04-20T11:25:33.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.236+0000 7f4673e00640 1 -- 172.21.15.44:0/4081709874 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467809cba0 con 0x55d4a265ba90 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 -- 172.21.15.44:0/4081709874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 msgr2=0x55d4a2664130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a2664130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46780ef470 tx=0x7f4678088430 comp rx=0 tx=0).stop 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4687400640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f466c0a3da0 0x7f466c0a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4680005fd0 tx=0x7f4680011040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 -- 172.21.15.44:0/4081709874 shutdown_connections 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a2666a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a2664130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 --2- 172.21.15.44:0/4081709874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 -- 172.21.15.44:0/4081709874 >> 172.21.15.44:0/4081709874 conn(0x55d4a26558a0 msgr2=0x55d4a2655c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 -- 172.21.15.44:0/4081709874 shutdown_connections 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.237+0000 7f4690beb740 1 -- 172.21.15.44:0/4081709874 wait complete. 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.238+0000 7f4690beb740 1 -- start start 2024-04-20T11:25:33.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.238+0000 7f4690beb740 1 Processor -- start 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.238+0000 7f4690beb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265b170 0x55d4a26616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.238+0000 7f4690beb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a2661c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.238+0000 7f4690beb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 0x55d4a2662140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4690beb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d4a2667220 con 0x55d4a265ba90 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4690beb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4a2579860 con 0x55d4a2664670 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4690beb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d4a26673a0 con 0x55d4a265b170 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4690beb740 10 client.0 _ll_register_callbacks cb 0x55d4a2547fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 0x55d4a2662140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4686000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a2661c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 0x55d4a2662140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35762/0 (socket says 172.21.15.44:35762) 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 -- 172.21.15.44:0/3066600618 learned_addr learned my addr 172.21.15.44:0/3066600618 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4686a00640 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265b170 0x55d4a26616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 -- 172.21.15.44:0/3066600618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265b170 msgr2=0x55d4a26616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265b170 0x55d4a26616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 -- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 msgr2=0x55d4a2661c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a2661c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.239+0000 7f4687400640 1 -- 172.21.15.44:0/3066600618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4a257d950 con 0x55d4a2664670 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4687400640 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 0x55d4a2662140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f468000dab0 tx=0x7f46800095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9b65f4481a577f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4670c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4670c00640 1 -- 172.21.15.44:0/3066600618 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468002b060 con 0x55d4a2664670 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4670c00640 1 -- 172.21.15.44:0/3066600618 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f468001bdc0 con 0x55d4a2664670 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4678003e50 con 0x55d4a2664670 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4670c00640 1 -- 172.21.15.44:0/3066600618 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4680030440 con 0x55d4a2664670 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4690beb740 10 client.26195 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.240+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d4a254b820 con 0x55d4a2664670 2024-04-20T11:25:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4670c00640 1 -- 172.21.15.44:0/3066600618 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4678003e50 con 0x55d4a2664670 2024-04-20T11:25:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4670c00640 1 -- 172.21.15.44:0/3066600618 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f468002e020 con 0x55d4a2664670 2024-04-20T11:25:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4690beb740 10 client.26195 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4690beb740 10 client.26195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4690beb740 20 client.26195 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4690beb740 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26744a0 0x55d4a26768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4690beb740 4 client.26195 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.241+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4a25cda70 con 0x55d4a26744a0 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.242+0000 7f4686a00640 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26744a0 0x55d4a26768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.242+0000 7f4686a00640 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26744a0 0x55d4a26768e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21aee6463c658b7 server_cookie=9379fca48d8d97de in_seq=0 out_seq=0 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.242+0000 7f4670c00640 10 client.26195 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4670c00640 1 -- 172.21.15.44:0/3066600618 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4a25cda70 con 0x55d4a26744a0 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4670c00640 10 client.26195 handle_command_reply: tid=0 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 client.26195 shutdown 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 msgr2=0x55d4a2662140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 0x55d4a2662140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f468000dab0 tx=0x7f46800095c0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 shutdown_connections 2024-04-20T11:25:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26744a0 0x55d4a26768e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a2664670 0x55d4a2662140 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a2661c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 --2- 172.21.15.44:0/3066600618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265b170 0x55d4a26616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 >> 172.21.15.44:0/3066600618 conn(0x55d4a26558a0 msgr2=0x55d4a26575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.243+0000 7f4670c00640 0 client.26195 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.244+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 shutdown_connections 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.244+0000 7f4690beb740 1 -- 172.21.15.44:0/3066600618 wait complete. 2024-04-20T11:25:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.244+0000 7f4690beb740 20 client.26195 trim_cache size 0 max 16384 2024-04-20T11:25:33.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.259+0000 7f4690beb740 1 Processor -- start 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.259+0000 7f4690beb740 1 -- start start 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.259+0000 7f4690beb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a2663df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.259+0000 7f4690beb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a266ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.259+0000 7f4690beb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a2664670 0x55d4a26547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4690beb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46881b4090 con 0x55d4a265ba90 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4690beb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46881b5d50 con 0x55d4a265b170 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4690beb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46881b4760 con 0x55d4a2664670 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a2663df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a266ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a2663df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35768/0 (socket says 172.21.15.44:35768) 2024-04-20T11:25:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 -- 172.21.15.44:0/507177564 learned_addr learned my addr 172.21.15.44:0/507177564 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4687400640 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a2664670 0x55d4a26547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 -- 172.21.15.44:0/507177564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a2664670 msgr2=0x55d4a26547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a2664670 0x55d4a26547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 -- 172.21.15.44:0/507177564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 msgr2=0x55d4a266ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a266ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.260+0000 7f4686a00640 1 -- 172.21.15.44:0/507177564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4a22d23f0 con 0x55d4a265b170 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.261+0000 7f4686a00640 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a2663df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4678009980 tx=0x7f467809c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8dcd47f50962ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.261+0000 7f4673400640 1 -- 172.21.15.44:0/507177564 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780c2130 con 0x55d4a265b170 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.261+0000 7f4673400640 1 -- 172.21.15.44:0/507177564 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4678003e50 con 0x55d4a265b170 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.261+0000 7f4673400640 1 -- 172.21.15.44:0/507177564 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780c32c0 con 0x55d4a265b170 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.261+0000 7f4690beb740 1 -- 172.21.15.44:0/507177564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 msgr2=0x55d4a2663df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.261+0000 7f4690beb740 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a2663df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4678009980 tx=0x7f467809c040 comp rx=0 tx=0).stop 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 -- 172.21.15.44:0/507177564 shutdown_connections 2024-04-20T11:25:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a2664670 0x55d4a26547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a265ba90 0x55d4a266ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 --2- 172.21.15.44:0/507177564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a2663df0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 -- 172.21.15.44:0/507177564 >> 172.21.15.44:0/507177564 conn(0x55d4a26558a0 msgr2=0x55d4a26540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 -- 172.21.15.44:0/507177564 shutdown_connections 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 -- 172.21.15.44:0/507177564 wait complete. 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 -- start start 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.262+0000 7f4690beb740 1 Processor -- start 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a265e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a265ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46881b55c0 con 0x55d4a2664670 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46881b4e90 con 0x55d4a265b170 2024-04-20T11:25:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46881b1fd0 con 0x55d4a265ba90 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4690beb740 10 client.0 _ll_register_callbacks cb 0x7f468000d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35770/0 (socket says 172.21.15.44:35770) 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 -- 172.21.15.44:0/3985437996 learned_addr learned my addr 172.21.15.44:0/3985437996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686000640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a265e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4687400640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a265ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 -- 172.21.15.44:0/3985437996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 msgr2=0x55d4a265e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a265e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 -- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 msgr2=0x55d4a265ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a265ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4686a00640 1 -- 172.21.15.44:0/3985437996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4a239c7d0 con 0x55d4a265b170 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.263+0000 7f4687400640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a265ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4686a00640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265e3d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f46780c22f0 tx=0x7f4678004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f100286cc3452 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4670c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4670c00640 1 -- 172.21.15.44:0/3985437996 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780d7060 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4670c00640 1 -- 172.21.15.44:0/3985437996 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f46780bc080 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4678003e50 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4670c00640 1 -- 172.21.15.44:0/3985437996 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f46780bd6e0 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4690beb740 10 client.26201 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.264+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d4a26dabc0 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4670c00640 1 -- 172.21.15.44:0/3985437996 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4678003e50 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4670c00640 1 -- 172.21.15.44:0/3985437996 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f467803c030 con 0x55d4a265b170 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4690beb740 10 client.26201 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4690beb740 10 client.26201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4690beb740 20 client.26201 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4690beb740 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26daf00 0x55d4a26dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4690beb740 4 client.26201 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:25:33.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4a25c78e0 con 0x55d4a26daf00 2024-04-20T11:25:33.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.265+0000 7f4686000640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26daf00 0x55d4a26dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4686000640 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26daf00 0x55d4a26dd2f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4ce320907387c60 server_cookie=3af2773b1d1c751 in_seq=0 out_seq=0 2024-04-20T11:25:33.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4670c00640 10 client.26201 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4670c00640 1 -- 172.21.15.44:0/3985437996 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4a25c78e0 con 0x55d4a26daf00 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4670c00640 10 client.26201 handle_command_reply: tid=0 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 client.26201 shutdown 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 msgr2=0x55d4a265e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265e3d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f46780c22f0 tx=0x7f4678004610 comp rx=0 tx=0).stop 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 shutdown_connections 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d4a26daf00 0x55d4a26dd2f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4670c00640 0 client.26201 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4a2664670 0x55d4a265ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4a265ba90 0x55d4a265e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 --2- 172.21.15.44:0/3985437996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4a265b170 0x55d4a265e3d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.266+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 >> 172.21.15.44:0/3985437996 conn(0x55d4a26558a0 msgr2=0x55d4a26574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.267+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 shutdown_connections 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.267+0000 7f4690beb740 1 -- 172.21.15.44:0/3985437996 wait complete. 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.267+0000 7f4690beb740 20 client.26201 trim_cache size 0 max 16384 2024-04-20T11:25:33.271 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:25:33.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f466c0a3da0 msgr2=0x7f466c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f466c0a3da0 0x7f466c0a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4680005fd0 tx=0x7f4680011040 comp rx=0 tx=0).stop 2024-04-20T11:25:33.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 msgr2=0x7f468819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 0x7f468819edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4678037040 tx=0x7f467802c850 comp rx=0 tx=0).stop 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 shutdown_connections 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f466c0a3da0 0x7f466c0a6260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46881061b0 0x7f46881a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46881016d0 0x7f468819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 --2- 172.21.15.44:0/1682624276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4688100db0 0x7f468819edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 >> 172.21.15.44:0/1682624276 conn(0x7f46880fc7e0 msgr2=0x7f46880fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 shutdown_connections 2024-04-20T11:25:33.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.268+0000 7f4687e00640 1 -- 172.21.15.44:0/1682624276 wait complete. 2024-04-20T11:25:33.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:25:33.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.495+0000 7ff95b000640 1 -- 172.21.15.44:0/3525069644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954107150 msgr2=0x7ff95410ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.495+0000 7ff95b000640 1 --2- 172.21.15.44:0/3525069644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954107150 0x7ff95410ed20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009900 tx=0x7ff94c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 -- 172.21.15.44:0/3525069644 shutdown_connections 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 --2- 172.21.15.44:0/3525069644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954107150 0x7ff95410ed20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 --2- 172.21.15.44:0/3525069644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff954102a30 0x7ff954106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 --2- 172.21.15.44:0/3525069644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff954102080 0x7ff954102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 -- 172.21.15.44:0/3525069644 >> 172.21.15.44:0/3525069644 conn(0x7ff9540fd900 msgr2=0x7ff9540ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 -- 172.21.15.44:0/3525069644 shutdown_connections 2024-04-20T11:25:33.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.496+0000 7ff95b000640 1 -- 172.21.15.44:0/3525069644 wait complete. 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.497+0000 7ff95b000640 1 Processor -- start 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.497+0000 7ff95b000640 1 -- start start 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 0x7ff9541a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff954107150 0x7ff9541a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95419a7f0 0x7ff95419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541122f0 con 0x7ff95419a7f0 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff954112170 con 0x7ff954102080 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff954112470 con 0x7ff954107150 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff959c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 0x7ff9541a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff959c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 0x7ff9541a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35788/0 (socket says 172.21.15.44:35788) 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff959c00640 1 -- 172.21.15.44:0/3919831590 learned_addr learned my addr 172.21.15.44:0/3919831590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff95a600640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95419a7f0 0x7ff95419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff959200640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff954107150 0x7ff9541a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff959c00640 1 -- 172.21.15.44:0/3919831590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff954107150 msgr2=0x7ff9541a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.498+0000 7ff959c00640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff954107150 0x7ff9541a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff959c00640 1 -- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95419a7f0 msgr2=0x7ff95419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff959c00640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95419a7f0 0x7ff95419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff959c00640 1 -- 172.21.15.44:0/3919831590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95419b380 con 0x7ff954102080 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff959c00640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 0x7ff9541a0680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff94800eb90 tx=0x7ff94800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff952000640 1 -- 172.21.15.44:0/3919831590 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948004d20 con 0x7ff954102080 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff952000640 1 -- 172.21.15.44:0/3919831590 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94800ee90 con 0x7ff954102080 2024-04-20T11:25:33.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95419b670 con 0x7ff954102080 2024-04-20T11:25:33.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff952000640 1 -- 172.21.15.44:0/3919831590 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9480108b0 con 0x7ff954102080 2024-04-20T11:25:33.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.499+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a7800 con 0x7ff954102080 2024-04-20T11:25:33.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 Processor -- start 2024-04-20T11:25:33.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 -- start start 2024-04-20T11:25:33.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff952000640 1 -- 172.21.15.44:0/3919831590 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff948005020 con 0x7ff954102080 2024-04-20T11:25:33.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 0x556ab1a06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06a90 0x556ab1a0f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a11a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ab196b050 con 0x556ab1a06a90 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ab1911490 con 0x556ab1a06170 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95f404740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556ab1967010 con 0x556ab1a0f670 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 0x556ab1a06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 0x556ab1a06550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35800/0 (socket says 172.21.15.44:35800) 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.503+0000 7ff95a600640 1 -- 172.21.15.44:0/1909303432 learned_addr learned my addr 172.21.15.44:0/1909303432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff959200640 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a11a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff959c00640 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06a90 0x556ab1a0f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff95a600640 1 -- 172.21.15.44:0/1909303432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 msgr2=0x556ab1a11a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff95a600640 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a11a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff95a600640 1 -- 172.21.15.44:0/1909303432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06a90 msgr2=0x556ab1a0f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff95a600640 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06a90 0x556ab1a0f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff95a600640 1 -- 172.21.15.44:0/1909303432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab1919070 con 0x556ab1a06170 2024-04-20T11:25:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.504+0000 7ff95a600640 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 0x556ab1a06550 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009230 tx=0x7ff94c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91535d832af5717a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.505+0000 7ff952000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff93c0a3d20 0x7ff93c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.505+0000 7ff946a00640 1 -- 172.21.15.44:0/1909303432 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c02c710 con 0x556ab1a06170 2024-04-20T11:25:33.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.505+0000 7ff946a00640 1 -- 172.21.15.44:0/1909303432 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94c004040 con 0x556ab1a06170 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.505+0000 7ff946a00640 1 -- 172.21.15.44:0/1909303432 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0055b0 con 0x556ab1a06170 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.505+0000 7ff952000640 1 -- 172.21.15.44:0/3919831590 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff9480d0e70 con 0x7ff954102080 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.505+0000 7ff959c00640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff93c0a3d20 0x7ff93c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 -- 172.21.15.44:0/1909303432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 msgr2=0x556ab1a06550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 0x556ab1a06550 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009230 tx=0x7ff94c009560 comp rx=0 tx=0).stop 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff959c00640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff93c0a3d20 0x7ff93c0a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ce6e0 tx=0x7ff9480d49c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 -- 172.21.15.44:0/1909303432 shutdown_connections 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a11a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06a90 0x556ab1a0f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 --2- 172.21.15.44:0/1909303432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06170 0x556ab1a06550 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 -- 172.21.15.44:0/1909303432 >> 172.21.15.44:0/1909303432 conn(0x556ab1a008a0 msgr2=0x556ab1a00c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 -- 172.21.15.44:0/1909303432 shutdown_connections 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 -- 172.21.15.44:0/1909303432 wait complete. 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.506+0000 7ff95f404740 1 -- start start 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 Processor -- start 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a0cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556ab1a12400 con 0x556ab1a06170 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ab1924860 con 0x556ab1a06a90 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556ab1a12580 con 0x556ab1a0f670 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.507+0000 7ff95f404740 10 client.0 _ll_register_callbacks cb 0x556ab18f2fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a0cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a0cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35802/0 (socket says 172.21.15.44:35802) 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 -- 172.21.15.44:0/1224844974 learned_addr learned my addr 172.21.15.44:0/1224844974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff95a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 -- 172.21.15.44:0/1224844974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 msgr2=0x556ab1a0d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 -- 172.21.15.44:0/1224844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 msgr2=0x556ab1a0c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 -- 172.21.15.44:0/1224844974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab1928950 con 0x556ab1a06a90 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff959200640 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a0cc10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff940012730 tx=0x7ff940012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df9f51222a959158 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.508+0000 7ff93b400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff93b400640 1 -- 172.21.15.44:0/1224844974 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940004300 con 0x556ab1a06a90 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff93b400640 1 -- 172.21.15.44:0/1224844974 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff9400119a0 con 0x556ab1a06a90 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94c004040 con 0x556ab1a06a90 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff93b400640 1 -- 172.21.15.44:0/1224844974 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff9400044a0 con 0x556ab1a06a90 2024-04-20T11:25:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff95f404740 10 client.26213 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ab18f6820 con 0x556ab1a06a90 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff93b400640 1 -- 172.21.15.44:0/1224844974 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94c004040 con 0x556ab1a06a90 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff93b400640 1 -- 172.21.15.44:0/1224844974 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7ff940024020 con 0x556ab1a06a90 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff95f404740 10 client.26213 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff95f404740 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.509+0000 7ff95f404740 20 client.26213 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.510+0000 7ff95f404740 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a21a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.510+0000 7ff95f404740 4 client.26213 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.510+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab1978a70 con 0x556ab1a1f630 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.510+0000 7ff95a600640 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a21a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.510+0000 7ff95a600640 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a21a80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbe9ed48461fd45 server_cookie=56467d3a02763fdc in_seq=0 out_seq=0 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.510+0000 7ff93b400640 10 client.26213 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.511+0000 7ff93b400640 1 -- 172.21.15.44:0/1224844974 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ab1978a70 con 0x556ab1a1f630 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.511+0000 7ff93b400640 10 client.26213 handle_command_reply: tid=0 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.511+0000 7ff95f404740 1 client.26213 shutdown 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 msgr2=0x556ab1a0cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a0cc10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff940012730 tx=0x7ff940012bf0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 shutdown_connections 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a21a80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a0d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff93b400640 0 client.26213 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a0cc10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 --2- 172.21.15.44:0/1224844974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 >> 172.21.15.44:0/1224844974 conn(0x556ab1a008a0 msgr2=0x556ab1a025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 shutdown_connections 2024-04-20T11:25:33.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 1 -- 172.21.15.44:0/1224844974 wait complete. 2024-04-20T11:25:33.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.512+0000 7ff95f404740 20 client.26213 trim_cache size 0 max 16384 2024-04-20T11:25:33.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 Processor -- start 2024-04-20T11:25:33.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 -- start start 2024-04-20T11:25:33.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a1b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a27390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541afc50 con 0x556ab1a06170 2024-04-20T11:25:33.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b1910 con 0x556ab1a06a90 2024-04-20T11:25:33.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95f404740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b0320 con 0x556ab1a0f670 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff95a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff959200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a1b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff959200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a1b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35808/0 (socket says 172.21.15.44:35808) 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.530+0000 7ff959200640 1 -- 172.21.15.44:0/3430431000 learned_addr learned my addr 172.21.15.44:0/3430431000 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff95a600640 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a27390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff959200640 1 -- 172.21.15.44:0/3430431000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 msgr2=0x556ab1a27390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff959200640 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a27390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff959200640 1 -- 172.21.15.44:0/3430431000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 msgr2=0x556ab1a0ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff959200640 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff959200640 1 -- 172.21.15.44:0/3430431000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab1a0b680 con 0x556ab1a06a90 2024-04-20T11:25:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff95a600640 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a27390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:25:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff959200640 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a1b2c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff940026700 tx=0x7ff940009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c976dd470f9aaaaa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff946000640 1 -- 172.21.15.44:0/3430431000 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940026780 con 0x556ab1a06a90 2024-04-20T11:25:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff946000640 1 -- 172.21.15.44:0/3430431000 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94002b070 con 0x556ab1a06a90 2024-04-20T11:25:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.531+0000 7ff946000640 1 -- 172.21.15.44:0/3430431000 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940005330 con 0x556ab1a06a90 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 -- 172.21.15.44:0/3430431000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 msgr2=0x556ab1a1b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a1b2c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff940026700 tx=0x7ff940009fd0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 -- 172.21.15.44:0/3430431000 shutdown_connections 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a27390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a1b2c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 --2- 172.21.15.44:0/3430431000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a0ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 -- 172.21.15.44:0/3430431000 >> 172.21.15.44:0/3430431000 conn(0x556ab1a008a0 msgr2=0x556ab19ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 -- 172.21.15.44:0/3430431000 shutdown_connections 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.532+0000 7ff95f404740 1 -- 172.21.15.44:0/3430431000 wait complete. 2024-04-20T11:25:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 -- start start 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 Processor -- start 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a13de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a14320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a7dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b1180 con 0x556ab1a06170 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541b0a50 con 0x556ab1a06a90 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541adb90 con 0x556ab1a0f670 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95f404740 10 client.0 _ll_register_callbacks cb 0x7ff940009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.533+0000 7ff95a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a13de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a14320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff95a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a13de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33430/0 (socket says 172.21.15.44:33430) 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff95a600640 1 -- 172.21.15.44:0/2100212072 learned_addr learned my addr 172.21.15.44:0/2100212072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:33.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff95a600640 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a7dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 -- 172.21.15.44:0/2100212072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 msgr2=0x556ab1a7dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a7dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 -- 172.21.15.44:0/2100212072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 msgr2=0x556ab1a13de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a13de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 -- 172.21.15.44:0/2100212072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab167d3f0 con 0x556ab1a06a90 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff959200640 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a14320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x556ab1a21d10 tx=0x7ff940026d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577798777b98995c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff93b400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.534+0000 7ff93b400640 1 -- 172.21.15.44:0/2100212072 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940011bb0 con 0x556ab1a06a90 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94002b070 con 0x556ab1a06a90 2024-04-20T11:25:33.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff93b400640 1 -- 172.21.15.44:0/2100212072 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff940011d50 con 0x556ab1a06a90 2024-04-20T11:25:33.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff93b400640 1 -- 172.21.15.44:0/2100212072 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff9400050b0 con 0x556ab1a06a90 2024-04-20T11:25:33.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 10 client.26219 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ab17477d0 con 0x556ab1a06a90 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff93b400640 1 -- 172.21.15.44:0/2100212072 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94002b070 con 0x556ab1a06a90 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff93b400640 1 -- 172.21.15.44:0/2100212072 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7ff940041020 con 0x556ab1a06a90 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 10 client.26219 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 20 client.26219 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 4 client.26219 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.535+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ab19728e0 con 0x556ab1a1f630 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.536+0000 7ff95a600640 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.536+0000 7ff95a600640 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a889a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1783aeb32eb0257 server_cookie=c4f9230aecf61cb7 in_seq=0 out_seq=0 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.536+0000 7ff93b400640 10 client.26219 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:33.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff93b400640 1 -- 172.21.15.44:0/2100212072 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ab19728e0 con 0x556ab1a1f630 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff93b400640 10 client.26219 handle_command_reply: tid=0 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 client.26219 shutdown 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 msgr2=0x556ab1a14320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a14320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x556ab1a21d10 tx=0x7ff940026d40 comp rx=0 tx=0).stop 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 shutdown_connections 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556ab1a1f630 0x556ab1a889a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ab1a0f670 0x556ab1a7dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ab1a06a90 0x556ab1a14320 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff93b400640 0 client.26219 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 --2- 172.21.15.44:0/2100212072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ab1a06170 0x556ab1a13de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 >> 172.21.15.44:0/2100212072 conn(0x556ab1a008a0 msgr2=0x556ab1a0c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 shutdown_connections 2024-04-20T11:25:33.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 1 -- 172.21.15.44:0/2100212072 wait complete. 2024-04-20T11:25:33.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.537+0000 7ff95f404740 20 client.26219 trim_cache size 0 max 16384 2024-04-20T11:25:33.541 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff93c0a3d20 msgr2=0x7ff93c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff93c0a3d20 0x7ff93c0a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ce6e0 tx=0x7ff9480d49c0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 msgr2=0x7ff9541a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 0x7ff9541a0680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff94800eb90 tx=0x7ff94800c6a0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 shutdown_connections 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff93c0a3d20 0x7ff93c0a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95419a7f0 0x7ff95419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff954107150 0x7ff9541a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 --2- 172.21.15.44:0/3919831590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff954102080 0x7ff9541a0680 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:33.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.538+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 >> 172.21.15.44:0/3919831590 conn(0x7ff9540fd900 msgr2=0x7ff95410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:33.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.539+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 shutdown_connections 2024-04-20T11:25:33.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:33.539+0000 7ff95b000640 1 -- 172.21.15.44:0/3919831590 wait complete. 2024-04-20T11:25:34.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:25:34.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:25:34.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:25:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 -- 172.21.15.44:0/1339451172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c104280 msgr2=0x7fdb5c104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 --2- 172.21.15.44:0/1339451172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c104280 0x7fdb5c104660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009900 tx=0x7fdb4c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:25:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 -- 172.21.15.44:0/1339451172 shutdown_connections 2024-04-20T11:25:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 --2- 172.21.15.44:0/1339451172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb5c109350 0x7fdb5c110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 --2- 172.21.15.44:0/1339451172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c104c30 0x7fdb5c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 --2- 172.21.15.44:0/1339451172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c104280 0x7fdb5c104660 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.780+0000 7fdb60a00640 1 -- 172.21.15.44:0/1339451172 >> 172.21.15.44:0/1339451172 conn(0x7fdb5c0ffaa0 msgr2=0x7fdb5c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.781+0000 7fdb60a00640 1 -- 172.21.15.44:0/1339451172 shutdown_connections 2024-04-20T11:25:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.781+0000 7fdb60a00640 1 -- 172.21.15.44:0/1339451172 wait complete. 2024-04-20T11:25:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 Processor -- start 2024-04-20T11:25:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 -- start start 2024-04-20T11:25:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb5c104280 0x7fdb5c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c109350 0x7fdb5c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 0x7fdb5c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1144d0 con 0x7fdb5c109350 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c114350 con 0x7fdb5c19a850 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.782+0000 7fdb60a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c114650 con 0x7fdb5c104280 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 0x7fdb5c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c109350 0x7fdb5c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 0x7fdb5c19ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35844/0 (socket says 172.21.15.44:35844) 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 -- 172.21.15.44:0/3972722214 learned_addr learned my addr 172.21.15.44:0/3972722214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb5c104280 0x7fdb5c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 -- 172.21.15.44:0/3972722214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb5c104280 msgr2=0x7fdb5c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb5c104280 0x7fdb5c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 -- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c109350 msgr2=0x7fdb5c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c109350 0x7fdb5c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 -- 172.21.15.44:0/3972722214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c19b3e0 con 0x7fdb5c19a850 2024-04-20T11:25:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.783+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 0x7fdb5c19ad00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009fd0 tx=0x7fdb5400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.784+0000 7fdb53e00640 1 -- 172.21.15.44:0/3972722214 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54017070 con 0x7fdb5c19a850 2024-04-20T11:25:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.784+0000 7fdb53e00640 1 -- 172.21.15.44:0/3972722214 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb54004020 con 0x7fdb5c19a850 2024-04-20T11:25:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.784+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c19b6d0 con 0x7fdb5c19a850 2024-04-20T11:25:34.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.784+0000 7fdb53e00640 1 -- 172.21.15.44:0/3972722214 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540106c0 con 0x7fdb5c19a850 2024-04-20T11:25:34.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.784+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a78b0 con 0x7fdb5c19a850 2024-04-20T11:25:34.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.787+0000 7fdb64d76740 1 Processor -- start 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.787+0000 7fdb53e00640 1 -- 172.21.15.44:0/3972722214 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdb540041d0 con 0x7fdb5c19a850 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.787+0000 7fdb64d76740 1 -- start start 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e85ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e8678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 0x55d32e86a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb64d76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d32e7bad20 con 0x55d32e85e900 2024-04-20T11:25:34.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb64d76740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d32e7613e0 con 0x55d32e85f220 2024-04-20T11:25:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb64d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d32e7b6ce0 con 0x55d32e867e00 2024-04-20T11:25:34.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 0x55d32e86a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.788+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 0x55d32e86a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45366/0 (socket says 172.21.15.44:45366) 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1526263094 learned_addr learned my addr 172.21.15.44:0/1526263094 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e85ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5b400640 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e8678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1526263094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 msgr2=0x55d32e8678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e8678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1526263094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 msgr2=0x55d32e85ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e85ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1526263094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32e780980 con 0x55d32e867e00 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb5b400640 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e8678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:25:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.789+0000 7fdb53e00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb400a3da0 0x7fdb400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.790+0000 7fdb53e00640 1 -- 172.21.15.44:0/3972722214 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fdb540d0390 con 0x7fdb5c19a850 2024-04-20T11:25:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.790+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb400a3da0 0x7fdb400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.790+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 0x55d32e86a1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009fc0 tx=0x7fdb48005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ccbd9eda3732e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.791+0000 7fdb47e00640 1 -- 172.21.15.44:0/1526263094 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48004030 con 0x55d32e867e00 2024-04-20T11:25:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.791+0000 7fdb47e00640 1 -- 172.21.15.44:0/1526263094 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb48005520 con 0x55d32e867e00 2024-04-20T11:25:34.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.791+0000 7fdb47e00640 1 -- 172.21.15.44:0/1526263094 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480035f0 con 0x55d32e867e00 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 -- 172.21.15.44:0/1526263094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 msgr2=0x55d32e86a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 0x55d32e86a1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009fc0 tx=0x7fdb48005850 comp rx=0 tx=0).stop 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb400a3da0 0x7fdb400a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c006010 tx=0x7fdb4c005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 -- 172.21.15.44:0/1526263094 shutdown_connections 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e867e00 0x55d32e86a1f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e8678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1526263094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e85ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 -- 172.21.15.44:0/1526263094 >> 172.21.15.44:0/1526263094 conn(0x55d32e859030 msgr2=0x55d32e859410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 -- 172.21.15.44:0/1526263094 shutdown_connections 2024-04-20T11:25:34.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.792+0000 7fdb64d76740 1 -- 172.21.15.44:0/1526263094 wait complete. 2024-04-20T11:25:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.793+0000 7fdb64d76740 1 -- start start 2024-04-20T11:25:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.793+0000 7fdb64d76740 1 Processor -- start 2024-04-20T11:25:34.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.793+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e864eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e8653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e865930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb64d76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d32e86abf0 con 0x55d32e85e900 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb64d76740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d32e7747a0 con 0x55d32e867e00 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb64d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d32e86ad70 con 0x55d32e85f220 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb64d76740 10 client.0 _ll_register_callbacks cb 0x55d32e776b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e865930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e865930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35864/0 (socket says 172.21.15.44:35864) 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 -- 172.21.15.44:0/3641678346 learned_addr learned my addr 172.21.15.44:0/3641678346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e864eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e8653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 -- 172.21.15.44:0/3641678346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 msgr2=0x55d32e8653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e8653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 -- 172.21.15.44:0/3641678346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 msgr2=0x55d32e864eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e864eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.794+0000 7fdb5be00640 1 -- 172.21.15.44:0/3641678346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32e5957e0 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e865930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb54098dc0 tx=0x7fdb54005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b592be4a13cda6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb44c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb44c00640 1 -- 172.21.15.44:0/3641678346 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54020070 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb44c00640 1 -- 172.21.15.44:0/3641678346 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fdb54098a60 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb48005520 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb44c00640 1 -- 172.21.15.44:0/3641678346 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb540a4fb0 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb64d76740 10 client.26228 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d32e5f5f70 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb44c00640 1 -- 172.21.15.44:0/3641678346 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb48005520 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.795+0000 7fdb44c00640 1 -- 172.21.15.44:0/3641678346 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7fdb54098340 con 0x55d32e867e00 2024-04-20T11:25:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb64d76740 10 client.26228 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb64d76740 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb64d76740 20 client.26228 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8782a0 0x55d32e87a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb64d76740 4 client.26228 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d32e7c8740 con 0x55d32e8782a0 2024-04-20T11:25:34.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8782a0 0x55d32e87a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8782a0 0x55d32e87a720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc624b1c00f7ab6 server_cookie=2f909864c1ae9 in_seq=0 out_seq=0 2024-04-20T11:25:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.796+0000 7fdb44c00640 10 client.26228 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb44c00640 1 -- 172.21.15.44:0/3641678346 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d32e7c8740 con 0x55d32e8782a0 2024-04-20T11:25:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb44c00640 10 client.26228 handle_command_reply: tid=0 2024-04-20T11:25:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 client.26228 shutdown 2024-04-20T11:25:34.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 msgr2=0x55d32e865930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e865930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb54098dc0 tx=0x7fdb54005b90 comp rx=0 tx=0).stop 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 shutdown_connections 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8782a0 0x55d32e87a720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e865930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e8653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3641678346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e864eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb44c00640 0 client.26228 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 >> 172.21.15.44:0/3641678346 conn(0x55d32e859030 msgr2=0x55d32e85a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 shutdown_connections 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 1 -- 172.21.15.44:0/3641678346 wait complete. 2024-04-20T11:25:34.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.798+0000 7fdb64d76740 20 client.26228 trim_cache size 0 max 16384 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.820+0000 7fdb64d76740 1 Processor -- start 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.820+0000 7fdb64d76740 1 -- start start 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.820+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e8675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.820+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e873ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e87fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb64d76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ab910 con 0x55d32e85e900 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb64d76740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ad5d0 con 0x55d32e867e00 2024-04-20T11:25:34.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb64d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1abfe0 con 0x55d32e85f220 2024-04-20T11:25:34.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e8675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e87fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e87fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35872/0 (socket says 172.21.15.44:35872) 2024-04-20T11:25:34.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e8675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33466/0 (socket says 172.21.15.44:33466) 2024-04-20T11:25:34.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 -- 172.21.15.44:0/3587947903 learned_addr learned my addr 172.21.15.44:0/3587947903 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:34.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e873ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 -- 172.21.15.44:0/3587947903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 msgr2=0x55d32e873ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e873ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 -- 172.21.15.44:0/3587947903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 msgr2=0x55d32e8675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e8675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5be00640 1 -- 172.21.15.44:0/3587947903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32e85f050 con 0x55d32e867e00 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.821+0000 7fdb5b400640 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e8675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.822+0000 7fdb5be00640 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e87fff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb5409c380 tx=0x7fdb5400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13974a76150d605 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.822+0000 7fdb47400640 1 -- 172.21.15.44:0/3587947903 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540b9030 con 0x55d32e867e00 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.822+0000 7fdb47400640 1 -- 172.21.15.44:0/3587947903 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb540bb2c0 con 0x55d32e867e00 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.822+0000 7fdb47400640 1 -- 172.21.15.44:0/3587947903 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54020070 con 0x55d32e867e00 2024-04-20T11:25:34.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 -- 172.21.15.44:0/3587947903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 msgr2=0x55d32e87fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e87fff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb5409c380 tx=0x7fdb5400bea0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 -- 172.21.15.44:0/3587947903 shutdown_connections 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e867e00 0x55d32e87fff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85f220 0x55d32e873ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 --2- 172.21.15.44:0/3587947903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e85e900 0x55d32e8675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 -- 172.21.15.44:0/3587947903 >> 172.21.15.44:0/3587947903 conn(0x55d32e859030 msgr2=0x55d32e85acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 -- 172.21.15.44:0/3587947903 shutdown_connections 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 -- 172.21.15.44:0/3587947903 wait complete. 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 -- start start 2024-04-20T11:25:34.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.823+0000 7fdb64d76740 1 Processor -- start 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85e900 0x55d32e86c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e86c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e867e00 0x55d32e8dd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ace40 con 0x55d32e867e00 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ac710 con 0x55d32e85f220 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1a9890 con 0x55d32e85e900 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb64d76740 10 client.0 _ll_register_callbacks cb 0x7fdb540044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e86c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb5aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e86c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35880/0 (socket says 172.21.15.44:35880) 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1139723350 learned_addr learned my addr 172.21.15.44:0/1139723350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb5be00640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e867e00 0x55d32e8dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.824+0000 7fdb5b400640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85e900 0x55d32e86c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1139723350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85e900 msgr2=0x55d32e86c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85e900 0x55d32e86c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1139723350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e867e00 msgr2=0x55d32e8dd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e867e00 0x55d32e8dd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb5aa00640 1 -- 172.21.15.44:0/1139723350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32e48d170 con 0x55d32e85f220 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb5aa00640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e86c6b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55d32e85a1c0 tx=0x7fdb4801e460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f736ca40ff598981 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb44c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb44c00640 1 -- 172.21.15.44:0/1139723350 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4802f060 con 0x55d32e85f220 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb44c00640 1 -- 172.21.15.44:0/1139723350 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fdb4801e9e0 con 0x55d32e85f220 2024-04-20T11:25:34.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.825+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d32e7c25b0 con 0x55d32e85f220 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb44c00640 1 -- 172.21.15.44:0/1139723350 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb4801ebe0 con 0x55d32e85f220 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 10 client.26234 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d32e8def60 con 0x55d32e85f220 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb44c00640 1 -- 172.21.15.44:0/1139723350 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55d32e7c25b0 con 0x55d32e85f220 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb44c00640 1 -- 172.21.15.44:0/1139723350 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7fdb48032020 con 0x55d32e85f220 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 10 client.26234 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 20 client.26234 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8df440 0x55d32e8e1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 4 client.26234 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.826+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d32e73eb10 con 0x55d32e8df440 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.827+0000 7fdb5b400640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8df440 0x55d32e8e1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.827+0000 7fdb5b400640 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8df440 0x55d32e8e1860 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ddd3c7fb7eb186a server_cookie=d3c75dd6bca98edf in_seq=0 out_seq=0 2024-04-20T11:25:34.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.827+0000 7fdb44c00640 10 client.26234 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb44c00640 1 -- 172.21.15.44:0/1139723350 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d32e73eb10 con 0x55d32e8df440 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb44c00640 10 client.26234 handle_command_reply: tid=0 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 client.26234 shutdown 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 msgr2=0x55d32e86c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e86c6b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55d32e85a1c0 tx=0x7fdb4801e460 comp rx=0 tx=0).stop 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 shutdown_connections 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d32e8df440 0x55d32e8e1860 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d32e867e00 0x55d32e8dd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d32e85f220 0x55d32e86c6b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 --2- 172.21.15.44:0/1139723350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d32e85e900 0x55d32e86c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb44c00640 0 client.26234 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.828+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 >> 172.21.15.44:0/1139723350 conn(0x55d32e859030 msgr2=0x55d32e864e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.829+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 shutdown_connections 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.829+0000 7fdb64d76740 1 -- 172.21.15.44:0/1139723350 wait complete. 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.829+0000 7fdb64d76740 20 client.26234 trim_cache size 0 max 16384 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "365d2521-4343-4d4c-ac7c-9437ef721c3d", 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:25:34.833 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb400a3da0 msgr2=0x7fdb400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb400a3da0 0x7fdb400a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c006010 tx=0x7fdb4c005e40 comp rx=0 tx=0).stop 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 msgr2=0x7fdb5c19ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 0x7fdb5c19ad00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009fd0 tx=0x7fdb5400b260 comp rx=0 tx=0).stop 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 shutdown_connections 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c19a850 0x7fdb5c19ad00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb400a3da0 0x7fdb400a6260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c109350 0x7fdb5c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 --2- 172.21.15.44:0/3972722214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb5c104280 0x7fdb5c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 >> 172.21.15.44:0/3972722214 conn(0x7fdb5c0ffaa0 msgr2=0x7fdb5c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 shutdown_connections 2024-04-20T11:25:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:34.830+0000 7fdb60a00640 1 -- 172.21.15.44:0/3972722214 wait complete. 2024-04-20T11:25:34.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:25:34.850 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 -- 172.21.15.44:0/98636754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f401810d3d0 msgr2=0x7f401810d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 --2- 172.21.15.44:0/98636754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f401810d3d0 0x7f401810d7b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4010009ca0 tx=0x7f401002c180 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 -- 172.21.15.44:0/98636754 shutdown_connections 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 --2- 172.21.15.44:0/98636754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f40181152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 --2- 172.21.15.44:0/98636754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4018108500 0x7f4018108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 --2- 172.21.15.44:0/98636754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f401810d3d0 0x7f401810d7b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 -- 172.21.15.44:0/98636754 >> 172.21.15.44:0/98636754 conn(0x7f4018077d80 msgr2=0x7f4018106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 -- 172.21.15.44:0/98636754 shutdown_connections 2024-04-20T11:25:35.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.061+0000 7f401e400640 1 -- 172.21.15.44:0/98636754 wait complete. 2024-04-20T11:25:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.062+0000 7f401e400640 1 Processor -- start 2024-04-20T11:25:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.062+0000 7f401e400640 1 -- start start 2024-04-20T11:25:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4018108500 0x7f401819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f401819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401810d3d0 0x7f40181a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4018118950 con 0x7f401810d3d0 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401e400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40181187d0 con 0x7f4018108ea0 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4018118ad0 con 0x7f4018108500 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f401819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401810d3d0 0x7f40181a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f401819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35894/0 (socket says 172.21.15.44:35894) 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 -- 172.21.15.44:0/3239297894 learned_addr learned my addr 172.21.15.44:0/3239297894 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401d000640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4018108500 0x7f401819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 -- 172.21.15.44:0/3239297894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4018108500 msgr2=0x7f401819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4018108500 0x7f401819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 -- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401810d3d0 msgr2=0x7f40181a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401810d3d0 0x7f40181a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f4017e00640 1 -- 172.21.15.44:0/3239297894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40181a3dc0 con 0x7f4018108ea0 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.063+0000 7f401d000640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4018108500 0x7f401819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.064+0000 7f4017e00640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f401819f310 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f400400e9f0 tx=0x7f400400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.064+0000 7f4015600640 1 -- 172.21.15.44:0/3239297894 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400400cd80 con 0x7f4018108ea0 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.064+0000 7f4015600640 1 -- 172.21.15.44:0/3239297894 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4004010930 con 0x7f4018108ea0 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.064+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40181a40b0 con 0x7f4018108ea0 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.064+0000 7f4015600640 1 -- 172.21.15.44:0/3239297894 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004018b00 con 0x7f4018108ea0 2024-04-20T11:25:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.064+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40181ab940 con 0x7f4018108ea0 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.068+0000 7f4015600640 1 -- 172.21.15.44:0/3239297894 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4004018ca0 con 0x7f4018108ea0 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.068+0000 7f40227af740 1 Processor -- start 2024-04-20T11:25:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.068+0000 7f40227af740 1 -- start start 2024-04-20T11:25:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.068+0000 7f40227af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b463a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.068+0000 7f40227af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4643340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f40227af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4645c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f40227af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e3b45a0150 con 0x55e3b463a380 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f40227af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3b4546370 con 0x55e3b463aca0 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f40227af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e3b459c110 con 0x55e3b4643880 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f401da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b463a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f401d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4643340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f401da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b463a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33490/0 (socket says 172.21.15.44:33490) 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f401da00640 1 -- 172.21.15.44:0/3159024190 learned_addr learned my addr 172.21.15.44:0/3159024190 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.069+0000 7f4017e00640 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4645c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401da00640 1 -- 172.21.15.44:0/3159024190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 msgr2=0x55e3b4645c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401da00640 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4645c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401da00640 1 -- 172.21.15.44:0/3159024190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 msgr2=0x55e3b4643340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401da00640 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4643340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401da00640 1 -- 172.21.15.44:0/3159024190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b437c7d0 con 0x55e3b463a380 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401d000640 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4643340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f401da00640 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b463a760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4008009c10 tx=0x7f4008011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb44baa58d992b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f4015600640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ffc0a3d20 0x7f3ffc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f400e000640 1 -- 172.21.15.44:0/3159024190 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400801e070 con 0x55e3b463a380 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.070+0000 7f4015600640 1 -- 172.21.15.44:0/3239297894 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f40040d0b40 con 0x7f4018108ea0 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.071+0000 7f400e000640 1 -- 172.21.15.44:0/3159024190 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4008019b30 con 0x55e3b463a380 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.071+0000 7f400e000640 1 -- 172.21.15.44:0/3159024190 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400800ea90 con 0x55e3b463a380 2024-04-20T11:25:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.071+0000 7f401d000640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ffc0a3d20 0x7f3ffc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 -- 172.21.15.44:0/3159024190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 msgr2=0x55e3b463a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b463a760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4008009c10 tx=0x7f4008011810 comp rx=0 tx=0).stop 2024-04-20T11:25:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 -- 172.21.15.44:0/3159024190 shutdown_connections 2024-04-20T11:25:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4645c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4643340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 --2- 172.21.15.44:0/3159024190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b463a760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 -- 172.21.15.44:0/3159024190 >> 172.21.15.44:0/3159024190 conn(0x55e3b4634ab0 msgr2=0x55e3b4634e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 -- 172.21.15.44:0/3159024190 shutdown_connections 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f40227af740 1 -- 172.21.15.44:0/3159024190 wait complete. 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.072+0000 7f401d000640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ffc0a3d20 0x7f3ffc0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4010000f80 tx=0x7f4010009920 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 -- start start 2024-04-20T11:25:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 Processor -- start 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b46408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4640e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4641370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e3b4646670 con 0x55e3b463aca0 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.073+0000 7f40227af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3b4559740 con 0x55e3b463a380 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f40227af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e3b46467f0 con 0x55e3b4643880 2024-04-20T11:25:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f40227af740 10 client.0 _ll_register_callbacks cb 0x55e3b4554b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b46408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b46408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35908/0 (socket says 172.21.15.44:35908) 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4641370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 -- 172.21.15.44:0/3506855847 learned_addr learned my addr 172.21.15.44:0/3506855847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f4017e00640 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4640e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 -- 172.21.15.44:0/3506855847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 msgr2=0x55e3b4641370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4641370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 -- 172.21.15.44:0/3506855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 msgr2=0x55e3b4640e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4640e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.074+0000 7f401d000640 1 -- 172.21.15.44:0/3506855847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b42b23f0 con 0x55e3b463a380 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f401d000640 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b46408f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4010032910 tx=0x7f40100095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e92ed5a05ee4cad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f4002a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f4002a00640 1 -- 172.21.15.44:0/3506855847 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010039040 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f4002a00640 1 -- 172.21.15.44:0/3506855847 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4010004510 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4008019b30 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f4002a00640 1 -- 172.21.15.44:0/3506855847 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f401005a050 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f40227af740 10 client.26246 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3b42720f0 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.075+0000 7f4002a00640 1 -- 172.21.15.44:0/3506855847 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4008019b30 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f4002a00640 1 -- 172.21.15.44:0/3506855847 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f4010040460 con 0x55e3b463a380 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f40227af740 10 client.26246 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f40227af740 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f40227af740 20 client.26246 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f40227af740 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b4653d40 0x55e3b4656200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f40227af740 4 client.26246 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b45adb70 con 0x55e3b4653d40 2024-04-20T11:25:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.076+0000 7f4017e00640 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b4653d40 0x55e3b4656200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.077+0000 7f4017e00640 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b4653d40 0x55e3b4656200 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788d82f542ed3f5e server_cookie=af57cae01c3405cc in_seq=0 out_seq=0 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.077+0000 7f4002a00640 10 client.26246 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f4002a00640 1 -- 172.21.15.44:0/3506855847 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3b45adb70 con 0x55e3b4653d40 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f4002a00640 10 client.26246 handle_command_reply: tid=0 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 client.26246 shutdown 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 msgr2=0x55e3b46408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b46408f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4010032910 tx=0x7f40100095c0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 shutdown_connections 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b4653d40 0x55e3b4656200 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4641370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4640e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 --2- 172.21.15.44:0/3506855847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b46408f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.079+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 >> 172.21.15.44:0/3506855847 conn(0x55e3b4634ab0 msgr2=0x55e3b4636800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.080+0000 7f4002a00640 0 client.26246 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.080+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 shutdown_connections 2024-04-20T11:25:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.080+0000 7f40227af740 1 -- 172.21.15.44:0/3506855847 wait complete. 2024-04-20T11:25:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.080+0000 7f40227af740 20 client.26246 trim_cache size 0 max 16384 2024-04-20T11:25:35.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.100+0000 7f40227af740 1 Processor -- start 2024-04-20T11:25:35.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.100+0000 7f40227af740 1 -- start start 2024-04-20T11:25:35.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f40227af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b4643020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f40227af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4655d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f40227af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b464f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f40227af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40181b4050 con 0x55e3b463aca0 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f40227af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40181b5d10 con 0x55e3b463a380 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f40227af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40181b4720 con 0x55e3b4643880 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f401d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b4643020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f401d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b4643020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35910/0 (socket says 172.21.15.44:35910) 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f401d000640 1 -- 172.21.15.44:0/2691791752 learned_addr learned my addr 172.21.15.44:0/2691791752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f4017e00640 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4655d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.101+0000 7f401da00640 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b464f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f401d000640 1 -- 172.21.15.44:0/2691791752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 msgr2=0x55e3b464f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f401d000640 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b464f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f401d000640 1 -- 172.21.15.44:0/2691791752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 msgr2=0x55e3b4655d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f401d000640 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4655d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f401d000640 1 -- 172.21.15.44:0/2691791752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b46345e0 con 0x55e3b463a380 2024-04-20T11:25:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f4017e00640 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4655d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:25:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f401d000640 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b4643020 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4010009ca0 tx=0x7f4010009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79481fff773ac301 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f400d600640 1 -- 172.21.15.44:0/2691791752 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010062020 con 0x55e3b463a380 2024-04-20T11:25:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f400d600640 1 -- 172.21.15.44:0/2691791752 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4010051070 con 0x55e3b463a380 2024-04-20T11:25:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.102+0000 7f400d600640 1 -- 172.21.15.44:0/2691791752 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010039070 con 0x55e3b463a380 2024-04-20T11:25:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 -- 172.21.15.44:0/2691791752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 msgr2=0x55e3b4643020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b4643020 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4010009ca0 tx=0x7f4010009b20 comp rx=0 tx=0).stop 2024-04-20T11:25:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 -- 172.21.15.44:0/2691791752 shutdown_connections 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b464f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463aca0 0x55e3b4655d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 --2- 172.21.15.44:0/2691791752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463a380 0x55e3b4643020 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 -- 172.21.15.44:0/2691791752 >> 172.21.15.44:0/2691791752 conn(0x55e3b4634ab0 msgr2=0x55e3b4635ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.103+0000 7f40227af740 1 -- 172.21.15.44:0/2691791752 shutdown_connections 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.104+0000 7f40227af740 1 -- 172.21.15.44:0/2691791752 wait complete. 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.104+0000 7f40227af740 1 -- start start 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.104+0000 7f40227af740 1 Processor -- start 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b464bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4647100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4647640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40181b5580 con 0x55e3b463a380 2024-04-20T11:25:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40181b4e50 con 0x55e3b463aca0 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40181b1ff0 con 0x55e3b4643880 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f40227af740 10 client.0 _ll_register_callbacks cb 0x7f4008005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4647100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4647100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:35918/0 (socket says 172.21.15.44:35918) 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 -- 172.21.15.44:0/3308590127 learned_addr learned my addr 172.21.15.44:0/3308590127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f401d000640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b464bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f401da00640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4647640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 -- 172.21.15.44:0/3308590127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 msgr2=0x55e3b4647640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4647640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 -- 172.21.15.44:0/3308590127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 msgr2=0x55e3b464bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b464bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f4017e00640 1 -- 172.21.15.44:0/3308590127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b4647e70 con 0x55e3b463aca0 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.105+0000 7f401da00640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4647640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:25:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f4017e00640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4647100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f400409c8a0 tx=0x7f40040d3000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb60ec0b42d0eb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f4002a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f4002a00640 1 -- 172.21.15.44:0/3308590127 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040a0ea0 con 0x55e3b463aca0 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4010051070 con 0x55e3b463aca0 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f4002a00640 1 -- 172.21.15.44:0/3308590127 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4004099210 con 0x55e3b463aca0 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f4002a00640 1 -- 172.21.15.44:0/3308590127 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f40040a5760 con 0x55e3b463aca0 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f40227af740 10 client.26252 fetch_fsmap learned FSMap version 35 2024-04-20T11:25:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3b46ba310 con 0x55e3b463aca0 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.106+0000 7f4002a00640 1 -- 172.21.15.44:0/3308590127 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4010051070 con 0x55e3b463aca0 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f4002a00640 1 -- 172.21.15.44:0/3308590127 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8407+0+0 (secure 0 0 0) 0x7f400409c080 con 0x55e3b463aca0 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f40227af740 10 client.26252 fetch_fsmap finished waiting for FSMap version 35 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f40227af740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f40227af740 20 client.26252 populate_metadata read hostname 'smithi044' 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f40227af740 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b46bab00 0x55e3b46bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f40227af740 4 client.26252 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3b45a79e0 con 0x55e3b46bab00 2024-04-20T11:25:35.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f401d000640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b46bab00 0x55e3b46bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:25:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f401d000640 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b46bab00 0x55e3b46bcf20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecf4dfd3cd60781 server_cookie=eb359fb3ee3d096f in_seq=0 out_seq=0 2024-04-20T11:25:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.107+0000 7f4002a00640 10 client.26252 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.110+0000 7f4002a00640 1 -- 172.21.15.44:0/3308590127 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55e3b45a79e0 con 0x55e3b46bab00 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.110+0000 7f4002a00640 10 client.26252 handle_command_reply: tid=0 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.110+0000 7f40227af740 1 client.26252 shutdown 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 msgr2=0x55e3b4647100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4647100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f400409c8a0 tx=0x7f40040d3000 comp rx=0 tx=0).stop 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 shutdown_connections 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e3b46bab00 0x55e3b46bcf20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b4643880 0x55e3b4647640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3b463aca0 0x55e3b4647100 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 --2- 172.21.15.44:0/3308590127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3b463a380 0x55e3b464bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 >> 172.21.15.44:0/3308590127 conn(0x55e3b4634ab0 msgr2=0x55e3b4633280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f4002a00640 0 client.26252 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 shutdown_connections 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 1 -- 172.21.15.44:0/3308590127 wait complete. 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.111+0000 7f40227af740 20 client.26252 trim_cache size 0 max 16384 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (537 inodes in the stack)", 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "365d2521-4343-4d4c-ac7c-9437ef721c3d": { 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "365d2521-4343-4d4c-ac7c-9437ef721c3d", 2024-04-20T11:25:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:25:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:25:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:25:35.113 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:25:35.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ffc0a3d20 msgr2=0x7f3ffc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ffc0a3d20 0x7f3ffc0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4010000f80 tx=0x7f4010009920 comp rx=0 tx=0).stop 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 msgr2=0x7f401819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f401819f310 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f400400e9f0 tx=0x7f400400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 shutdown_connections 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ffc0a3d20 0x7f3ffc0a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f401810d3d0 0x7f40181a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4018108ea0 0x7f401819f310 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 --2- 172.21.15.44:0/3239297894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4018108500 0x7f401819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 >> 172.21.15.44:0/3239297894 conn(0x7f4018077d80 msgr2=0x7f40180783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 shutdown_connections 2024-04-20T11:25:35.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:25:35.112+0000 7f401e400640 1 -- 172.21.15.44:0/3239297894 wait complete. 2024-04-20T11:25:35.131 INFO:tasks.cephfs.filesystem:scrub status for tag:365d2521-4343-4d4c-ac7c-9437ef721c3d - {'path': '/', 'tag': '365d2521-4343-4d4c-ac7c-9437ef721c3d', 'options': 'recursive,force'} 2024-04-20T11:25:35.682 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:35 smithi165 ceph-mon[21975]: pgmap v92: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-04-20T11:25:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:35 smithi078 ceph-mon[21464]: pgmap v92: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-04-20T11:25:35.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:35 smithi044 ceph-mon[16855]: pgmap v92: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-04-20T11:25:36.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:25:36.755 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.755 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.756 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:36.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:36 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:36 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.761 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.763 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:36.764 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:36 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:37 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:37.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:37 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:37 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:25:37.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:37 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:37 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:37 smithi078 ceph-mon[21464]: pgmap v93: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-04-20T11:25:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:37 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:37.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:37 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:37 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:25:37.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:37 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:37 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:37 smithi044 ceph-mon[16855]: pgmap v93: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-04-20T11:25:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:37 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:37 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:37 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:25:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:37 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:37 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:25:37.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:37 smithi165 ceph-mon[21975]: pgmap v93: 97 pgs: 97 active+clean; 224 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-04-20T11:25:39.077 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2024-04-20T11:25:39.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:39 smithi078 ceph-mon[21464]: pgmap v94: 97 pgs: 97 active+clean; 230 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2024-04-20T11:25:39.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:39 smithi044 ceph-mon[16855]: pgmap v94: 97 pgs: 97 active+clean; 230 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2024-04-20T11:25:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:39 smithi165 ceph-mon[21975]: pgmap v94: 97 pgs: 97 active+clean; 230 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 259 op/s 2024-04-20T11:25:41.031 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar c linux-4.0.5 2024-04-20T11:25:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:41 smithi078 ceph-mon[21464]: pgmap v95: 97 pgs: 97 active+clean; 230 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-04-20T11:25:41.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:41 smithi044 ceph-mon[16855]: pgmap v95: 97 pgs: 97 active+clean; 230 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-04-20T11:25:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:41 smithi165 ceph-mon[21975]: pgmap v95: 97 pgs: 97 active+clean; 230 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-04-20T11:25:42.102 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo Ok 2024-04-20T11:25:42.102 INFO:tasks.workunit.client.0.smithi044.stdout:Ok 2024-04-20T11:25:42.103 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T11:25:42.104 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T11:25:42.447 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:42 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:42.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:42 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:42.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:42 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:43.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:43 smithi078 ceph-mon[21464]: pgmap v96: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-04-20T11:25:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:43 smithi044 ceph-mon[16855]: pgmap v96: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-04-20T11:25:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:43 smithi165 ceph-mon[21975]: pgmap v96: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-04-20T11:25:45.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:45 smithi078 ceph-mon[21464]: pgmap v97: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-04-20T11:25:45.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:45 smithi044 ceph-mon[16855]: pgmap v97: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-04-20T11:25:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:45 smithi165 ceph-mon[21975]: pgmap v97: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-04-20T11:25:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:47 smithi078 ceph-mon[21464]: pgmap v98: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-04-20T11:25:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:47 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:47.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:47 smithi044 ceph-mon[16855]: pgmap v98: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-04-20T11:25:47.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:47 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:47 smithi165 ceph-mon[21975]: pgmap v98: 97 pgs: 97 active+clean; 256 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 249 op/s 2024-04-20T11:25:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:47 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:48.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:48 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:48.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:48 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:48.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:48 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:48.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:48 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:48.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:48 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:25:48.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:48 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:48.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:48 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:48.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:48 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:48.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:48 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:48.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:48 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:25:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:48 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:48 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:48 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:48 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:48.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:48 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:25:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:49 smithi078 ceph-mon[21464]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-04-20T11:25:49.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:49 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:25:49.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:49 smithi044 ceph-mon[16855]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-04-20T11:25:49.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:49 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:25:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:49 smithi165 ceph-mon[21975]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2024-04-20T11:25:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:49 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:25:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:50 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:25:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:50 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:25:50.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:50 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:25:50.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:50 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:25:50.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:50 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:25:50.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:50 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:25:50.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:50 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:25:50.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:50 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:25:50.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:50 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:25:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:51 smithi078 ceph-mon[21464]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-04-20T11:25:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:51 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:51.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:51 smithi044 ceph-mon[16855]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-04-20T11:25:51.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:51 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:51 smithi165 ceph-mon[21975]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-04-20T11:25:51.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:51 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:25:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:53 smithi078 ceph-mon[21464]: pgmap v101: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 386 op/s 2024-04-20T11:25:53.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:53 smithi044 ceph-mon[16855]: pgmap v101: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 386 op/s 2024-04-20T11:25:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:53 smithi165 ceph-mon[21975]: pgmap v101: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 386 op/s 2024-04-20T11:25:55.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:55 smithi078 ceph-mon[21464]: pgmap v102: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 778 KiB/s wr, 179 op/s 2024-04-20T11:25:55.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:55 smithi044 ceph-mon[16855]: pgmap v102: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 778 KiB/s wr, 179 op/s 2024-04-20T11:25:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:55 smithi165 ceph-mon[21975]: pgmap v102: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 778 KiB/s wr, 179 op/s 2024-04-20T11:25:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:57 smithi078 ceph-mon[21464]: pgmap v103: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 778 KiB/s wr, 179 op/s 2024-04-20T11:25:57.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:57 smithi044 ceph-mon[16855]: pgmap v103: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 778 KiB/s wr, 179 op/s 2024-04-20T11:25:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:57 smithi165 ceph-mon[21975]: pgmap v103: 97 pgs: 97 active+clean; 262 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 778 KiB/s wr, 179 op/s 2024-04-20T11:25:59.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:25:59 smithi078 ceph-mon[21464]: pgmap v104: 97 pgs: 97 active+clean; 263 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-04-20T11:25:59.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:25:59 smithi044 ceph-mon[16855]: pgmap v104: 97 pgs: 97 active+clean; 263 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-04-20T11:25:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:25:59 smithi165 ceph-mon[21975]: pgmap v104: 97 pgs: 97 active+clean; 263 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-04-20T11:26:01.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:01 smithi078 ceph-mon[21464]: pgmap v105: 97 pgs: 97 active+clean; 263 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 722 KiB/s wr, 176 op/s 2024-04-20T11:26:01.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:01 smithi044 ceph-mon[16855]: pgmap v105: 97 pgs: 97 active+clean; 263 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 722 KiB/s wr, 176 op/s 2024-04-20T11:26:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:01 smithi165 ceph-mon[21975]: pgmap v105: 97 pgs: 97 active+clean; 263 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 722 KiB/s wr, 176 op/s 2024-04-20T11:26:02.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:02 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:26:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:02 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:26:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:02 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:26:03.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:03 smithi078 ceph-mon[21464]: pgmap v106: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 279 op/s 2024-04-20T11:26:03.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:03 smithi044 ceph-mon[16855]: pgmap v106: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 279 op/s 2024-04-20T11:26:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:03 smithi165 ceph-mon[21975]: pgmap v106: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 279 op/s 2024-04-20T11:26:05.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:26:05.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 -- 172.21.15.44:0/2728344565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe798104280 msgr2=0x7fe798104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 --2- 172.21.15.44:0/2728344565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe798104280 0x7fe798104660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe790009860 tx=0x7fe79002bb70 comp rx=0 tx=0).stop 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 -- 172.21.15.44:0/2728344565 shutdown_connections 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 --2- 172.21.15.44:0/2728344565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798109350 0x7fe798110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 --2- 172.21.15.44:0/2728344565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798104c30 0x7fe798108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 --2- 172.21.15.44:0/2728344565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe798104280 0x7fe798104660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 -- 172.21.15.44:0/2728344565 >> 172.21.15.44:0/2728344565 conn(0x7fe7980ffaa0 msgr2=0x7fe798101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 -- 172.21.15.44:0/2728344565 shutdown_connections 2024-04-20T11:26:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.331+0000 7fe79da00640 1 -- 172.21.15.44:0/2728344565 wait complete. 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.332+0000 7fe79da00640 1 Processor -- start 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.332+0000 7fe79da00640 1 -- start start 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 0x7fe7981a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798109350 0x7fe7981a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe79819a820 0x7fe79819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe798114520 con 0x7fe798109350 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79da00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981143a0 con 0x7fe79819a820 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981146a0 con 0x7fe798104280 2024-04-20T11:26:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 0x7fe7981a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 0x7fe7981a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49362/0 (socket says 172.21.15.44:49362) 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 -- 172.21.15.44:0/3879763189 learned_addr learned my addr 172.21.15.44:0/3879763189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797400640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798109350 0x7fe7981a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe79d000640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe79819a820 0x7fe79819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 -- 172.21.15.44:0/3879763189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe79819a820 msgr2=0x7fe79819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe79819a820 0x7fe79819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 -- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798109350 msgr2=0x7fe7981a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798109350 0x7fe7981a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797e00640 1 -- 172.21.15.44:0/3879763189 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79819b3b0 con 0x7fe798104280 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.333+0000 7fe797400640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798109350 0x7fe7981a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.334+0000 7fe797e00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 0x7fe7981a06b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe790037040 tx=0x7fe7900045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.334+0000 7fe794c00640 1 -- 172.21.15.44:0/3879763189 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790041020 con 0x7fe798104280 2024-04-20T11:26:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.334+0000 7fe794c00640 1 -- 172.21.15.44:0/3879763189 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe79002ca10 con 0x7fe798104280 2024-04-20T11:26:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.334+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79819b640 con 0x7fe798104280 2024-04-20T11:26:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.334+0000 7fe794c00640 1 -- 172.21.15.44:0/3879763189 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790039040 con 0x7fe798104280 2024-04-20T11:26:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.334+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a7820 con 0x7fe798104280 2024-04-20T11:26:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.337+0000 7fe794c00640 1 -- 172.21.15.44:0/3879763189 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe790044020 con 0x7fe798104280 2024-04-20T11:26:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 Processor -- start 2024-04-20T11:26:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 -- start start 2024-04-20T11:26:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879cb90 0x561f0879cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879d4b0 0x561f087a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561f086fa150 con 0x561f0879cb90 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561f086a0370 con 0x561f087a6090 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.339+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f086f6110 con 0x561f0879d4b0 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879d4b0 0x561f087a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe79d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879cb90 0x561f0879cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe794c00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe77c0a3da0 0x7fe77c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879d4b0 0x561f087a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49364/0 (socket says 172.21.15.44:49364) 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797e00640 1 -- 172.21.15.44:0/4145719077 learned_addr learned my addr 172.21.15.44:0/4145719077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe79d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879cb90 0x561f0879cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60292/0 (socket says 172.21.15.44:60292) 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe794c00640 1 -- 172.21.15.44:0/3879763189 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe7900f1f50 con 0x7fe798104280 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe79d000640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe77c0a3da0 0x7fe77c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797400640 1 -- 172.21.15.44:0/4145719077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879d4b0 msgr2=0x561f087a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797400640 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879d4b0 0x561f087a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797400640 1 -- 172.21.15.44:0/4145719077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879cb90 msgr2=0x561f0879cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797400640 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879cb90 0x561f0879cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.340+0000 7fe797400640 1 -- 172.21.15.44:0/4145719077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f084d67d0 con 0x561f087a6090 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.341+0000 7fe797400640 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a8480 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe79819bb70 tx=0x7fe788009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6a3e5db4a29d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.341+0000 7fe78d600640 1 -- 172.21.15.44:0/4145719077 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788003b70 con 0x561f087a6090 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.341+0000 7fe78d600640 1 -- 172.21.15.44:0/4145719077 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe788003d10 con 0x561f087a6090 2024-04-20T11:26:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.341+0000 7fe78d600640 1 -- 172.21.15.44:0/4145719077 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788011b60 con 0x561f087a6090 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 -- 172.21.15.44:0/4145719077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 msgr2=0x561f087a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a8480 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe79819bb70 tx=0x7fe788009f90 comp rx=0 tx=0).stop 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe79d000640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe77c0a3da0 0x7fe77c0a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe780005fd0 tx=0x7fe780004730 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 -- 172.21.15.44:0/4145719077 shutdown_connections 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a8480 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879d4b0 0x561f087a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/4145719077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879cb90 0x561f0879cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 -- 172.21.15.44:0/4145719077 >> 172.21.15.44:0/4145719077 conn(0x561f087972c0 msgr2=0x561f087976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.342+0000 7fe7a1dea740 1 -- 172.21.15.44:0/4145719077 shutdown_connections 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.343+0000 7fe7a1dea740 1 -- 172.21.15.44:0/4145719077 wait complete. 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.343+0000 7fe7a1dea740 1 -- start start 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.343+0000 7fe7a1dea740 1 Processor -- start 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561f087a8e80 con 0x561f0879d4b0 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561f086b3740 con 0x561f087a6090 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f087a9000 con 0x561f0879cb90 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe7a1dea740 10 client.0 _ll_register_callbacks cb 0x561f086aeb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe797400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe797400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60304/0 (socket says 172.21.15.44:60304) 2024-04-20T11:26:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe797400640 1 -- 172.21.15.44:0/1802634069 learned_addr learned my addr 172.21.15.44:0/1802634069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe797e00640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.344+0000 7fe79d000640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797e00640 1 -- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 msgr2=0x561f087a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797e00640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797e00640 1 -- 172.21.15.44:0/1802634069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 msgr2=0x561f087a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797e00640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797e00640 1 -- 172.21.15.44:0/1802634069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f0840c3f0 con 0x561f0879cb90 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797400640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe797e00640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087a30b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe7900be1d0 tx=0x7fe7900b9ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457b9bb6d66ab41f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe786000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe786000640 1 -- 172.21.15.44:0/1802634069 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900be010 con 0x561f0879cb90 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe786000640 1 -- 172.21.15.44:0/1802634069 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe7900de040 con 0x561f0879cb90 2024-04-20T11:26:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.345+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe788003d10 con 0x561f0879cb90 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe786000640 1 -- 172.21.15.44:0/1802634069 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7900be710 con 0x561f0879cb90 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 10 client.26269 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561f083cc0f0 con 0x561f0879cb90 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe786000640 1 -- 172.21.15.44:0/1802634069 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe788003d10 con 0x561f0879cb90 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe786000640 1 -- 172.21.15.44:0/1802634069 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fe79003c030 con 0x561f0879cb90 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 10 client.26269 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 20 client.26269 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f087b6530 0x561f087b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 4 client.26269 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.346+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f08707b70 con 0x561f087b6530 2024-04-20T11:26:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.347+0000 7fe797400640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f087b6530 0x561f087b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.347+0000 7fe797400640 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f087b6530 0x561f087b89f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0deb7a9d8607be1 server_cookie=571942509a3dbcb5 in_seq=0 out_seq=0 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.347+0000 7fe786000640 10 client.26269 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe786000640 1 -- 172.21.15.44:0/1802634069 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f08707b70 con 0x561f087b6530 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe786000640 10 client.26269 handle_command_reply: tid=0 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 client.26269 shutdown 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 msgr2=0x561f087a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087a30b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe7900be1d0 tx=0x7fe7900b9ca0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 shutdown_connections 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f087b6530 0x561f087b89f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe786000640 0 client.26269 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:05.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/1802634069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087a30b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 >> 172.21.15.44:0/1802634069 conn(0x561f087972c0 msgr2=0x561f08799010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 shutdown_connections 2024-04-20T11:26:05.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.348+0000 7fe7a1dea740 1 -- 172.21.15.44:0/1802634069 wait complete. 2024-04-20T11:26:05.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.349+0000 7fe7a1dea740 20 client.26269 trim_cache size 0 max 16384 2024-04-20T11:26:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.380+0000 7fe7a1dea740 1 Processor -- start 2024-04-20T11:26:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.380+0000 7fe7a1dea740 1 -- start start 2024-04-20T11:26:05.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981ab830 con 0x561f0879d4b0 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981ad550 con 0x561f087a6090 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981abf00 con 0x561f0879cb90 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe797400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe79d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe797400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60314/0 (socket says 172.21.15.44:60314) 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe797400640 1 -- 172.21.15.44:0/466544050 learned_addr learned my addr 172.21.15.44:0/466544050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.381+0000 7fe797e00640 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe797400640 1 -- 172.21.15.44:0/466544050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 msgr2=0x561f087b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe797400640 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe797400640 1 -- 172.21.15.44:0/466544050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 msgr2=0x561f087be2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe797400640 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087be2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe797400640 1 -- 172.21.15.44:0/466544050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f087a8d60 con 0x561f0879d4b0 2024-04-20T11:26:05.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe797400640 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a5770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe788011710 tx=0x7fe788009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6406e53271766c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.382+0000 7fe78cc00640 1 -- 172.21.15.44:0/466544050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78802f070 con 0x561f0879d4b0 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.383+0000 7fe78cc00640 1 -- 172.21.15.44:0/466544050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe788029030 con 0x561f0879d4b0 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.383+0000 7fe78cc00640 1 -- 172.21.15.44:0/466544050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78802a170 con 0x561f0879d4b0 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.383+0000 7fe7a1dea740 1 -- 172.21.15.44:0/466544050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 msgr2=0x561f087a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.383+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a5770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe788011710 tx=0x7fe788009590 comp rx=0 tx=0).stop 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 -- 172.21.15.44:0/466544050 shutdown_connections 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f087a6090 0x561f087be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f0879d4b0 0x561f087a5770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/466544050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 -- 172.21.15.44:0/466544050 >> 172.21.15.44:0/466544050 conn(0x561f087972c0 msgr2=0x561f0879e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 -- 172.21.15.44:0/466544050 shutdown_connections 2024-04-20T11:26:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 -- 172.21.15.44:0/466544050 wait complete. 2024-04-20T11:26:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.384+0000 7fe7a1dea740 1 -- start start 2024-04-20T11:26:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 Processor -- start 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f0879d4b0 0x561f0879f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f087a6090 0x561f0879ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981acdc0 con 0x561f087a6090 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981ac660 con 0x561f0879d4b0 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981a9790 con 0x561f0879cb90 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe7a1dea740 10 client.0 _ll_register_callbacks cb 0x7fe7900ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.385+0000 7fe797e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe79d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f087a6090 0x561f0879ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49390/0 (socket says 172.21.15.44:49390) 2024-04-20T11:26:05.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 -- 172.21.15.44:0/3141382825 learned_addr learned my addr 172.21.15.44:0/3141382825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797400640 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f0879d4b0 0x561f0879f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 -- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f0879d4b0 msgr2=0x561f0879f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f0879d4b0 0x561f0879f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 -- 172.21.15.44:0/3141382825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f087a6090 msgr2=0x561f0879ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f087a6090 0x561f0879ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 -- 172.21.15.44:0/3141382825 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f087a0610 con 0x561f0879cb90 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe797e00640 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087ae4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe790034900 tx=0x7fe7900bda30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2da97378e93e131 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.386+0000 7fe786000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe786000640 1 -- 172.21.15.44:0/3141382825 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900a1040 con 0x561f0879cb90 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe786000640 1 -- 172.21.15.44:0/3141382825 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fe7900efe00 con 0x561f0879cb90 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe788029030 con 0x561f0879cb90 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe786000640 1 -- 172.21.15.44:0/3141382825 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7900ba2b0 con 0x561f0879cb90 2024-04-20T11:26:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 10 client.26275 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561f087a0610 con 0x561f0879cb90 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe786000640 1 -- 172.21.15.44:0/3141382825 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe788029030 con 0x561f0879cb90 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe786000640 1 -- 172.21.15.44:0/3141382825 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fe7900b8b80 con 0x561f0879cb90 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 10 client.26275 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 10 client.26275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 20 client.26275 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f0881ceb0 0x561f0881f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 4 client.26275 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:26:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.387+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f087019e0 con 0x561f0881ceb0 2024-04-20T11:26:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.388+0000 7fe797400640 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f0881ceb0 0x561f0881f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.388+0000 7fe797400640 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f0881ceb0 0x561f0881f2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5971197ebabebe0e server_cookie=a7c213624e077998 in_seq=0 out_seq=0 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.388+0000 7fe786000640 10 client.26275 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.388+0000 7fe786000640 1 -- 172.21.15.44:0/3141382825 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f087019e0 con 0x561f0881ceb0 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.388+0000 7fe786000640 10 client.26275 handle_command_reply: tid=0 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.388+0000 7fe7a1dea740 1 client.26275 shutdown 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 msgr2=0x561f087ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087ae4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe790034900 tx=0x7fe7900bda30 comp rx=0 tx=0).stop 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 shutdown_connections 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f0881ceb0 0x561f0881f2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f087a6090 0x561f0879ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f0879d4b0 0x561f0879f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe786000640 0 client.26275 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:05.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 --2- 172.21.15.44:0/3141382825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f0879cb90 0x561f087ae4d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 >> 172.21.15.44:0/3141382825 conn(0x561f087972c0 msgr2=0x561f087a3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 shutdown_connections 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 1 -- 172.21.15.44:0/3141382825 wait complete. 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.389+0000 7fe7a1dea740 20 client.26275 trim_cache size 0 max 16384 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:26:05.393 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe77c0a3da0 msgr2=0x7fe77c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe77c0a3da0 0x7fe77c0a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe780005fd0 tx=0x7fe780004730 comp rx=0 tx=0).stop 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 msgr2=0x7fe7981a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 0x7fe7981a06b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe790037040 tx=0x7fe7900045b0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 shutdown_connections 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe79819a820 0x7fe79819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe77c0a3da0 0x7fe77c0a6260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798109350 0x7fe7981a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 --2- 172.21.15.44:0/3879763189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe798104280 0x7fe7981a06b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 >> 172.21.15.44:0/3879763189 conn(0x7fe7980ffaa0 msgr2=0x7fe798101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 shutdown_connections 2024-04-20T11:26:05.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.390+0000 7fe79da00640 1 -- 172.21.15.44:0/3879763189 wait complete. 2024-04-20T11:26:05.408 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:26:05.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:26:05.444 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:05 smithi044 ceph-mon[16855]: pgmap v107: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 773 KiB/s wr, 188 op/s 2024-04-20T11:26:05.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 -- 172.21.15.44:0/2233708439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc104b80 msgr2=0x7faddc108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 --2- 172.21.15.44:0/2233708439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc104b80 0x7faddc108bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009860 tx=0x7fadc402bb70 comp rx=0 tx=0).stop 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 -- 172.21.15.44:0/2233708439 shutdown_connections 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 --2- 172.21.15.44:0/2233708439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faddc1092a0 0x7faddc110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 --2- 172.21.15.44:0/2233708439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc104b80 0x7faddc108bc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 --2- 172.21.15.44:0/2233708439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.601+0000 7fade1600640 1 -- 172.21.15.44:0/2233708439 >> 172.21.15.44:0/2233708439 conn(0x7faddc0ff9f0 msgr2=0x7faddc101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.602+0000 7fade1600640 1 -- 172.21.15.44:0/2233708439 shutdown_connections 2024-04-20T11:26:05.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.602+0000 7fade1600640 1 -- 172.21.15.44:0/2233708439 wait complete. 2024-04-20T11:26:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 Processor -- start 2024-04-20T11:26:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 -- start start 2024-04-20T11:26:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc1092a0 0x7faddc1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faddc19a7d0 0x7faddc19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faddc114470 con 0x7faddc1041d0 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1142f0 con 0x7faddc1092a0 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.603+0000 7fade1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1145f0 con 0x7faddc19a7d0 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7faddbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7faddbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60332/0 (socket says 172.21.15.44:60332) 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7faddbe00640 1 -- 172.21.15.44:0/1656864350 learned_addr learned my addr 172.21.15.44:0/1656864350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fadd3e00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc1092a0 0x7faddc1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fade0c00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faddc19a7d0 0x7faddc19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fadd3e00640 1 -- 172.21.15.44:0/1656864350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faddc19a7d0 msgr2=0x7faddc19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fadd3e00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faddc19a7d0 0x7faddc19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fadd3e00640 1 -- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 msgr2=0x7faddc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fadd3e00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7fadd3e00640 1 -- 172.21.15.44:0/1656864350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc19b360 con 0x7faddc1092a0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.604+0000 7faddbe00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.605+0000 7fadd3e00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc1092a0 0x7faddc1a0ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fadc402c0b0 tx=0x7fadc40045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.605+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4041020 con 0x7faddc1092a0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.605+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadc402ca10 con 0x7faddc1092a0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.605+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4039040 con 0x7faddc1092a0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.605+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc19b5f0 con 0x7faddc1092a0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.605+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a7750 con 0x7faddc1092a0 2024-04-20T11:26:05.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.607+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc1056a0 con 0x7faddc1092a0 2024-04-20T11:26:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.609+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fadc4044020 con 0x7faddc1092a0 2024-04-20T11:26:05.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.610+0000 7fadd9600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadc00a3da0 0x7fadc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:05.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.610+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fadc40f1c70 con 0x7faddc1092a0 2024-04-20T11:26:05.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.611+0000 7faddbe00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadc00a3da0 0x7fadc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:05.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.611+0000 7faddbe00640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadc00a3da0 0x7fadc00a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0058c0 tx=0x7fadcc009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:05.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.614+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fadc404b020 con 0x7faddc1092a0 2024-04-20T11:26:05.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:05 smithi078 ceph-mon[21464]: pgmap v107: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 773 KiB/s wr, 188 op/s 2024-04-20T11:26:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:05 smithi165 ceph-mon[21975]: pgmap v107: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 773 KiB/s wr, 188 op/s 2024-04-20T11:26:05.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.888+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faddc104b80 con 0x7faddc1092a0 2024-04-20T11:26:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.889+0000 7fadd9600640 1 -- 172.21.15.44:0/1656864350 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11970 (secure 0 0 0) 0x7fadc40be430 con 0x7faddc1092a0 2024-04-20T11:26:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:26:05.890 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:26:00.778182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:26:05.890 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 39 2024-04-20T11:26:05.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadc00a3da0 msgr2=0x7fadc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadc00a3da0 0x7fadc00a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0058c0 tx=0x7fadcc009260 comp rx=0 tx=0).stop 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc1092a0 msgr2=0x7faddc1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc1092a0 0x7faddc1a0ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fadc402c0b0 tx=0x7fadc40045b0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 shutdown_connections 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faddc19a7d0 0x7faddc19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faddc1092a0 0x7faddc1a0ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadc00a3da0 0x7fadc00a6260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 --2- 172.21.15.44:0/1656864350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faddc1041d0 0x7faddc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 >> 172.21.15.44:0/1656864350 conn(0x7faddc0ff9f0 msgr2=0x7faddc101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 shutdown_connections 2024-04-20T11:26:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:05.892+0000 7fade1600640 1 -- 172.21.15.44:0/1656864350 wait complete. 2024-04-20T11:26:05.913 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 -- 172.21.15.44:0/1021439003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8107140 msgr2=0x7f8cf810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 --2- 172.21.15.44:0/1021439003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8107140 0x7f8cf810ed10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009ca0 tx=0x7f8cf002c100 comp rx=0 tx=0).stop 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 -- 172.21.15.44:0/1021439003 shutdown_connections 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 --2- 172.21.15.44:0/1021439003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8107140 0x7f8cf810ed10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 --2- 172.21.15.44:0/1021439003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf8102a20 0x7f8cf8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 --2- 172.21.15.44:0/1021439003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8102070 0x7f8cf8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 -- 172.21.15.44:0/1021439003 >> 172.21.15.44:0/1021439003 conn(0x7f8cf80fd930 msgr2=0x7f8cf80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 -- 172.21.15.44:0/1021439003 shutdown_connections 2024-04-20T11:26:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.200+0000 7f8cfde00640 1 -- 172.21.15.44:0/1021439003 wait complete. 2024-04-20T11:26:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.201+0000 7f8cfde00640 1 Processor -- start 2024-04-20T11:26:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 -- start start 2024-04-20T11:26:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8102070 0x7f8cf81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 0x7f8cf81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf819a820 0x7f8cf819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8112310 con 0x7f8cf8102070 2024-04-20T11:26:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8112190 con 0x7f8cf8107140 2024-04-20T11:26:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.202+0000 7f8cfde00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8112490 con 0x7f8cf819a820 2024-04-20T11:26:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 0x7f8cf81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf819a820 0x7f8cf819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 0x7f8cf81a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51054/0 (socket says 172.21.15.44:51054) 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cf7e00640 1 -- 172.21.15.44:0/2595842141 learned_addr learned my addr 172.21.15.44:0/2595842141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfca00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8102070 0x7f8cf81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 -- 172.21.15.44:0/2595842141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 msgr2=0x7f8cf81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 0x7f8cf81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 -- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8102070 msgr2=0x7f8cf81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8102070 0x7f8cf81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 -- 172.21.15.44:0/2595842141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf819b3b0 con 0x7f8cf819a820 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfca00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8102070 0x7f8cf81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 0x7f8cf81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:06.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.203+0000 7f8cfd400640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf819a820 0x7f8cf819acd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8cf002c710 tx=0x7f8cf0034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.204+0000 7f8cf5600640 1 -- 172.21.15.44:0/2595842141 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0039040 con 0x7f8cf819a820 2024-04-20T11:26:06.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.204+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf819b640 con 0x7f8cf819a820 2024-04-20T11:26:06.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.204+0000 7f8cf5600640 1 -- 172.21.15.44:0/2595842141 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8cf0044070 con 0x7f8cf819a820 2024-04-20T11:26:06.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.204+0000 7f8cf5600640 1 -- 172.21.15.44:0/2595842141 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00043a0 con 0x7f8cf819a820 2024-04-20T11:26:06.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.204+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf81a7790 con 0x7f8cf819a820 2024-04-20T11:26:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8cf5600640 1 -- 172.21.15.44:0/2595842141 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8cf0003b00 con 0x7f8cf819a820 2024-04-20T11:26:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 Processor -- start 2024-04-20T11:26:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 -- start start 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c6b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 0x564177c74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564177bd0050 con 0x564177c74670 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564177b76490 con 0x564177c6b170 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.208+0000 7f8d02185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564177bcc010 con 0x564177c6ba90 2024-04-20T11:26:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 0x564177c74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c6b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 0x564177c74130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49432/0 (socket says 172.21.15.44:49432) 2024-04-20T11:26:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 -- 172.21.15.44:0/39158004 learned_addr learned my addr 172.21.15.44:0/39158004 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 -- 172.21.15.44:0/39158004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 msgr2=0x564177c6b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c6b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 -- 172.21.15.44:0/39158004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 msgr2=0x564177c76a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c76a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.209+0000 7f8cfca00640 1 -- 172.21.15.44:0/39158004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564177b7e070 con 0x564177c6ba90 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8cfca00640 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 0x564177c74130 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009c30 tx=0x7f8cec010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b101f9612d4475d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8cf5600640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ce00a3d20 0x7f8ce00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8cf5600640 1 -- 172.21.15.44:0/2595842141 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8cf00f2c40 con 0x7f8cf819a820 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8ce5c00640 1 -- 172.21.15.44:0/39158004 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec004030 con 0x564177c6ba90 2024-04-20T11:26:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8ce5c00640 1 -- 172.21.15.44:0/39158004 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8cec0041d0 con 0x564177c6ba90 2024-04-20T11:26:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8ce5c00640 1 -- 172.21.15.44:0/39158004 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0127d0 con 0x564177c6ba90 2024-04-20T11:26:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.210+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ce00a3d20 0x7f8ce00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 -- 172.21.15.44:0/39158004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 msgr2=0x564177c74130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ce00a3d20 0x7f8ce00a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x564177c66a30 tx=0x7f8ce8004730 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 0x564177c74130 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009c30 tx=0x7f8cec010740 comp rx=0 tx=0).stop 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 -- 172.21.15.44:0/39158004 shutdown_connections 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c76a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6ba90 0x564177c74130 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 --2- 172.21.15.44:0/39158004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c6b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 -- 172.21.15.44:0/39158004 >> 172.21.15.44:0/39158004 conn(0x564177c658a0 msgr2=0x564177c65c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 -- 172.21.15.44:0/39158004 shutdown_connections 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.211+0000 7f8d02185740 1 -- 172.21.15.44:0/39158004 wait complete. 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 -- start start 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 Processor -- start 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c71730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c6ba90 0x564177c71c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c74670 0x564177c721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564177c77460 con 0x564177c6ba90 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564177b89860 con 0x564177c6b170 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564177c775e0 con 0x564177c74670 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.212+0000 7f8d02185740 10 client.0 _ll_register_callbacks cb 0x564177b57fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c71730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c71730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51074/0 (socket says 172.21.15.44:51074) 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 -- 172.21.15.44:0/3592484975 learned_addr learned my addr 172.21.15.44:0/3592484975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c6ba90 0x564177c71c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfd400640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c74670 0x564177c721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 -- 172.21.15.44:0/3592484975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c74670 msgr2=0x564177c721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c74670 0x564177c721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 -- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c6ba90 msgr2=0x564177c71c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c6ba90 0x564177c71c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfca00640 1 -- 172.21.15.44:0/3592484975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564177b8d950 con 0x564177c6b170 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.213+0000 7f8cfd400640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c74670 0x564177c721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8cfca00640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c71730 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0124b0 tx=0x7f8cec0099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9100860c2ec143bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8cda000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8cda000640 1 -- 172.21.15.44:0/3592484975 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec029060 con 0x564177c6b170 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8cda000640 1 -- 172.21.15.44:0/3592484975 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8cec0037f0 con 0x564177c6b170 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cec0041d0 con 0x564177c6b170 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8cda000640 1 -- 172.21.15.44:0/3592484975 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8cec004000 con 0x564177c6b170 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8d02185740 10 client.26288 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:06.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564177b5b820 con 0x564177c6b170 2024-04-20T11:26:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.214+0000 7f8cda000640 1 -- 172.21.15.44:0/3592484975 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8cec0041d0 con 0x564177c6b170 2024-04-20T11:26:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8cda000640 1 -- 172.21.15.44:0/3592484975 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f8cec02c020 con 0x564177c6b170 2024-04-20T11:26:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8d02185740 10 client.26288 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8d02185740 10 client.26288 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:26:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8d02185740 20 client.26288 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8d02185740 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177c84690 0x564177c86a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8d02185740 4 client.26288 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564177bdda70 con 0x564177c84690 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.215+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177c84690 0x564177c86a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.216+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177c84690 0x564177c86a80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36ef03cb3008f6f9 server_cookie=6fe448d92945d4be in_seq=0 out_seq=0 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.216+0000 7f8cda000640 10 client.26288 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.217+0000 7f8cda000640 1 -- 172.21.15.44:0/3592484975 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564177bdda70 con 0x564177c84690 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.217+0000 7f8cda000640 10 client.26288 handle_command_reply: tid=0 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.217+0000 7f8d02185740 1 client.26288 shutdown 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.217+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 msgr2=0x564177c71730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.217+0000 7f8d02185740 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c71730 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0124b0 tx=0x7f8cec0099a0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 shutdown_connections 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c74670 0x564177c721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c6ba90 0x564177c71c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177c84690 0x564177c86a80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 --2- 172.21.15.44:0/3592484975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6b170 0x564177c71730 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8cda000640 0 client.26288 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 >> 172.21.15.44:0/3592484975 conn(0x564177c658a0 msgr2=0x564177c675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 shutdown_connections 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 1 -- 172.21.15.44:0/3592484975 wait complete. 2024-04-20T11:26:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.218+0000 7f8d02185740 20 client.26288 trim_cache size 0 max 16384 2024-04-20T11:26:06.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.234+0000 7f8d02185740 1 Processor -- start 2024-04-20T11:26:06.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.234+0000 7f8d02185740 1 -- start start 2024-04-20T11:26:06.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8d02185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c73e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8d02185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c80320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8d02185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c8c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8d02185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81afd00 con 0x564177c74670 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8d02185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b1a20 con 0x564177c6ba90 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8d02185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b03d0 con 0x564177c6b170 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c8c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c8c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60370/0 (socket says 172.21.15.44:60370) 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8cfd400640 1 -- 172.21.15.44:0/3889293794 learned_addr learned my addr 172.21.15.44:0/3889293794 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c80320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.235+0000 7f8cfca00640 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c73e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cfd400640 1 -- 172.21.15.44:0/3889293794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 msgr2=0x564177c73e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cfd400640 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c73e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cfd400640 1 -- 172.21.15.44:0/3889293794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 msgr2=0x564177c80320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cfd400640 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c80320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cfd400640 1 -- 172.21.15.44:0/3889293794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564177c86df0 con 0x564177c74670 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c80320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8cfd400640 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c8c2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00bea00 tx=0x7f8cf00be660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b8d15ce492eb18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8ce5200640 1 -- 172.21.15.44:0/3889293794 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00d8070 con 0x564177c74670 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8ce5200640 1 -- 172.21.15.44:0/3889293794 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8cf00bf400 con 0x564177c74670 2024-04-20T11:26:06.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.236+0000 7f8ce5200640 1 -- 172.21.15.44:0/3889293794 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00bef60 con 0x564177c74670 2024-04-20T11:26:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 -- 172.21.15.44:0/3889293794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 msgr2=0x564177c8c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c8c2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00bea00 tx=0x7f8cf00be660 comp rx=0 tx=0).stop 2024-04-20T11:26:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 -- 172.21.15.44:0/3889293794 shutdown_connections 2024-04-20T11:26:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c8c2a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c80320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 --2- 172.21.15.44:0/3889293794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c73e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 -- 172.21.15.44:0/3889293794 >> 172.21.15.44:0/3889293794 conn(0x564177c658a0 msgr2=0x564177c65c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 -- 172.21.15.44:0/3889293794 shutdown_connections 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.237+0000 7f8d02185740 1 -- 172.21.15.44:0/3889293794 wait complete. 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 -- start start 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 Processor -- start 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b1290 con 0x564177c74670 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b0b30 con 0x564177c6ba90 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81adc40 con 0x564177c6b170 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.238+0000 7f8d02185740 10 client.0 _ll_register_callbacks cb 0x7f8cec009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49472/0 (socket says 172.21.15.44:49472) 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 -- 172.21.15.44:0/1191424678 learned_addr learned my addr 172.21.15.44:0/1191424678 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfd400640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 -- 172.21.15.44:0/1191424678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 msgr2=0x564177c787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 -- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 msgr2=0x564177c78d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c78d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 -- 172.21.15.44:0/1191424678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641778e23f0 con 0x564177c6b170 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cfca00640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c782a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8cec004fb0 tx=0x7f8cec003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2798f35ab7210f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.239+0000 7f8cda000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.240+0000 7f8cda000640 1 -- 172.21.15.44:0/1191424678 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec029070 con 0x564177c6b170 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.240+0000 7f8cda000640 1 -- 172.21.15.44:0/1191424678 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8cec004590 con 0x564177c6b170 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.240+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf00bf400 con 0x564177c6b170 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.240+0000 7f8cda000640 1 -- 172.21.15.44:0/1191424678 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8cec009530 con 0x564177c6b170 2024-04-20T11:26:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.240+0000 7f8d02185740 10 client.26296 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.240+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5641779ac7d0 con 0x564177c6b170 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8cda000640 1 -- 172.21.15.44:0/1191424678 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8cf00bf400 con 0x564177c6b170 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8cda000640 1 -- 172.21.15.44:0/1191424678 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f8cec03f050 con 0x564177c6b170 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8d02185740 10 client.26296 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8d02185740 10 client.26296 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8d02185740 20 client.26296 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8d02185740 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177ceb290 0x564177ced6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8d02185740 4 client.26296 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564177bd78e0 con 0x564177ceb290 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.241+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177ceb290 0x564177ced6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8cf7e00640 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177ceb290 0x564177ced6b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5aefe9590dfe5dc server_cookie=3da224784988f6da in_seq=0 out_seq=0 2024-04-20T11:26:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8cda000640 10 client.26296 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:06.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8cda000640 1 -- 172.21.15.44:0/1191424678 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564177bd78e0 con 0x564177ceb290 2024-04-20T11:26:06.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8cda000640 10 client.26296 handle_command_reply: tid=0 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 client.26296 shutdown 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 msgr2=0x564177c782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c782a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8cec004fb0 tx=0x7f8cec003ac0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 shutdown_connections 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564177c74670 0x564177c78d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x564177ceb290 0x564177ced6b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564177c6ba90 0x564177c787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 --2- 172.21.15.44:0/1191424678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564177c6b170 0x564177c782a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 >> 172.21.15.44:0/1191424678 conn(0x564177c658a0 msgr2=0x564177c640d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.242+0000 7f8cda000640 0 client.26296 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 shutdown_connections 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8d02185740 1 -- 172.21.15.44:0/1191424678 wait complete. 2024-04-20T11:26:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8d02185740 20 client.26296 trim_cache size 0 max 16384 2024-04-20T11:26:06.248 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:06.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ce00a3d20 msgr2=0x7f8ce00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ce00a3d20 0x7f8ce00a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x564177c66a30 tx=0x7f8ce8004730 comp rx=0 tx=0).stop 2024-04-20T11:26:06.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf819a820 msgr2=0x7f8cf819acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf819a820 0x7f8cf819acd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8cf002c710 tx=0x7f8cf0034440 comp rx=0 tx=0).stop 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 shutdown_connections 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf819a820 0x7f8cf819acd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf8107140 0x7f8cf81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ce00a3d20 0x7f8ce00a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 --2- 172.21.15.44:0/2595842141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cf8102070 0x7f8cf81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.243+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 >> 172.21.15.44:0/2595842141 conn(0x7f8cf80fd930 msgr2=0x7f8cf810d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.244+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 shutdown_connections 2024-04-20T11:26:06.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.244+0000 7f8cfde00640 1 -- 172.21.15.44:0/2595842141 wait complete. 2024-04-20T11:26:06.264 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:26:06.348 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-04-20T11:26:06.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 -- 172.21.15.44:0/1971620553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074310 msgr2=0x7fcd600746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 --2- 172.21.15.44:0/1971620553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074310 0x7fcd600746f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009900 tx=0x7fcd5002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 -- 172.21.15.44:0/1971620553 shutdown_connections 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 --2- 172.21.15.44:0/1971620553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd6010e840 0x7fcd60110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 --2- 172.21.15.44:0/1971620553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd60074c30 0x7fcd6010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 --2- 172.21.15.44:0/1971620553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074310 0x7fcd600746f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 -- 172.21.15.44:0/1971620553 >> 172.21.15.44:0/1971620553 conn(0x7fcd600fda70 msgr2=0x7fcd600ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 -- 172.21.15.44:0/1971620553 shutdown_connections 2024-04-20T11:26:06.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.462+0000 7fcd65600640 1 -- 172.21.15.44:0/1971620553 wait complete. 2024-04-20T11:26:06.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 Processor -- start 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 -- start start 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd60074310 0x7fcd601a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 0x7fcd601a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010e840 0x7fcd6019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601146b0 con 0x7fcd60074c30 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114530 con 0x7fcd6010e840 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.463+0000 7fcd65600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114830 con 0x7fcd60074310 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 0x7fcd601a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd60074310 0x7fcd601a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 0x7fcd601a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60394/0 (socket says 172.21.15.44:60394) 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5e000640 1 -- 172.21.15.44:0/3258790773 learned_addr learned my addr 172.21.15.44:0/3258790773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5f400640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010e840 0x7fcd6019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5ea00640 1 -- 172.21.15.44:0/3258790773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010e840 msgr2=0x7fcd6019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5ea00640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010e840 0x7fcd6019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5ea00640 1 -- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 msgr2=0x7fcd601a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5ea00640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 0x7fcd601a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5ea00640 1 -- 172.21.15.44:0/3258790773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019b170 con 0x7fcd60074310 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.464+0000 7fcd5e000640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 0x7fcd601a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.465+0000 7fcd5ea00640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd60074310 0x7fcd601a07f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcd5002c210 tx=0x7fcd50004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.465+0000 7fcd57400640 1 -- 172.21.15.44:0/3258790773 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd50043020 con 0x7fcd60074310 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.465+0000 7fcd57400640 1 -- 172.21.15.44:0/3258790773 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd50004250 con 0x7fcd60074310 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.465+0000 7fcd57400640 1 -- 172.21.15.44:0/3258790773 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd500039d0 con 0x7fcd60074310 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.465+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019b400 con 0x7fcd60074310 2024-04-20T11:26:06.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.465+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a76b0 con 0x7fcd60074310 2024-04-20T11:26:06.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.468+0000 7fcd68371740 1 Processor -- start 2024-04-20T11:26:06.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.468+0000 7fcd57400640 1 -- 172.21.15.44:0/3258790773 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcd50046020 con 0x7fcd60074310 2024-04-20T11:26:06.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.468+0000 7fcd68371740 1 -- start start 2024-04-20T11:26:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd68371740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 0x562692a2f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd68371740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2f8d0 0x562692a37f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd68371740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a384b0 0x562692a3a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd68371740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562692994050 con 0x562692a384b0 2024-04-20T11:26:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd68371740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56269293a490 con 0x562692a2efb0 2024-04-20T11:26:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd68371740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562692990010 con 0x562692a2f8d0 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 0x562692a2f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 0x562692a2f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51128/0 (socket says 172.21.15.44:51128) 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd5f400640 1 -- 172.21.15.44:0/3715123609 learned_addr learned my addr 172.21.15.44:0/3715123609 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd5ea00640 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2f8d0 0x562692a37f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.469+0000 7fcd5e000640 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a384b0 0x562692a3a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5f400640 1 -- 172.21.15.44:0/3715123609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2f8d0 msgr2=0x562692a37f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5f400640 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2f8d0 0x562692a37f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5f400640 1 -- 172.21.15.44:0/3715123609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a384b0 msgr2=0x562692a3a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5f400640 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a384b0 0x562692a3a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5f400640 1 -- 172.21.15.44:0/3715123609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562692942070 con 0x562692a2efb0 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5e000640 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a384b0 0x562692a3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd5f400640 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 0x562692a2f390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009f50 tx=0x7fcd580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d55d967493d7af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd57400640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd440a3d20 0x7fcd440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd4b400640 1 -- 172.21.15.44:0/3715123609 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004d20 con 0x562692a2efb0 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd4b400640 1 -- 172.21.15.44:0/3715123609 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd58004050 con 0x562692a2efb0 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.470+0000 7fcd4b400640 1 -- 172.21.15.44:0/3715123609 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58005260 con 0x562692a2efb0 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd57400640 1 -- 172.21.15.44:0/3258790773 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcd500f2560 con 0x7fcd60074310 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd5ea00640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd440a3d20 0x7fcd440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 -- 172.21.15.44:0/3715123609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 msgr2=0x562692a2f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 0x562692a2f390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009f50 tx=0x7fcd580118d0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 -- 172.21.15.44:0/3715123609 shutdown_connections 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a384b0 0x562692a3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2f8d0 0x562692a37f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 --2- 172.21.15.44:0/3715123609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a2efb0 0x562692a2f390 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 -- 172.21.15.44:0/3715123609 >> 172.21.15.44:0/3715123609 conn(0x562692a296e0 msgr2=0x562692a29ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd5ea00640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd440a3d20 0x7fcd440a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x562692a2ea80 tx=0x7fcd500f5f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 -- 172.21.15.44:0/3715123609 shutdown_connections 2024-04-20T11:26:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.471+0000 7fcd68371740 1 -- 172.21.15.44:0/3715123609 wait complete. 2024-04-20T11:26:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.472+0000 7fcd68371740 1 -- start start 2024-04-20T11:26:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.472+0000 7fcd68371740 1 Processor -- start 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a35570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a35ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562692a3b2a0 con 0x562692a2f8d0 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56269294d860 con 0x562692a384b0 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562692a3b420 con 0x562692a2efb0 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd68371740 10 client.0 _ll_register_callbacks cb 0x56269291bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a35570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a35570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49510/0 (socket says 172.21.15.44:49510) 2024-04-20T11:26:06.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5f400640 1 -- 172.21.15.44:0/1037523426 learned_addr learned my addr 172.21.15.44:0/1037523426 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a35ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60410/0 (socket says 172.21.15.44:60410) 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5f400640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a35ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 -- 172.21.15.44:0/1037523426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 msgr2=0x562692a35570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a35570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 -- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 msgr2=0x562692a35ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a35ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5e000640 1 -- 172.21.15.44:0/1037523426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562692951950 con 0x562692a2f8d0 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.473+0000 7fcd5f400640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a35ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd5e000640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a35ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcd60100090 tx=0x7fcd4c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7886a667bf8514 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd3fe00640 1 -- 172.21.15.44:0/1037523426 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0120d0 con 0x562692a2f8d0 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd3fe00640 1 -- 172.21.15.44:0/1037523426 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcd4c005020 con 0x562692a2f8d0 2024-04-20T11:26:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd58004050 con 0x562692a2f8d0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd3fe00640 1 -- 172.21.15.44:0/1037523426 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c012270 con 0x562692a2f8d0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd68371740 10 client.16179 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.474+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56269291f820 con 0x562692a2f8d0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd3fe00640 1 -- 172.21.15.44:0/1037523426 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd58004050 con 0x562692a2f8d0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd3fe00640 1 -- 172.21.15.44:0/1037523426 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fcd4c024050 con 0x562692a2f8d0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd68371740 10 client.16179 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd68371740 10 client.16179 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd68371740 20 client.16179 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd68371740 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692a48520 0x562692a4a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd68371740 4 client.16179 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.475+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626929a1a70 con 0x562692a48520 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.476+0000 7fcd5f400640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692a48520 0x562692a4a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.476+0000 7fcd5f400640 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692a48520 0x562692a4a940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f79e0d3cf479dac3 server_cookie=d7f023ef4dd15fb8 in_seq=0 out_seq=0 2024-04-20T11:26:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.476+0000 7fcd3fe00640 10 client.16179 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:06.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd3fe00640 1 -- 172.21.15.44:0/1037523426 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626929a1a70 con 0x562692a48520 2024-04-20T11:26:06.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd3fe00640 10 client.16179 handle_command_reply: tid=0 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 client.16179 shutdown 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 msgr2=0x562692a35ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a35ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcd60100090 tx=0x7fcd4c00f790 comp rx=0 tx=0).stop 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 shutdown_connections 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a35ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692a48520 0x562692a4a940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a35ab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 --2- 172.21.15.44:0/1037523426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a35570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 >> 172.21.15.44:0/1037523426 conn(0x562692a296e0 msgr2=0x562692a2b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd3fe00640 0 client.16179 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 shutdown_connections 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.477+0000 7fcd68371740 1 -- 172.21.15.44:0/1037523426 wait complete. 2024-04-20T11:26:06.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.478+0000 7fcd68371740 20 client.16179 trim_cache size 0 max 16384 2024-04-20T11:26:06.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 Processor -- start 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 -- start start 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a44160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601afee0 con 0x562692a2f8d0 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b1c00 con 0x562692a384b0 2024-04-20T11:26:06.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd68371740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b05b0 con 0x562692a2efb0 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a44160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a44160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60420/0 (socket says 172.21.15.44:60420) 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd5e000640 1 -- 172.21.15.44:0/2215282917 learned_addr learned my addr 172.21.15.44:0/2215282917 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.494+0000 7fcd5f400640 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a346d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51152/0 (socket says 172.21.15.44:51152) 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5f400640 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5e000640 1 -- 172.21.15.44:0/2215282917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 msgr2=0x562692a37ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5e000640 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a37ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5e000640 1 -- 172.21.15.44:0/2215282917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 msgr2=0x562692a346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5e000640 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5e000640 1 -- 172.21.15.44:0/2215282917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562692a2f700 con 0x562692a2f8d0 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5f400640 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd5e000640 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a44160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000c00 tx=0x7fcd4c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0701cac5f27797 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.495+0000 7fcd4aa00640 1 -- 172.21.15.44:0/2215282917 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c01c7a0 con 0x562692a2f8d0 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd4aa00640 1 -- 172.21.15.44:0/2215282917 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd4c005540 con 0x562692a2f8d0 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd4aa00640 1 -- 172.21.15.44:0/2215282917 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0106f0 con 0x562692a2f8d0 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 -- 172.21.15.44:0/2215282917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 msgr2=0x562692a44160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a44160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000c00 tx=0x7fcd4c005c50 comp rx=0 tx=0).stop 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 -- 172.21.15.44:0/2215282917 shutdown_connections 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a44160 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 --2- 172.21.15.44:0/2215282917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 -- 172.21.15.44:0/2215282917 >> 172.21.15.44:0/2215282917 conn(0x562692a296e0 msgr2=0x562692a35530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.496+0000 7fcd68371740 1 -- 172.21.15.44:0/2215282917 shutdown_connections 2024-04-20T11:26:06.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.497+0000 7fcd68371740 1 -- 172.21.15.44:0/2215282917 wait complete. 2024-04-20T11:26:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.497+0000 7fcd68371740 1 -- start start 2024-04-20T11:26:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.497+0000 7fcd68371740 1 Processor -- start 2024-04-20T11:26:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a40b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a3bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b1470 con 0x562692a2f8d0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b0d10 con 0x562692a384b0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601ade20 con 0x562692a2efb0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd68371740 10 client.0 _ll_register_callbacks cb 0x7fcd4c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a3bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a3bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60422/0 (socket says 172.21.15.44:60422) 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd5e000640 1 -- 172.21.15.44:0/4217015357 learned_addr learned my addr 172.21.15.44:0/4217015357 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd5f400640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd5f400640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a40b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.498+0000 7fcd5e000640 1 -- 172.21.15.44:0/4217015357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 msgr2=0x562692a40b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd5e000640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a40b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd5e000640 1 -- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 msgr2=0x562692a3c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd5e000640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a3c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd5e000640 1 -- 172.21.15.44:0/4217015357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626926a63f0 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd5f400640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd5e000640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a3bef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c01dfd0 tx=0x7fcd4c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3a19b357f5ac6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd3fe00640 1 -- 172.21.15.44:0/4217015357 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c01cde0 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd3fe00640 1 -- 172.21.15.44:0/4217015357 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcd4c02b040 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.499+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4c005540 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd3fe00640 1 -- 172.21.15.44:0/4217015357 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c010670 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 10 client.16185 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5626927707d0 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd3fe00640 1 -- 172.21.15.44:0/4217015357 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd4c005540 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd3fe00640 1 -- 172.21.15.44:0/4217015357 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fcd4c024040 con 0x562692a2f8d0 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 10 client.16185 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 10 client.16185 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:26:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 20 client.16185 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692aaef50 0x562692ab1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 4 client.16185 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.500+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56269299b8e0 con 0x562692aaef50 2024-04-20T11:26:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.501+0000 7fcd5f400640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692aaef50 0x562692ab1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.501+0000 7fcd5f400640 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692aaef50 0x562692ab1370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9469d1ba8ceb7330 server_cookie=a9a838201f5b5477 in_seq=0 out_seq=0 2024-04-20T11:26:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.501+0000 7fcd3fe00640 10 client.16185 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd3fe00640 1 -- 172.21.15.44:0/4217015357 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56269299b8e0 con 0x562692aaef50 2024-04-20T11:26:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd3fe00640 10 client.16185 handle_command_reply: tid=0 2024-04-20T11:26:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 client.16185 shutdown 2024-04-20T11:26:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 msgr2=0x562692a3bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a3bef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c01dfd0 tx=0x7fcd4c005d40 comp rx=0 tx=0).stop 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 shutdown_connections 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562692a384b0 0x562692a3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562692aaef50 0x562692ab1370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562692a2f8d0 0x562692a3bef0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 --2- 172.21.15.44:0/4217015357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562692a2efb0 0x562692a40b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd3fe00640 0 client.16185 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 >> 172.21.15.44:0/4217015357 conn(0x562692a296e0 msgr2=0x562692a28740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 shutdown_connections 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 1 -- 172.21.15.44:0/4217015357 wait complete. 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.502+0000 7fcd68371740 20 client.16185 trim_cache size 0 max 16384 2024-04-20T11:26:06.506 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd440a3d20 msgr2=0x7fcd440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd440a3d20 0x7fcd440a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x562692a2ea80 tx=0x7fcd500f5f90 comp rx=0 tx=0).stop 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd60074310 msgr2=0x7fcd601a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd60074310 0x7fcd601a07f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcd5002c210 tx=0x7fcd50004970 comp rx=0 tx=0).stop 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 shutdown_connections 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6010e840 0x7fcd6019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd440a3d20 0x7fcd440a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd60074c30 0x7fcd601a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 --2- 172.21.15.44:0/3258790773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd60074310 0x7fcd601a07f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 >> 172.21.15.44:0/3258790773 conn(0x7fcd600fda70 msgr2=0x7fcd600ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 shutdown_connections 2024-04-20T11:26:06.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.503+0000 7fcd65600640 1 -- 172.21.15.44:0/3258790773 wait complete. 2024-04-20T11:26:06.518 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:06 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1656864350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:06.521 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:26:06.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:06 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1656864350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:06.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:06 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1656864350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:06.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 -- 172.21.15.44:0/3923840087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2534117090 msgr2=0x7f2534119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 --2- 172.21.15.44:0/3923840087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2534117090 0x7f2534119480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f252c009ca0 tx=0x7f252c02c100 comp rx=0 tx=0).stop 2024-04-20T11:26:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 -- 172.21.15.44:0/3923840087 shutdown_connections 2024-04-20T11:26:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 --2- 172.21.15.44:0/3923840087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2534117090 0x7f2534119480 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 --2- 172.21.15.44:0/3923840087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534109530 0x7f25341099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 --2- 172.21.15.44:0/3923840087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f253410c720 0x7f2534108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.777+0000 7f253b400640 1 -- 172.21.15.44:0/3923840087 >> 172.21.15.44:0/3923840087 conn(0x7f253406d070 msgr2=0x7f253406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.778+0000 7f253b400640 1 -- 172.21.15.44:0/3923840087 shutdown_connections 2024-04-20T11:26:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.778+0000 7f253b400640 1 -- 172.21.15.44:0/3923840087 wait complete. 2024-04-20T11:26:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.779+0000 7f253b400640 1 Processor -- start 2024-04-20T11:26:06.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.779+0000 7f253b400640 1 -- start start 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f253b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2534109530 0x7f253410cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f253b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534117090 0x7f253410d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f253b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 0x7f25341b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f253b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f253411ce30 con 0x7f2534109530 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f253b400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f253411ccb0 con 0x7f253410d660 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f253b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f253411cfb0 con 0x7f2534117090 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f2539400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 0x7f25341b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f2539400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 0x7f25341b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51184/0 (socket says 172.21.15.44:51184) 2024-04-20T11:26:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.780+0000 7f2539400640 1 -- 172.21.15.44:0/443685517 learned_addr learned my addr 172.21.15.44:0/443685517 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2538a00640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2534109530 0x7f253410cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2533e00640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534117090 0x7f253410d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2539400640 1 -- 172.21.15.44:0/443685517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534117090 msgr2=0x7f253410d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2539400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534117090 0x7f253410d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2539400640 1 -- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2534109530 msgr2=0x7f253410cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2539400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2534109530 0x7f253410cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2539400640 1 -- 172.21.15.44:0/443685517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25341a3080 con 0x7f253410d660 2024-04-20T11:26:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2538a00640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2534109530 0x7f253410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.781+0000 7f2533e00640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534117090 0x7f253410d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.782+0000 7f2539400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 0x7f25341b38e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f252c004ae0 tx=0x7f252c034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.783+0000 7f2531600640 1 -- 172.21.15.44:0/443685517 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c039040 con 0x7f253410d660 2024-04-20T11:26:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.783+0000 7f2531600640 1 -- 172.21.15.44:0/443685517 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f252c044070 con 0x7f253410d660 2024-04-20T11:26:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.783+0000 7f2531600640 1 -- 172.21.15.44:0/443685517 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c004350 con 0x7f253410d660 2024-04-20T11:26:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.783+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25341a32b0 con 0x7f253410d660 2024-04-20T11:26:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.783+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25341a3750 con 0x7f253410d660 2024-04-20T11:26:06.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.788+0000 7f253e1bf740 1 Processor -- start 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.791+0000 7f2531600640 1 -- 172.21.15.44:0/443685517 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f252c003a00 con 0x7f253410d660 2024-04-20T11:26:06.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.791+0000 7f253e1bf740 1 -- start start 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1e74b0 0x558dfe1e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f253e1bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558dfe143050 con 0x558dfe1e74b0 2024-04-20T11:26:06.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f253e1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dfe0e9490 con 0x558dfe1ddfb0 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f253e1bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558dfe13f010 con 0x558dfe1de8d0 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f2533e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f2538a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f2533e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1de390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51188/0 (socket says 172.21.15.44:51188) 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f2538a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49560/0 (socket says 172.21.15.44:49560) 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f2533e00640 1 -- 172.21.15.44:0/2261864470 learned_addr learned my addr 172.21.15.44:0/2261864470 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.794+0000 7f2533e00640 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1e74b0 0x558dfe1e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2533e00640 1 -- 172.21.15.44:0/2261864470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 msgr2=0x558dfe1e6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2533e00640 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2533e00640 1 -- 172.21.15.44:0/2261864470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1e74b0 msgr2=0x558dfe1e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2533e00640 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1e74b0 0x558dfe1e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2533e00640 1 -- 172.21.15.44:0/2261864470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfe0f1070 con 0x558dfe1ddfb0 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2533e00640 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1de390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2520013530 tx=0x7f25200096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e2c8afd643e034 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2531600640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f251c0a3f20 0x7f251c0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.795+0000 7f2525c00640 1 -- 172.21.15.44:0/2261864470 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520019070 con 0x558dfe1ddfb0 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.796+0000 7f2531600640 1 -- 172.21.15.44:0/443685517 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f252c0f2710 con 0x7f253410d660 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.796+0000 7f2525c00640 1 -- 172.21.15.44:0/2261864470 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f252001eb60 con 0x558dfe1ddfb0 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.796+0000 7f2525c00640 1 -- 172.21.15.44:0/2261864470 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252000f460 con 0x558dfe1ddfb0 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.796+0000 7f2538a00640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f251c0a3f20 0x7f251c0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.797+0000 7f253e1bf740 1 -- 172.21.15.44:0/2261864470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 msgr2=0x558dfe1de390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.797+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1de390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2520013530 tx=0x7f25200096e0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.797+0000 7f2538a00640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f251c0a3f20 0x7f251c0a63e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x558dfe1dda80 tx=0x7f2528005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.798+0000 7f253e1bf740 1 -- 172.21.15.44:0/2261864470 shutdown_connections 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.798+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1e74b0 0x558dfe1e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.798+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.798+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2261864470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1de390 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.798+0000 7f253e1bf740 1 -- 172.21.15.44:0/2261864470 >> 172.21.15.44:0/2261864470 conn(0x558dfe1d86e0 msgr2=0x558dfe1d8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.799+0000 7f253e1bf740 1 -- 172.21.15.44:0/2261864470 shutdown_connections 2024-04-20T11:26:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.799+0000 7f253e1bf740 1 -- 172.21.15.44:0/2261864470 wait complete. 2024-04-20T11:26:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.799+0000 7f253e1bf740 1 -- start start 2024-04-20T11:26:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.800+0000 7f253e1bf740 1 Processor -- start 2024-04-20T11:26:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.801+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.801+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.801+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.801+0000 7f2538a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f2538a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60454/0 (socket says 172.21.15.44:60454) 2024-04-20T11:26:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f2538a00640 1 -- 172.21.15.44:0/2848875027 learned_addr learned my addr 172.21.15.44:0/2848875027 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558dfe1ea2a0 con 0x558dfe1ddfb0 2024-04-20T11:26:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dfe0fc860 con 0x558dfe1e74b0 2024-04-20T11:26:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558dfe1ea420 con 0x558dfe1de8d0 2024-04-20T11:26:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f253e1bf740 10 client.0 _ll_register_callbacks cb 0x558dfe0cafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:06.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.802+0000 7f2539400640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2538a00640 1 -- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 msgr2=0x558dfe1e4a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2533e00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2538a00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2538a00640 1 -- 172.21.15.44:0/2848875027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 msgr2=0x558dfe1e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2538a00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2538a00640 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfe100950 con 0x558dfe1ddfb0 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2539400640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2533e00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f2538a00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e4540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2528011730 tx=0x7f25280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c4a808e2d6ed9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.803+0000 7f251a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:06.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.804+0000 7f251a000640 1 -- 172.21.15.44:0/2848875027 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528010920 con 0x558dfe1ddfb0 2024-04-20T11:26:06.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.805+0000 7f251a000640 1 -- 172.21.15.44:0/2848875027 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2528010ac0 con 0x558dfe1ddfb0 2024-04-20T11:26:06.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.805+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252001eb60 con 0x558dfe1ddfb0 2024-04-20T11:26:06.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.805+0000 7f251a000640 1 -- 172.21.15.44:0/2848875027 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2528010cc0 con 0x558dfe1ddfb0 2024-04-20T11:26:06.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.806+0000 7f253e1bf740 10 client.16194 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:06.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.806+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558dfe0ce820 con 0x558dfe1ddfb0 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.806+0000 7f251a000640 1 -- 172.21.15.44:0/2848875027 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f252001eb60 con 0x558dfe1ddfb0 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f251a000640 1 -- 172.21.15.44:0/2848875027 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f252800f040 con 0x558dfe1ddfb0 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f253e1bf740 10 client.16194 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f253e1bf740 10 client.16194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f253e1bf740 20 client.16194 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe1f7550 0x558dfe1f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f253e1bf740 4 client.16194 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:06.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dfe150a70 con 0x558dfe1f7550 2024-04-20T11:26:06.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.807+0000 7f2533e00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe1f7550 0x558dfe1f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.808+0000 7f2533e00640 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe1f7550 0x558dfe1f99e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5de1146350be037 server_cookie=ce03ed230f7ed8ed in_seq=0 out_seq=0 2024-04-20T11:26:06.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.808+0000 7f251a000640 10 client.16194 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:06.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.809+0000 7f251a000640 1 -- 172.21.15.44:0/2848875027 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dfe150a70 con 0x558dfe1f7550 2024-04-20T11:26:06.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.809+0000 7f251a000640 10 client.16194 handle_command_reply: tid=0 2024-04-20T11:26:06.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.809+0000 7f253e1bf740 1 client.16194 shutdown 2024-04-20T11:26:06.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.809+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 msgr2=0x558dfe1e4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.809+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e4540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2528011730 tx=0x7f25280046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 shutdown_connections 2024-04-20T11:26:06.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe1f7550 0x558dfe1f99e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f253e1bf740 1 --2- 172.21.15.44:0/2848875027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e4540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f251a000640 0 client.16194 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.810+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 >> 172.21.15.44:0/2848875027 conn(0x558dfe1d86e0 msgr2=0x558dfe1da430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.811+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 shutdown_connections 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.811+0000 7f253e1bf740 1 -- 172.21.15.44:0/2848875027 wait complete. 2024-04-20T11:26:06.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.811+0000 7f253e1bf740 20 client.16194 trim_cache size 0 max 16384 2024-04-20T11:26:06.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 Processor -- start 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 -- start start 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25341c5330 con 0x558dfe1ddfb0 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25341c7050 con 0x558dfe1e74b0 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.826+0000 7f253e1bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25341c5a00 con 0x558dfe1de8d0 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2538a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1ff2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51216/0 (socket says 172.21.15.44:51216) 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 -- 172.21.15.44:0/865227649 learned_addr learned my addr 172.21.15.44:0/865227649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2533e00640 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 -- 172.21.15.44:0/865227649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 msgr2=0x558dfe1f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 -- 172.21.15.44:0/865227649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 msgr2=0x558dfe1e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 -- 172.21.15.44:0/865227649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfde553f0 con 0x558dfe1e74b0 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2538a00640 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.827+0000 7f2539400640 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1ff2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f252c0bb150 tx=0x7f252c009170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dea6fe89d0b01e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.828+0000 7f2525200640 1 -- 172.21.15.44:0/865227649 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0be7d0 con 0x558dfe1e74b0 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.828+0000 7f2525200640 1 -- 172.21.15.44:0/865227649 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f252c0c2b80 con 0x558dfe1e74b0 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.828+0000 7f2525200640 1 -- 172.21.15.44:0/865227649 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0c6fb0 con 0x558dfe1e74b0 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.828+0000 7f253e1bf740 1 -- 172.21.15.44:0/865227649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 msgr2=0x558dfe1ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.828+0000 7f253e1bf740 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1ff2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f252c0bb150 tx=0x7f252c009170 comp rx=0 tx=0).stop 2024-04-20T11:26:06.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 -- 172.21.15.44:0/865227649 shutdown_connections 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1e74b0 0x558dfe1ff2f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1de8d0 0x558dfe1f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 --2- 172.21.15.44:0/865227649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1ddfb0 0x558dfe1e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 -- 172.21.15.44:0/865227649 >> 172.21.15.44:0/865227649 conn(0x558dfe1d86e0 msgr2=0x558dfe1d7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 -- 172.21.15.44:0/865227649 shutdown_connections 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 -- 172.21.15.44:0/865227649 wait complete. 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 -- start start 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.829+0000 7f253e1bf740 1 Processor -- start 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 0x558dfe1e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1e74b0 0x558dfe1e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25341c68c0 con 0x558dfe1de8d0 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f25341c6160 con 0x558dfe1ddfb0 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f253410e110 con 0x558dfe1e74b0 2024-04-20T11:26:06.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f253e1bf740 10 client.0 _ll_register_callbacks cb 0x7f25280113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 0x558dfe1e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 0x558dfe1e0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60478/0 (socket says 172.21.15.44:60478) 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 -- 172.21.15.44:0/441219551 learned_addr learned my addr 172.21.15.44:0/441219551 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2539400640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1e74b0 0x558dfe1e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2538a00640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 -- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1e74b0 msgr2=0x558dfe1e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1e74b0 0x558dfe1e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 -- 172.21.15.44:0/441219551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 msgr2=0x558dfe1efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.830+0000 7f2533e00640 1 -- 172.21.15.44:0/441219551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfdf1f7d0 con 0x558dfe1de8d0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.831+0000 7f2533e00640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 0x558dfe1e0f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2520013e80 tx=0x7f2520009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f873ccda13509600 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.831+0000 7f251a000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.831+0000 7f251a000640 1 -- 172.21.15.44:0/441219551 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252002b030 con 0x558dfe1de8d0 2024-04-20T11:26:06.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.831+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252c0c2b80 con 0x558dfe1de8d0 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.831+0000 7f251a000640 1 -- 172.21.15.44:0/441219551 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2520013250 con 0x558dfe1de8d0 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.831+0000 7f251a000640 1 -- 172.21.15.44:0/441219551 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2520019040 con 0x558dfe1de8d0 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 10 client.16200 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558dfe25d9b0 con 0x558dfe1de8d0 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f251a000640 1 -- 172.21.15.44:0/441219551 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f252c0c2b80 con 0x558dfe1de8d0 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f251a000640 1 -- 172.21.15.44:0/441219551 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f2520038050 con 0x558dfe1de8d0 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 10 client.16200 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 10 client.16200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 20 client.16200 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe25e010 0x558dfe260430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 4 client.16200 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:06.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.832+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dfe14a8e0 con 0x558dfe25e010 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.833+0000 7f2538a00640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe25e010 0x558dfe260430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.833+0000 7f2538a00640 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe25e010 0x558dfe260430 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9cf8024c490736c server_cookie=e7ed788f8f5a0804 in_seq=0 out_seq=0 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.833+0000 7f251a000640 10 client.16200 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.833+0000 7f251a000640 1 -- 172.21.15.44:0/441219551 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dfe14a8e0 con 0x558dfe25e010 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.833+0000 7f251a000640 10 client.16200 handle_command_reply: tid=0 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.833+0000 7f253e1bf740 1 client.16200 shutdown 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 msgr2=0x558dfe1e0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 0x558dfe1e0f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2520013e80 tx=0x7f2520009d10 comp rx=0 tx=0).stop 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 shutdown_connections 2024-04-20T11:26:06.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dfe25e010 0x558dfe260430 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfe1e74b0 0x558dfe1e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfe1de8d0 0x558dfe1e0f40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 --2- 172.21.15.44:0/441219551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfe1ddfb0 0x558dfe1efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 >> 172.21.15.44:0/441219551 conn(0x558dfe1d86e0 msgr2=0x558dfe1e4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f251a000640 0 client.16200 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 shutdown_connections 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 1 -- 172.21.15.44:0/441219551 wait complete. 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.834+0000 7f253e1bf740 20 client.16200 trim_cache size 0 max 16384 2024-04-20T11:26:06.840 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.835+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f251c0a3f20 msgr2=0x7f251c0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.835+0000 7f253b400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f251c0a3f20 0x7f251c0a63e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x558dfe1dda80 tx=0x7f2528005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.835+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 msgr2=0x7f25341b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.835+0000 7f253b400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 0x7f25341b38e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f252c004ae0 tx=0x7f252c034440 comp rx=0 tx=0).stop 2024-04-20T11:26:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 shutdown_connections 2024-04-20T11:26:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f253410d660 0x7f25341b38e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2534117090 0x7f253410d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f251c0a3f20 0x7f251c0a63e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 --2- 172.21.15.44:0/443685517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2534109530 0x7f253410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 >> 172.21.15.44:0/443685517 conn(0x7f253406d070 msgr2=0x7f2534117470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 shutdown_connections 2024-04-20T11:26:06.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:06.836+0000 7f253b400640 1 -- 172.21.15.44:0/443685517 wait complete. 2024-04-20T11:26:06.853 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:26:07.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 -- 172.21.15.44:0/3291706059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc103080 msgr2=0x7f17fc103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 --2- 172.21.15.44:0/3291706059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc103080 0x7f17fc103460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009860 tx=0x7f17ec02bc10 comp rx=0 tx=0).stop 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 -- 172.21.15.44:0/3291706059 shutdown_connections 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 --2- 172.21.15.44:0/3291706059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17fc10c450 0x7f17fc10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 --2- 172.21.15.44:0/3291706059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc1039a0 0x7f17fc10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 --2- 172.21.15.44:0/3291706059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc103080 0x7f17fc103460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.034+0000 7f1801400640 1 -- 172.21.15.44:0/3291706059 >> 172.21.15.44:0/3291706059 conn(0x7f17fc0fc7e0 msgr2=0x7f17fc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.035+0000 7f1801400640 1 -- 172.21.15.44:0/3291706059 shutdown_connections 2024-04-20T11:26:07.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.035+0000 7f1801400640 1 -- 172.21.15.44:0/3291706059 wait complete. 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.036+0000 7f1801400640 1 Processor -- start 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.036+0000 7f1801400640 1 -- start start 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.036+0000 7f1801400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 0x7f17fc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.036+0000 7f1801400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17fc1039a0 0x7f17fc19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f1801400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc10c450 0x7f17fc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f1801400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112300 con 0x7f17fc103080 2024-04-20T11:26:07.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f1801400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112180 con 0x7f17fc10c450 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f1801400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc112480 con 0x7f17fc1039a0 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 0x7f17fc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 0x7f17fc19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60492/0 (socket says 172.21.15.44:60492) 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 -- 172.21.15.44:0/2083874139 learned_addr learned my addr 172.21.15.44:0/2083874139 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17f3400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17fc1039a0 0x7f17fc19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f1800a00640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc10c450 0x7f17fc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 -- 172.21.15.44:0/2083874139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17fc1039a0 msgr2=0x7f17fc19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17fc1039a0 0x7f17fc19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 -- 172.21.15.44:0/2083874139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc10c450 msgr2=0x7f17fc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc10c450 0x7f17fc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.037+0000 7f17fbe00640 1 -- 172.21.15.44:0/2083874139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc198da0 con 0x7f17fc103080 2024-04-20T11:26:07.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.038+0000 7f17fbe00640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 0x7f17fc19e420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0057c0 tx=0x7f17ec02c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.038+0000 7f17f9600640 1 -- 172.21.15.44:0/2083874139 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec041020 con 0x7f17fc103080 2024-04-20T11:26:07.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.038+0000 7f17f9600640 1 -- 172.21.15.44:0/2083874139 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17ec034940 con 0x7f17fc103080 2024-04-20T11:26:07.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.038+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc199030 con 0x7f17fc103080 2024-04-20T11:26:07.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.038+0000 7f17f9600640 1 -- 172.21.15.44:0/2083874139 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec039040 con 0x7f17fc103080 2024-04-20T11:26:07.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.038+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a52e0 con 0x7f17fc103080 2024-04-20T11:26:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 Processor -- start 2024-04-20T11:26:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 -- start start 2024-04-20T11:26:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f17f9600640 1 -- 172.21.15.44:0/2083874139 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f17ec044020 con 0x7f17fc103080 2024-04-20T11:26:07.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 0x55f72c5d4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5e1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5e1a80 0x55f72c5e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f72c539050 con 0x55f72c5e1a80 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f72c4df490 con 0x55f72c5d47c0 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.043+0000 7f180565f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f72c535010 con 0x55f72c5d3ea0 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f17fbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 0x55f72c5d4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 0x55f72c5d4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49598/0 (socket says 172.21.15.44:49598) 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 -- 172.21.15.44:0/253180465 learned_addr learned my addr 172.21.15.44:0/253180465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 -- 172.21.15.44:0/253180465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 msgr2=0x55f72c5e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5e1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 -- 172.21.15.44:0/253180465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5e1a80 msgr2=0x55f72c5e3e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5e1a80 0x55f72c5e3e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 -- 172.21.15.44:0/253180465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72c4e7070 con 0x55f72c5d3ea0 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.044+0000 7f1800a00640 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 0x55f72c5d4280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009f30 tx=0x7f17f40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5379526cb640f6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f17f1200640 1 -- 172.21.15.44:0/253180465 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004d20 con 0x55f72c5d3ea0 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f17f1200640 1 -- 172.21.15.44:0/253180465 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17f4004050 con 0x55f72c5d3ea0 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f17f1200640 1 -- 172.21.15.44:0/253180465 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4005280 con 0x55f72c5d3ea0 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f17f9600640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17e00a3d20 0x7f17e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f17f9600640 1 -- 172.21.15.44:0/2083874139 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17ec0f1ca0 con 0x7f17fc103080 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f17fbe00640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17e00a3d20 0x7f17e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f180565f740 1 -- 172.21.15.44:0/253180465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 msgr2=0x55f72c5d4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.045+0000 7f180565f740 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 0x55f72c5d4280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009f30 tx=0x7f17f40118d0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 -- 172.21.15.44:0/253180465 shutdown_connections 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5e1a80 0x55f72c5e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 --2- 172.21.15.44:0/253180465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5d3ea0 0x55f72c5d4280 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 -- 172.21.15.44:0/253180465 >> 172.21.15.44:0/253180465 conn(0x55f72c5ce8a0 msgr2=0x55f72c5cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f17fbe00640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17e00a3d20 0x7f17e00a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55f72c5d5080 tx=0x7f17ec0de040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 -- 172.21.15.44:0/253180465 shutdown_connections 2024-04-20T11:26:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.047+0000 7f180565f740 1 -- 172.21.15.44:0/253180465 wait complete. 2024-04-20T11:26:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.048+0000 7f180565f740 1 -- start start 2024-04-20T11:26:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.048+0000 7f180565f740 1 Processor -- start 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f72c5e4840 con 0x55f72c5d3ea0 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f72c4f2860 con 0x55f72c5d47c0 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f72c5e49c0 con 0x55f72c5e1a80 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f180565f740 10 client.0 _ll_register_callbacks cb 0x55f72c4c0fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f1800a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f1800a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5e13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60498/0 (socket says 172.21.15.44:60498) 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f1800a00640 1 -- 172.21.15.44:0/587895371 learned_addr learned my addr 172.21.15.44:0/587895371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f17f3400640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.049+0000 7f1800a00640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17f3400640 1 -- 172.21.15.44:0/587895371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 msgr2=0x55f72c5dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17f3400640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5dac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17f3400640 1 -- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 msgr2=0x55f72c5e13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17f3400640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5e13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17f3400640 1 -- 172.21.15.44:0/587895371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72c4f6950 con 0x55f72c5d47c0 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17f3400640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5da6f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0fee00 tx=0x7f17e4011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df287e1e3a4185b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17e9600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17e9600640 1 -- 172.21.15.44:0/587895371 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400f8f0 con 0x55f72c5d47c0 2024-04-20T11:26:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f17e9600640 1 -- 172.21.15.44:0/587895371 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17e400fa90 con 0x55f72c5d47c0 2024-04-20T11:26:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.050+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f4004050 con 0x55f72c5d47c0 2024-04-20T11:26:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.051+0000 7f17e9600640 1 -- 172.21.15.44:0/587895371 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17e400fc90 con 0x55f72c5d47c0 2024-04-20T11:26:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.051+0000 7f180565f740 10 client.26342 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.051+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f72c4c4820 con 0x55f72c5d47c0 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.051+0000 7f17e9600640 1 -- 172.21.15.44:0/587895371 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17f4004050 con 0x55f72c5d47c0 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f17e9600640 1 -- 172.21.15.44:0/587895371 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f17e4018070 con 0x55f72c5d47c0 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f180565f740 10 client.26342 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f180565f740 10 client.26342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f180565f740 20 client.26342 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f180565f740 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c606fc0 0x55f72c609430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f180565f740 4 client.26342 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:07.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72c546a70 con 0x55f72c606fc0 2024-04-20T11:26:07.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.052+0000 7f1800a00640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c606fc0 0x55f72c609430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.053+0000 7f1800a00640 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c606fc0 0x55f72c609430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6e94840bf6aee9a server_cookie=ab98daa352139af8 in_seq=0 out_seq=0 2024-04-20T11:26:07.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.053+0000 7f17e9600640 10 client.26342 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.054+0000 7f17e9600640 1 -- 172.21.15.44:0/587895371 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f72c546a70 con 0x55f72c606fc0 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.054+0000 7f17e9600640 10 client.26342 handle_command_reply: tid=0 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 client.26342 shutdown 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 msgr2=0x55f72c5da6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5da6f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0fee00 tx=0x7f17e4011b50 comp rx=0 tx=0).stop 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 shutdown_connections 2024-04-20T11:26:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c606fc0 0x55f72c609430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5da6f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 --2- 172.21.15.44:0/587895371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5e13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f17e9600640 0 client.26342 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 >> 172.21.15.44:0/587895371 conn(0x55f72c5ce8a0 msgr2=0x55f72c5d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 shutdown_connections 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.055+0000 7f180565f740 1 -- 172.21.15.44:0/587895371 wait complete. 2024-04-20T11:26:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.056+0000 7f180565f740 20 client.26342 trim_cache size 0 max 16384 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.070+0000 7f180565f740 1 Processor -- start 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 -- start start 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 0x55f72c5ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d47c0 0x55f72c5d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc071d80 con 0x55f72c5d47c0 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc073aa0 con 0x55f72c5d3ea0 2024-04-20T11:26:07.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f180565f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc072450 con 0x55f72c5e1a80 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f1800a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 0x55f72c5ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f17f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d47c0 0x55f72c5d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f1800a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 0x55f72c5ded10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51272/0 (socket says 172.21.15.44:51272) 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f1800a00640 1 -- 172.21.15.44:0/3051522071 learned_addr learned my addr 172.21.15.44:0/3051522071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.071+0000 7f1800a00640 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f1800a00640 1 -- 172.21.15.44:0/3051522071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 msgr2=0x55f72c5e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f1800a00640 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f1800a00640 1 -- 172.21.15.44:0/3051522071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d47c0 msgr2=0x55f72c5d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f1800a00640 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d47c0 0x55f72c5d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f1800a00640 1 -- 172.21.15.44:0/3051522071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72c24b3f0 con 0x55f72c5d3ea0 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f1800a00640 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 0x55f72c5ded10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17f4005c40 tx=0x7f17f4003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3760b4247e61497 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f17ebe00640 1 -- 172.21.15.44:0/3051522071 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f402fd50 con 0x55f72c5d3ea0 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f17ebe00640 1 -- 172.21.15.44:0/3051522071 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17f402f240 con 0x55f72c5d3ea0 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.072+0000 7f17ebe00640 1 -- 172.21.15.44:0/3051522071 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4004220 con 0x55f72c5d3ea0 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 -- 172.21.15.44:0/3051522071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 msgr2=0x55f72c5ded10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 0x55f72c5ded10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17f4005c40 tx=0x7f17f4003f60 comp rx=0 tx=0).stop 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 -- 172.21.15.44:0/3051522071 shutdown_connections 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d47c0 0x55f72c5d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 --2- 172.21.15.44:0/3051522071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d3ea0 0x55f72c5ded10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 -- 172.21.15.44:0/3051522071 >> 172.21.15.44:0/3051522071 conn(0x55f72c5ce8a0 msgr2=0x55f72c5cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 -- 172.21.15.44:0/3051522071 shutdown_connections 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 -- 172.21.15.44:0/3051522071 wait complete. 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.073+0000 7f180565f740 1 -- start start 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 Processor -- start 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc073310 con 0x55f72c5d3ea0 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc072bb0 con 0x55f72c5d47c0 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc06fcc0 con 0x55f72c5e1a80 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f180565f740 10 client.0 _ll_register_callbacks cb 0x7f17e4009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.074+0000 7f1800a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f1800a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60522/0 (socket says 172.21.15.44:60522) 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f1800a00640 1 -- 172.21.15.44:0/235317240 learned_addr learned my addr 172.21.15.44:0/235317240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f1800a00640 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 -- 172.21.15.44:0/235317240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 msgr2=0x55f72c5d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5d7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 -- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 msgr2=0x55f72c5f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 -- 172.21.15.44:0/235317240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72c3157d0 con 0x55f72c5d47c0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17f3400640 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5d73f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17e401a6d0 tx=0x7f17e400ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d70b35c9e5c75776 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17e9600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.075+0000 7f17e9600640 1 -- 172.21.15.44:0/235317240 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4027070 con 0x55f72c5d47c0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f402f240 con 0x55f72c5d47c0 2024-04-20T11:26:07.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f17e9600640 1 -- 172.21.15.44:0/235317240 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17e401ab40 con 0x55f72c5d47c0 2024-04-20T11:26:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f17e9600640 1 -- 172.21.15.44:0/235317240 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17e401ad60 con 0x55f72c5d47c0 2024-04-20T11:26:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 10 client.26348 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f72c66d1b0 con 0x55f72c5d47c0 2024-04-20T11:26:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f17e9600640 1 -- 172.21.15.44:0/235317240 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17f402f240 con 0x55f72c5d47c0 2024-04-20T11:26:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f17e9600640 1 -- 172.21.15.44:0/235317240 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f17e4018070 con 0x55f72c5d47c0 2024-04-20T11:26:07.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 10 client.26348 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 10 client.26348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 20 client.26348 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c66da40 0x55f72c66fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 4 client.26348 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.076+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f72c5dafc0 con 0x55f72c66da40 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.077+0000 7f1800a00640 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c66da40 0x55f72c66fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.077+0000 7f1800a00640 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c66da40 0x55f72c66fe60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e85e3a8be33650f5 server_cookie=6df423abb6d5d1c in_seq=0 out_seq=0 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.077+0000 7f17e9600640 10 client.26348 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.077+0000 7f17e9600640 1 -- 172.21.15.44:0/235317240 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f72c5dafc0 con 0x55f72c66da40 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.077+0000 7f17e9600640 10 client.26348 handle_command_reply: tid=0 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 client.26348 shutdown 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 msgr2=0x55f72c5d73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5d73f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17e401a6d0 tx=0x7f17e400ff70 comp rx=0 tx=0).stop 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 shutdown_connections 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55f72c66da40 0x55f72c66fe60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72c5e1a80 0x55f72c5d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f72c5d47c0 0x55f72c5d73f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 --2- 172.21.15.44:0/235317240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f72c5d3ea0 0x55f72c5f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 >> 172.21.15.44:0/235317240 conn(0x55f72c5ce8a0 msgr2=0x55f72c5d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f17e9600640 0 client.26348 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 shutdown_connections 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 1 -- 172.21.15.44:0/235317240 wait complete. 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.078+0000 7f180565f740 20 client.26348 trim_cache size 0 max 16384 2024-04-20T11:26:07.083 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17e00a3d20 msgr2=0x7f17e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17e00a3d20 0x7f17e00a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55f72c5d5080 tx=0x7f17ec0de040 comp rx=0 tx=0).stop 2024-04-20T11:26:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 msgr2=0x7f17fc19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 0x7f17fc19e420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0057c0 tx=0x7f17ec02c630 comp rx=0 tx=0).stop 2024-04-20T11:26:07.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 shutdown_connections 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17fc10c450 0x7f17fc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17fc1039a0 0x7f17fc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17e00a3d20 0x7f17e00a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 --2- 172.21.15.44:0/2083874139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17fc103080 0x7f17fc19e420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 >> 172.21.15.44:0/2083874139 conn(0x7f17fc0fc7e0 msgr2=0x7f17fc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 shutdown_connections 2024-04-20T11:26:07.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.079+0000 7f1801400640 1 -- 172.21.15.44:0/2083874139 wait complete. 2024-04-20T11:26:07.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:26:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 -- 172.21.15.44:0/3041622042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72501038e0 msgr2=0x7f725010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 --2- 172.21.15.44:0/3041622042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72501038e0 0x7f725010bdc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f723c009900 tx=0x7f723c02bcc0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 -- 172.21.15.44:0/3041622042 shutdown_connections 2024-04-20T11:26:07.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 --2- 172.21.15.44:0/3041622042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f725010c3e0 0x7f725010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 --2- 172.21.15.44:0/3041622042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72501038e0 0x7f725010bdc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 --2- 172.21.15.44:0/3041622042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 -- 172.21.15.44:0/3041622042 >> 172.21.15.44:0/3041622042 conn(0x7f72500fc7e0 msgr2=0x7f72500fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 -- 172.21.15.44:0/3041622042 shutdown_connections 2024-04-20T11:26:07.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.307+0000 7f7255400640 1 -- 172.21.15.44:0/3041622042 wait complete. 2024-04-20T11:26:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 Processor -- start 2024-04-20T11:26:07.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 -- start start 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72501038e0 0x7f72501a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725010c3e0 0x7f725019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7250112300 con 0x7f7250102fc0 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7250112180 con 0x7f725010c3e0 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.309+0000 7f7255400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7250112480 con 0x7f72501038e0 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60538/0 (socket says 172.21.15.44:60538) 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 -- 172.21.15.44:0/2523495348 learned_addr learned my addr 172.21.15.44:0/2523495348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724f400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72501038e0 0x7f72501a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f7254a00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725010c3e0 0x7f725019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 -- 172.21.15.44:0/2523495348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72501038e0 msgr2=0x7f72501a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72501038e0 0x7f72501a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 -- 172.21.15.44:0/2523495348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725010c3e0 msgr2=0x7f725019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725010c3e0 0x7f725019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.310+0000 7f724fe00640 1 -- 172.21.15.44:0/2523495348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725019af90 con 0x7f7250102fc0 2024-04-20T11:26:07.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.311+0000 7f724fe00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501a0650 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f724000e9f0 tx=0x7f724000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.311+0000 7f724cc00640 1 -- 172.21.15.44:0/2523495348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724000cd80 con 0x7f7250102fc0 2024-04-20T11:26:07.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.311+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725019b280 con 0x7f7250102fc0 2024-04-20T11:26:07.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.311+0000 7f724cc00640 1 -- 172.21.15.44:0/2523495348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7240010930 con 0x7f7250102fc0 2024-04-20T11:26:07.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.311+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72501a74a0 con 0x7f7250102fc0 2024-04-20T11:26:07.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.311+0000 7f724cc00640 1 -- 172.21.15.44:0/2523495348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240018aa0 con 0x7f7250102fc0 2024-04-20T11:26:07.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.314+0000 7f725961f740 1 Processor -- start 2024-04-20T11:26:07.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.314+0000 7f724cc00640 1 -- 172.21.15.44:0/2523495348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7240018c40 con 0x7f7250102fc0 2024-04-20T11:26:07.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.314+0000 7f725961f740 1 -- start start 2024-04-20T11:26:07.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f725961f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a370550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f725961f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a379130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f725961f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f725961f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a2d5050 con 0x55bf3a379670 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f725961f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a27b490 con 0x55bf3a370170 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f725961f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a2d1010 con 0x55bf3a370a90 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f7254a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a370550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a379130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f7254a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a370550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51318/0 (socket says 172.21.15.44:51318) 2024-04-20T11:26:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f7254a00640 1 -- 172.21.15.44:0/1215548668 learned_addr learned my addr 172.21.15.44:0/1215548668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724fe00640 1 -- 172.21.15.44:0/1215548668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 msgr2=0x55bf3a370550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724fe00640 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a370550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724fe00640 1 -- 172.21.15.44:0/1215548668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 msgr2=0x55bf3a37ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724fe00640 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.315+0000 7f724fe00640 1 -- 172.21.15.44:0/1215548668 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3a283070 con 0x55bf3a370a90 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f724cc00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f72340a3d20 0x7f72340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f724fe00640 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a379130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f72400ce300 tx=0x7f72400d9660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c96658c599975a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f7254a00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f72340a3d20 0x7f72340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f724cc00640 1 -- 172.21.15.44:0/2523495348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f72400d0ba0 con 0x7f7250102fc0 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f7245200640 1 -- 172.21.15.44:0/1215548668 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400e0020 con 0x55bf3a370a90 2024-04-20T11:26:07.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f7245200640 1 -- 172.21.15.44:0/1215548668 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72400d5040 con 0x55bf3a370a90 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.316+0000 7f7245200640 1 -- 172.21.15.44:0/1215548668 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724009c7e0 con 0x55bf3a370a90 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 -- 172.21.15.44:0/1215548668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 msgr2=0x55bf3a379130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a379130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f72400ce300 tx=0x7f72400d9660 comp rx=0 tx=0).stop 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f7254a00640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f72340a3d20 0x7f72340a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7248009fd0 tx=0x7f7248009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 -- 172.21.15.44:0/1215548668 shutdown_connections 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a379130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 --2- 172.21.15.44:0/1215548668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a370550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 -- 172.21.15.44:0/1215548668 >> 172.21.15.44:0/1215548668 conn(0x55bf3a36a8a0 msgr2=0x55bf3a36ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 -- 172.21.15.44:0/1215548668 shutdown_connections 2024-04-20T11:26:07.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.317+0000 7f725961f740 1 -- 172.21.15.44:0/1215548668 wait complete. 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 -- start start 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 Processor -- start 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a376780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a376cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a377200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a37c460 con 0x55bf3a379670 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a28e860 con 0x55bf3a370170 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.318+0000 7f725961f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a37c5e0 con 0x55bf3a370a90 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f725961f740 10 client.0 _ll_register_callbacks cb 0x55bf3a25cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a376cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f7254a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a377200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a376cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49648/0 (socket says 172.21.15.44:49648) 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724f400640 1 -- 172.21.15.44:0/934060822 learned_addr learned my addr 172.21.15.44:0/934060822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a376780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 -- 172.21.15.44:0/934060822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 msgr2=0x55bf3a376cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a376cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 -- 172.21.15.44:0/934060822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 msgr2=0x55bf3a377200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a377200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 -- 172.21.15.44:0/934060822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3a292950 con 0x55bf3a370170 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724f400640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a376cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f724fe00640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a376780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f72400d4250 tx=0x7f72400d0e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f732ba2569eecf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.319+0000 7f7239600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f7239600640 1 -- 172.21.15.44:0/934060822 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400e0070 con 0x55bf3a370170 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f7239600640 1 -- 172.21.15.44:0/934060822 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f72400ce580 con 0x55bf3a370170 2024-04-20T11:26:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72400d5040 con 0x55bf3a370170 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f7239600640 1 -- 172.21.15.44:0/934060822 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f724009c600 con 0x55bf3a370170 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 10 client.26360 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bf3a260820 con 0x55bf3a370170 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f7239600640 1 -- 172.21.15.44:0/934060822 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72400d5040 con 0x55bf3a370170 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f7239600640 1 -- 172.21.15.44:0/934060822 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f7240010ae0 con 0x55bf3a370170 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 10 client.26360 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 10 client.26360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 20 client.26360 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.320+0000 7f725961f740 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3895d0 0x55bf3a38b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.321+0000 7f725961f740 4 client.26360 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.321+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf3a2e2a70 con 0x55bf3a3895d0 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.321+0000 7f724f400640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3895d0 0x55bf3a38b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.321+0000 7f724f400640 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3895d0 0x55bf3a38b9c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23092ff15430d5e6 server_cookie=7537420128c157c4 in_seq=0 out_seq=0 2024-04-20T11:26:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.321+0000 7f7239600640 10 client.26360 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f7239600640 1 -- 172.21.15.44:0/934060822 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf3a2e2a70 con 0x55bf3a3895d0 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f7239600640 10 client.26360 handle_command_reply: tid=0 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 client.26360 shutdown 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 msgr2=0x55bf3a376780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a376780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f72400d4250 tx=0x7f72400d0e20 comp rx=0 tx=0).stop 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 shutdown_connections 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3895d0 0x55bf3a38b9c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a377200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370a90 0x55bf3a376cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f7239600640 0 client.26360 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 --2- 172.21.15.44:0/934060822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a376780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 >> 172.21.15.44:0/934060822 conn(0x55bf3a36a8a0 msgr2=0x55bf3a36c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 shutdown_connections 2024-04-20T11:26:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.322+0000 7f725961f740 1 -- 172.21.15.44:0/934060822 wait complete. 2024-04-20T11:26:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.323+0000 7f725961f740 20 client.26360 trim_cache size 0 max 16384 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 Processor -- start 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 -- start start 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a385320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a370a90 0x55bf3a378e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 0x55bf3a36c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72501afcd0 con 0x55bf3a370a90 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72501b19f0 con 0x55bf3a370170 2024-04-20T11:26:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.337+0000 7f725961f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72501b03a0 con 0x55bf3a379670 2024-04-20T11:26:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f724f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a370a90 0x55bf3a378e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 0x55bf3a36c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 0x55bf3a36c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49652/0 (socket says 172.21.15.44:49652) 2024-04-20T11:26:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 -- 172.21.15.44:0/1816463940 learned_addr learned my addr 172.21.15.44:0/1816463940 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f724fe00640 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a385320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 -- 172.21.15.44:0/1816463940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 msgr2=0x55bf3a385320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a385320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 -- 172.21.15.44:0/1816463940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a370a90 msgr2=0x55bf3a378e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a370a90 0x55bf3a378e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.338+0000 7f7254a00640 1 -- 172.21.15.44:0/1816463940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf39fe73f0 con 0x55bf3a379670 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.339+0000 7f7254a00640 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 0x55bf3a36c3e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f724801e4d0 tx=0x7f7248019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d115d5d81cc0bf65 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.339+0000 7f723be00640 1 -- 172.21.15.44:0/1816463940 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248011500 con 0x55bf3a379670 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.339+0000 7f723be00640 1 -- 172.21.15.44:0/1816463940 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7248011ba0 con 0x55bf3a379670 2024-04-20T11:26:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.339+0000 7f723be00640 1 -- 172.21.15.44:0/1816463940 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724801cc60 con 0x55bf3a379670 2024-04-20T11:26:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 -- 172.21.15.44:0/1816463940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 msgr2=0x55bf3a36c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 0x55bf3a36c3e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f724801e4d0 tx=0x7f7248019000 comp rx=0 tx=0).stop 2024-04-20T11:26:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 -- 172.21.15.44:0/1816463940 shutdown_connections 2024-04-20T11:26:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a379670 0x55bf3a36c3e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a370a90 0x55bf3a378e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 --2- 172.21.15.44:0/1816463940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370170 0x55bf3a385320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 -- 172.21.15.44:0/1816463940 >> 172.21.15.44:0/1816463940 conn(0x55bf3a36a8a0 msgr2=0x55bf3a36bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 -- 172.21.15.44:0/1816463940 shutdown_connections 2024-04-20T11:26:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.340+0000 7f725961f740 1 -- 172.21.15.44:0/1816463940 wait complete. 2024-04-20T11:26:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.341+0000 7f725961f740 1 -- start start 2024-04-20T11:26:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.341+0000 7f725961f740 1 Processor -- start 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.341+0000 7f725961f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 0x55bf3a381d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.341+0000 7f725961f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370a90 0x55bf3a37d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.341+0000 7f725961f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f725961f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72501b1260 con 0x55bf3a379670 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f725961f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72501b0b00 con 0x55bf3a370a90 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f725961f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72501adc10 con 0x55bf3a370170 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f725961f740 10 client.0 _ll_register_callbacks cb 0x7f72400d4b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 0x55bf3a381d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f7254a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 0x55bf3a381d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:49666/0 (socket says 172.21.15.44:49666) 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 -- 172.21.15.44:0/2422681239 learned_addr learned my addr 172.21.15.44:0/2422681239 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724f400640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370a90 0x55bf3a37d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 -- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370a90 msgr2=0x55bf3a37d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370a90 0x55bf3a37d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 -- 172.21.15.44:0/2422681239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 msgr2=0x55bf3a37d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 -- 172.21.15.44:0/2422681239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3a0b17d0 con 0x55bf3a370170 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f7254a00640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f724fe00640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 0x55bf3a381d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7240099670 tx=0x7f72400d0f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33158e3ccc3c707 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.342+0000 7f7239600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.343+0000 7f7239600640 1 -- 172.21.15.44:0/2422681239 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400e0040 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.343+0000 7f7239600640 1 -- 172.21.15.44:0/2422681239 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f72400ce580 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.343+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7248011ba0 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.343+0000 7f7239600640 1 -- 172.21.15.44:0/2422681239 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f724009ca70 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.343+0000 7f725961f740 10 client.26362 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.343+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bf3a3ef7a0 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f7239600640 1 -- 172.21.15.44:0/2422681239 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7248011ba0 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f7239600640 1 -- 172.21.15.44:0/2422681239 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f724009c1f0 con 0x55bf3a370170 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f725961f740 10 client.26362 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f725961f740 10 client.26362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f725961f740 20 client.26362 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f725961f740 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3efc50 0x55bf3a3f2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f725961f740 4 client.26362 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf3a2dc8e0 con 0x55bf3a3efc50 2024-04-20T11:26:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.344+0000 7f724f400640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3efc50 0x55bf3a3f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:07.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f724f400640 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3efc50 0x55bf3a3f2070 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b7b6e901b66af4 server_cookie=554373c1a9bcd87d in_seq=0 out_seq=0 2024-04-20T11:26:07.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f7239600640 10 client.26362 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f7239600640 1 -- 172.21.15.44:0/2422681239 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf3a2dc8e0 con 0x55bf3a3efc50 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f7239600640 10 client.26362 handle_command_reply: tid=0 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 client.26362 shutdown 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 msgr2=0x55bf3a381d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 0x55bf3a381d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7240099670 tx=0x7f72400d0f20 comp rx=0 tx=0).stop 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 shutdown_connections 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bf3a3efc50 0x55bf3a3f2070 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf3a379670 0x55bf3a37d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f7239600640 0 client.26362 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3a370a90 0x55bf3a37d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 --2- 172.21.15.44:0/2422681239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf3a370170 0x55bf3a381d90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 >> 172.21.15.44:0/2422681239 conn(0x55bf3a36a8a0 msgr2=0x55bf3a36c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 shutdown_connections 2024-04-20T11:26:07.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.345+0000 7f725961f740 1 -- 172.21.15.44:0/2422681239 wait complete. 2024-04-20T11:26:07.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f725961f740 20 client.26362 trim_cache size 0 max 16384 2024-04-20T11:26:07.350 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f72340a3d20 msgr2=0x7f72340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f72340a3d20 0x7f72340a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7248009fd0 tx=0x7f7248009500 comp rx=0 tx=0).stop 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 msgr2=0x7f72501a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501a0650 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f724000e9f0 tx=0x7f724000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 shutdown_connections 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725010c3e0 0x7f725019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72501038e0 0x7f72501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f72340a3d20 0x7f72340a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 --2- 172.21.15.44:0/2523495348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7250102fc0 0x7f72501a0650 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:07.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.346+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 >> 172.21.15.44:0/2523495348 conn(0x7f72500fc7e0 msgr2=0x7f7250100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.347+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 shutdown_connections 2024-04-20T11:26:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:07.347+0000 7f7255400640 1 -- 172.21.15.44:0/2523495348 wait complete. 2024-04-20T11:26:07.559 DEBUG:teuthology.parallel:result is None 2024-04-20T11:26:07.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T11:26:07.628 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:07 smithi044 ceph-mon[16855]: pgmap v108: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 773 KiB/s wr, 188 op/s 2024-04-20T11:26:07.648 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T11:26:07.649 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-20T11:26:07.680 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-20T11:26:07.680 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T11:26:07.680 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:26:07.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-20T11:26:07.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:07 smithi078 ceph-mon[21464]: pgmap v108: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 773 KiB/s wr, 188 op/s 2024-04-20T11:26:07.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:07 smithi165 ceph-mon[21975]: pgmap v108: 97 pgs: 97 active+clean; 262 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 773 KiB/s wr, 188 op/s 2024-04-20T11:26:07.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-20T11:26:08.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:26:08.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:26:08.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 -- 172.21.15.44:0/2296942941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 msgr2=0x7ff960115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 --2- 172.21.15.44:0/2296942941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff960115200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff958009860 tx=0x7ff95802bc10 comp rx=0 tx=0).stop 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 -- 172.21.15.44:0/2296942941 shutdown_connections 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 --2- 172.21.15.44:0/2296942941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff960115200 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 --2- 172.21.15.44:0/2296942941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff9601128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 --2- 172.21.15.44:0/2296942941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff960074300 0x7ff9600746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 -- 172.21.15.44:0/2296942941 >> 172.21.15.44:0/2296942941 conn(0x7ff9600fd930 msgr2=0x7ff9600ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 -- 172.21.15.44:0/2296942941 shutdown_connections 2024-04-20T11:26:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.591+0000 7ff964e00640 1 -- 172.21.15.44:0/2296942941 wait complete. 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 Processor -- start 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 -- start start 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff960074300 0x7ff96019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff96019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff9601a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff960118930 con 0x7ff960074c20 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601187b0 con 0x7ff960112e10 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.592+0000 7ff964e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff960118ab0 con 0x7ff960074300 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff96019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff9601a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff96019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36716/0 (socket says 172.21.15.44:36716) 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95ea00640 1 -- 172.21.15.44:0/3717500848 learned_addr learned my addr 172.21.15.44:0/3717500848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95f400640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff960074300 0x7ff96019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95fe00640 1 -- 172.21.15.44:0/3717500848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff960074300 msgr2=0x7ff96019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95fe00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff960074300 0x7ff96019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95fe00640 1 -- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 msgr2=0x7ff96019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95fe00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff96019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95fe00640 1 -- 172.21.15.44:0/3717500848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9601a3dc0 con 0x7ff960112e10 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.593+0000 7ff95ea00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff96019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.594+0000 7ff95fe00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff9601a3690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff958009780 tx=0x7ff95802c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.594+0000 7ff957e00640 1 -- 172.21.15.44:0/3717500848 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958041030 con 0x7ff960112e10 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.594+0000 7ff957e00640 1 -- 172.21.15.44:0/3717500848 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff9580044a0 con 0x7ff960112e10 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.594+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9601a4050 con 0x7ff960112e10 2024-04-20T11:26:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.594+0000 7ff957e00640 1 -- 172.21.15.44:0/3717500848 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958039040 con 0x7ff960112e10 2024-04-20T11:26:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.594+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9601ab940 con 0x7ff960112e10 2024-04-20T11:26:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.597+0000 7ff957e00640 1 -- 172.21.15.44:0/3717500848 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff958044070 con 0x7ff960112e10 2024-04-20T11:26:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.597+0000 7ff96907b740 1 Processor -- start 2024-04-20T11:26:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 -- start start 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x56154970f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970fa10 0x5615497180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 0x56154971a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561549673d20 con 0x5615497185f0 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56154961a3e0 con 0x56154970f0f0 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff96907b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56154966fce0 con 0x56154970fa10 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 0x56154971a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 0x56154971a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36724/0 (socket says 172.21.15.44:36724) 2024-04-20T11:26:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff95ea00640 1 -- 172.21.15.44:0/1983774988 learned_addr learned my addr 172.21.15.44:0/1983774988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff95f400640 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970fa10 0x5615497180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.598+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x56154970f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95ea00640 1 -- 172.21.15.44:0/1983774988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970fa10 msgr2=0x5615497180b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970fa10 0x5615497180b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95ea00640 1 -- 172.21.15.44:0/1983774988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 msgr2=0x56154970f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x56154970f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff957e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff9440a3d20 0x7ff9440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95ea00640 1 -- 172.21.15.44:0/1983774988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561549639980 con 0x5615497185f0 2024-04-20T11:26:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff957e00640 1 -- 172.21.15.44:0/3717500848 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff9580f1760 con 0x7ff960112e10 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 0x56154971a9e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c470 tx=0x7ff94c010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7585b570e4810675 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.599+0000 7ff95fe00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff9440a3d20 0x7ff9440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff954c00640 1 -- 172.21.15.44:0/1983774988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c003970 con 0x5615497185f0 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff954c00640 1 -- 172.21.15.44:0/1983774988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94c004520 con 0x5615497185f0 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff954c00640 1 -- 172.21.15.44:0/1983774988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00b790 con 0x5615497185f0 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 -- 172.21.15.44:0/1983774988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 msgr2=0x56154971a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 0x56154971a9e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c470 tx=0x7ff94c010040 comp rx=0 tx=0).stop 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff95fe00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff9440a3d20 0x7ff9440a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff9580091b0 tx=0x7ff9580a8040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 -- 172.21.15.44:0/1983774988 shutdown_connections 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615497185f0 0x56154971a9e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970fa10 0x5615497180b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 --2- 172.21.15.44:0/1983774988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x56154970f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 -- 172.21.15.44:0/1983774988 >> 172.21.15.44:0/1983774988 conn(0x561549709820 msgr2=0x561549709c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.600+0000 7ff96907b740 1 -- 172.21.15.44:0/1983774988 shutdown_connections 2024-04-20T11:26:08.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.601+0000 7ff96907b740 1 -- 172.21.15.44:0/1983774988 wait complete. 2024-04-20T11:26:08.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.601+0000 7ff96907b740 1 -- start start 2024-04-20T11:26:08.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.601+0000 7ff96907b740 1 Processor -- start 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x5615497156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x561549715bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549716130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56154971b380 con 0x56154970fa10 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56154962d7a0 con 0x56154970f0f0 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56154971b500 con 0x5615497185f0 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff96907b740 10 client.0 _ll_register_callbacks cb 0x56154962fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x561549715bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x561549715bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36732/0 (socket says 172.21.15.44:36732) 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 -- 172.21.15.44:0/889348038 learned_addr learned my addr 172.21.15.44:0/889348038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95ea00640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x5615497156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95ea00640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549716130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 -- 172.21.15.44:0/889348038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 msgr2=0x561549716130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549716130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 -- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 msgr2=0x5615497156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x5615497156b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95f400640 1 -- 172.21.15.44:0/889348038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154944e7e0 con 0x56154970fa10 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.602+0000 7ff95ea00640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x5615497156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff95f400640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x561549715bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff950012740 tx=0x7ff950012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ce59339b4c80f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff949600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff949600640 1 -- 172.21.15.44:0/889348038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95000f920 con 0x56154970fa10 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff949600640 1 -- 172.21.15.44:0/889348038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff95000fac0 con 0x56154970fa10 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94c004520 con 0x56154970fa10 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff949600640 1 -- 172.21.15.44:0/889348038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff95000fcc0 con 0x56154970fa10 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff96907b740 10 client.16245 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:08.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.603+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5615494aef70 con 0x56154970fa10 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff949600640 1 -- 172.21.15.44:0/889348038 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94c004520 con 0x56154970fa10 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff949600640 1 -- 172.21.15.44:0/889348038 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7ff95001c420 con 0x56154970fa10 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff96907b740 10 client.16245 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff96907b740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff96907b740 20 client.16245 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff96907b740 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561549728aa0 0x56154972af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff96907b740 4 client.16245 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561549681740 con 0x561549728aa0 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.604+0000 7ff95ea00640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561549728aa0 0x56154972af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.605+0000 7ff95ea00640 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561549728aa0 0x56154972af20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929dd4d6a58bab70 server_cookie=6921eb1a3f1df799 in_seq=0 out_seq=0 2024-04-20T11:26:08.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.605+0000 7ff949600640 10 client.16245 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff949600640 1 -- 172.21.15.44:0/889348038 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561549681740 con 0x561549728aa0 2024-04-20T11:26:08.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff949600640 10 client.16245 handle_command_reply: tid=0 2024-04-20T11:26:08.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 client.16245 shutdown 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 msgr2=0x561549715bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x561549715bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff950012740 tx=0x7ff950012c00 comp rx=0 tx=0).stop 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 shutdown_connections 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561549728aa0 0x56154972af20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549716130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x561549715bf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 --2- 172.21.15.44:0/889348038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970f0f0 0x5615497156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff949600640 0 client.16245 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 >> 172.21.15.44:0/889348038 conn(0x561549709820 msgr2=0x56154970b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 shutdown_connections 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.606+0000 7ff96907b740 1 -- 172.21.15.44:0/889348038 wait complete. 2024-04-20T11:26:08.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.607+0000 7ff96907b740 20 client.16245 trim_cache size 0 max 16384 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.626+0000 7ff96907b740 1 Processor -- start 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.626+0000 7ff96907b740 1 -- start start 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff96907b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 0x561549717de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff96907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970fa10 0x561549724240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff96907b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549708880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff96907b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601b3fe0 con 0x56154970f0f0 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff96907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601b5d00 con 0x56154970fa10 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff96907b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601b46b0 con 0x5615497185f0 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 0x561549717de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 0x561549717de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36740/0 (socket says 172.21.15.44:36740) 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 -- 172.21.15.44:0/175603194 learned_addr learned my addr 172.21.15.44:0/175603194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95f400640 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970fa10 0x561549724240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549708880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 -- 172.21.15.44:0/175603194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 msgr2=0x561549708880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549708880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 -- 172.21.15.44:0/175603194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970fa10 msgr2=0x561549724240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970fa10 0x561549724240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95ea00640 1 -- 172.21.15.44:0/175603194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154970f840 con 0x56154970f0f0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.627+0000 7ff95f400640 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970fa10 0x561549724240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.628+0000 7ff95ea00640 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 0x561549717de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff94c003dc0 tx=0x7ff94c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f3f1f5955ab2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.628+0000 7ff94be00640 1 -- 172.21.15.44:0/175603194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c003a70 con 0x56154970f0f0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.628+0000 7ff94be00640 1 -- 172.21.15.44:0/175603194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94c02f0b0 con 0x56154970f0f0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.628+0000 7ff94be00640 1 -- 172.21.15.44:0/175603194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00b7b0 con 0x56154970f0f0 2024-04-20T11:26:08.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 -- 172.21.15.44:0/175603194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 msgr2=0x561549717de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 0x561549717de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff94c003dc0 tx=0x7ff94c009f90 comp rx=0 tx=0).stop 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 -- 172.21.15.44:0/175603194 shutdown_connections 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615497185f0 0x561549708880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56154970fa10 0x561549724240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 --2- 172.21.15.44:0/175603194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970f0f0 0x561549717de0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 -- 172.21.15.44:0/175603194 >> 172.21.15.44:0/175603194 conn(0x561549709820 msgr2=0x561549708170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 -- 172.21.15.44:0/175603194 shutdown_connections 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 -- 172.21.15.44:0/175603194 wait complete. 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.629+0000 7ff96907b740 1 -- start start 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 Processor -- start 2024-04-20T11:26:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970f0f0 0x56154971c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x56154971c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615497185f0 0x56154971cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601b5570 con 0x56154970fa10 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601b4e10 con 0x5615497185f0 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601b1f20 con 0x56154970f0f0 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff96907b740 10 client.0 _ll_register_callbacks cb 0x7ff950009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff95f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x56154971c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff95f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x56154971c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36748/0 (socket says 172.21.15.44:36748) 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff95f400640 1 -- 172.21.15.44:0/1899013398 learned_addr learned my addr 172.21.15.44:0/1899013398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.630+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970f0f0 0x56154971c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615497185f0 0x56154971cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95f400640 1 -- 172.21.15.44:0/1899013398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970f0f0 msgr2=0x56154971c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95f400640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970f0f0 0x56154971c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95f400640 1 -- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615497185f0 msgr2=0x56154971cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95f400640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615497185f0 0x56154971cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95f400640 1 -- 172.21.15.44:0/1899013398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561549346170 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615497185f0 0x56154971cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff95f400640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x56154971c5e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff950001eb0 tx=0x7ff950005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cffeeee193fde50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff949600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff949600640 1 -- 172.21.15.44:0/1899013398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950029070 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94c02f0b0 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.631+0000 7ff949600640 1 -- 172.21.15.44:0/1899013398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff95001c420 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff949600640 1 -- 172.21.15.44:0/1899013398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff950034050 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 10 client.16251 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56154978f0f0 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff949600640 1 -- 172.21.15.44:0/1899013398 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff94c02f0b0 con 0x56154970fa10 2024-04-20T11:26:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff949600640 1 -- 172.21.15.44:0/1899013398 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7ff950003970 con 0x56154970fa10 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 10 client.16251 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 20 client.16251 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56154978f960 0x561549791d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 4 client.16251 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.632+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56154967b5b0 con 0x56154978f960 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.633+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56154978f960 0x561549791d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.633+0000 7ff95ea00640 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56154978f960 0x561549791d80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e1ed587dc6cf3 server_cookie=38873df30bcbfe86 in_seq=0 out_seq=0 2024-04-20T11:26:08.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.633+0000 7ff949600640 10 client.16251 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff949600640 1 -- 172.21.15.44:0/1899013398 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56154967b5b0 con 0x56154978f960 2024-04-20T11:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff949600640 10 client.16251 handle_command_reply: tid=0 2024-04-20T11:26:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 client.16251 shutdown 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 msgr2=0x56154971c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x56154971c5e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff950001eb0 tx=0x7ff950005c50 comp rx=0 tx=0).stop 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 shutdown_connections 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56154978f960 0x561549791d80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615497185f0 0x56154971cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56154970fa10 0x56154971c5e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 --2- 172.21.15.44:0/1899013398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56154970f0f0 0x56154971c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 >> 172.21.15.44:0/1899013398 conn(0x561549709820 msgr2=0x561549708a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.635+0000 7ff949600640 0 client.16251 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 shutdown_connections 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff96907b740 1 -- 172.21.15.44:0/1899013398 wait complete. 2024-04-20T11:26:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff96907b740 20 client.16251 trim_cache size 0 max 16384 2024-04-20T11:26:08.639 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:26:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "7f22ba9d-246e-47c5-aa8c-ad5b24dee873", 2024-04-20T11:26:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:26:08.639 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff9440a3d20 msgr2=0x7ff9440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff964e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff9440a3d20 0x7ff9440a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff9580091b0 tx=0x7ff9580a8040 comp rx=0 tx=0).stop 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 msgr2=0x7ff9601a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.636+0000 7ff964e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff9601a3690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff958009780 tx=0x7ff95802c7e0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 shutdown_connections 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff960112e10 0x7ff9601a3690 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff9440a3d20 0x7ff9440a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff960074c20 0x7ff96019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 --2- 172.21.15.44:0/3717500848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff960074300 0x7ff96019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 >> 172.21.15.44:0/3717500848 conn(0x7ff9600fd930 msgr2=0x7ff9600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 shutdown_connections 2024-04-20T11:26:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.637+0000 7ff964e00640 1 -- 172.21.15.44:0/3717500848 wait complete. 2024-04-20T11:26:08.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:26:08.655 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:26:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 -- 172.21.15.44:0/3388115172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 msgr2=0x7ff8cc074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/3388115172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc074770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009ca0 tx=0x7ff8bc02c180 comp rx=0 tx=0).stop 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 -- 172.21.15.44:0/3388115172 shutdown_connections 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/3388115172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/3388115172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc074cb0 0x7ff8cc10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/3388115172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc074770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 -- 172.21.15.44:0/3388115172 >> 172.21.15.44:0/3388115172 conn(0x7ff8cc0fd930 msgr2=0x7ff8cc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 -- 172.21.15.44:0/3388115172 shutdown_connections 2024-04-20T11:26:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.869+0000 7ff8cbe00640 1 -- 172.21.15.44:0/3388115172 wait complete. 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 Processor -- start 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 -- start start 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc074cb0 0x7ff8cc1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc114500 con 0x7ff8cc10e640 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc114380 con 0x7ff8cc074cb0 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.870+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc114680 con 0x7ff8cc074390 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8cb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8cb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc1027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36764/0 (socket says 172.21.15.44:36764) 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54710/0 (socket says 172.21.15.44:54710) 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8cb400640 1 -- 172.21.15.44:0/4093371101 learned_addr learned my addr 172.21.15.44:0/4093371101 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8ca000640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc074cb0 0x7ff8cc1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 -- 172.21.15.44:0/4093371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc074cb0 msgr2=0x7ff8cc1022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc074cb0 0x7ff8cc1022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 -- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 msgr2=0x7ff8cc1027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc1027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8caa00640 1 -- 172.21.15.44:0/4093371101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1ab050 con 0x7ff8cc074390 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.871+0000 7ff8cb400640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.872+0000 7ff8caa00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc101d60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006010 tx=0x7ff8bc0346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.872+0000 7ff8c3400640 1 -- 172.21.15.44:0/4093371101 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc039040 con 0x7ff8cc074390 2024-04-20T11:26:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.872+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc1ab2e0 con 0x7ff8cc074390 2024-04-20T11:26:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.872+0000 7ff8c3400640 1 -- 172.21.15.44:0/4093371101 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8bc044070 con 0x7ff8cc074390 2024-04-20T11:26:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.872+0000 7ff8c3400640 1 -- 172.21.15.44:0/4093371101 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0043c0 con 0x7ff8cc074390 2024-04-20T11:26:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.872+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1ab7f0 con 0x7ff8cc074390 2024-04-20T11:26:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.875+0000 7ff8d49c1740 1 Processor -- start 2024-04-20T11:26:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.875+0000 7ff8d49c1740 1 -- start start 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2ca0 0x558dfd2ab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 0x558dfd2adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558dfd208150 con 0x558dfd2ab880 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dfd1ae370 con 0x558dfd2a2380 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558dfd204110 con 0x558dfd2a2ca0 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8c3400640 1 -- 172.21.15.44:0/4093371101 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff8bc004560 con 0x7ff8cc074390 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 0x558dfd2adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 0x558dfd2adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36766/0 (socket says 172.21.15.44:36766) 2024-04-20T11:26:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8ca000640 1 -- 172.21.15.44:0/95874317 learned_addr learned my addr 172.21.15.44:0/95874317 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8cb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.876+0000 7ff8caa00640 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2ca0 0x558dfd2ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8ca000640 1 -- 172.21.15.44:0/95874317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2ca0 msgr2=0x558dfd2ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8ca000640 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2ca0 0x558dfd2ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8ca000640 1 -- 172.21.15.44:0/95874317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 msgr2=0x558dfd2a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8ca000640 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8ca000640 1 -- 172.21.15.44:0/95874317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfcfe47d0 con 0x558dfd2ab880 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8ca000640 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 0x558dfd2adc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009c60 tx=0x7ff8b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814bc8d0a7ad0df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8bbe00640 1 -- 172.21.15.44:0/95874317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004230 con 0x558dfd2ab880 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8bbe00640 1 -- 172.21.15.44:0/95874317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8b40043d0 con 0x558dfd2ab880 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.877+0000 7ff8bbe00640 1 -- 172.21.15.44:0/95874317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4019840 con 0x558dfd2ab880 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8c3400640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff8b00a3d20 0x7ff8b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8c3400640 1 -- 172.21.15.44:0/4093371101 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff8bc0f2a30 con 0x7ff8cc074390 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 -- 172.21.15.44:0/95874317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 msgr2=0x558dfd2adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8cb400640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff8b00a3d20 0x7ff8b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 0x558dfd2adc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009c60 tx=0x7ff8b4011810 comp rx=0 tx=0).stop 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 -- 172.21.15.44:0/95874317 shutdown_connections 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2ab880 0x558dfd2adc70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2ca0 0x558dfd2ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/95874317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 -- 172.21.15.44:0/95874317 >> 172.21.15.44:0/95874317 conn(0x558dfd29cab0 msgr2=0x558dfd29ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 -- 172.21.15.44:0/95874317 shutdown_connections 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.878+0000 7ff8d49c1740 1 -- 172.21.15.44:0/95874317 wait complete. 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.879+0000 7ff8cb400640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff8b00a3d20 0x7ff8b00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009fd0 tx=0x7ff8c4005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.879+0000 7ff8d49c1740 1 -- start start 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.879+0000 7ff8d49c1740 1 Processor -- start 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2ab880 0x558dfd2a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558dfd2ae670 con 0x558dfd2a2ca0 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dfd1c1740 con 0x558dfd2a2380 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558dfd2ae7f0 con 0x558dfd2ab880 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8d49c1740 10 client.0 _ll_register_callbacks cb 0x558dfd1bcb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8caa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36770/0 (socket says 172.21.15.44:36770) 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8ca000640 1 -- 172.21.15.44:0/1740031156 learned_addr learned my addr 172.21.15.44:0/1740031156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8cb400640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2ab880 0x558dfd2a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8caa00640 1 -- 172.21.15.44:0/1740031156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2ab880 msgr2=0x558dfd2a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8caa00640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2ab880 0x558dfd2a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8caa00640 1 -- 172.21.15.44:0/1740031156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 msgr2=0x558dfd2a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8caa00640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8caa00640 1 -- 172.21.15.44:0/1740031156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfcf1a3f0 con 0x558dfd2a2380 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8cb400640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2ab880 0x558dfd2a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.880+0000 7ff8ca000640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8caa00640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a8900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0f0270 tx=0x7ff8bc0099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02dab18cbc40222 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8b8c00640 1 -- 172.21.15.44:0/1740031156 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0be450 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8b8c00640 1 -- 172.21.15.44:0/1740031156 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff8bc0bcdf0 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b40043d0 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8b8c00640 1 -- 172.21.15.44:0/1740031156 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff8bc0d8050 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8d49c1740 10 client.26393 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.881+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558dfceda0f0 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8b8c00640 1 -- 172.21.15.44:0/1740031156 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8b40043d0 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8b8c00640 1 -- 172.21.15.44:0/1740031156 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7ff8bc03c030 con 0x558dfd2a2380 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8d49c1740 10 client.26393 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8d49c1740 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8d49c1740 20 client.26393 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd2bbc50 0x558dfd2be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8d49c1740 4 client.26393 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dfd215b70 con 0x558dfd2bbc50 2024-04-20T11:26:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.882+0000 7ff8ca000640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd2bbc50 0x558dfd2be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.883+0000 7ff8ca000640 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd2bbc50 0x558dfd2be110 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d80ee16611cb2b2 server_cookie=fc65fff3a779ad91 in_seq=0 out_seq=0 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.883+0000 7ff8b8c00640 10 client.26393 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8b8c00640 1 -- 172.21.15.44:0/1740031156 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dfd215b70 con 0x558dfd2bbc50 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8b8c00640 10 client.26393 handle_command_reply: tid=0 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8d49c1740 1 client.26393 shutdown 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 msgr2=0x558dfd2a8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a8900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0f0270 tx=0x7ff8bc0099c0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 shutdown_connections 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd2bbc50 0x558dfd2be110 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.884+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2ab880 0x558dfd2a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8b8c00640 0 client.26393 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/1740031156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2a2380 0x558dfd2a8900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 >> 172.21.15.44:0/1740031156 conn(0x558dfd29cab0 msgr2=0x558dfd29e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 shutdown_connections 2024-04-20T11:26:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8d49c1740 1 -- 172.21.15.44:0/1740031156 wait complete. 2024-04-20T11:26:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.885+0000 7ff8d49c1740 20 client.26393 trim_cache size 0 max 16384 2024-04-20T11:26:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.904+0000 7ff8d49c1740 1 Processor -- start 2024-04-20T11:26:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.904+0000 7ff8d49c1740 1 -- start start 2024-04-20T11:26:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.904+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.904+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd29b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.904+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1b4040 con 0x558dfd2a2ca0 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1b5d60 con 0x558dfd2ab880 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1b4710 con 0x558dfd2a2380 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54738/0 (socket says 172.21.15.44:54738) 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 -- 172.21.15.44:0/4277701871 learned_addr learned my addr 172.21.15.44:0/4277701871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8ca000640 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd29b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8cb400640 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 -- 172.21.15.44:0/4277701871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 msgr2=0x558dfd2c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 -- 172.21.15.44:0/4277701871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 msgr2=0x558dfd29b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd29b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8caa00640 1 -- 172.21.15.44:0/4277701871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfd2adc70 con 0x558dfd2a2380 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.905+0000 7ff8ca000640 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd29b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.906+0000 7ff8caa00640 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2b7560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009ca0 tx=0x7ff8bc035f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca5cd54a0034c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.906+0000 7ff8bb400640 1 -- 172.21.15.44:0/4277701871 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0be450 con 0x558dfd2a2380 2024-04-20T11:26:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.906+0000 7ff8bb400640 1 -- 172.21.15.44:0/4277701871 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8bc0beaf0 con 0x558dfd2a2380 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.906+0000 7ff8bb400640 1 -- 172.21.15.44:0/4277701871 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0df040 con 0x558dfd2a2380 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 -- 172.21.15.44:0/4277701871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 msgr2=0x558dfd2b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2b7560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009ca0 tx=0x7ff8bc035f40 comp rx=0 tx=0).stop 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 -- 172.21.15.44:0/4277701871 shutdown_connections 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd29b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/4277701871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2b7560 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 -- 172.21.15.44:0/4277701871 >> 172.21.15.44:0/4277701871 conn(0x558dfd29cab0 msgr2=0x558dfd29b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 -- 172.21.15.44:0/4277701871 shutdown_connections 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 -- 172.21.15.44:0/4277701871 wait complete. 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.907+0000 7ff8d49c1740 1 -- start start 2024-04-20T11:26:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 Processor -- start 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1b55d0 con 0x558dfd2a2ca0 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1b4e70 con 0x558dfd2ab880 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1b1f80 con 0x558dfd2a2380 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8d49c1740 10 client.0 _ll_register_callbacks cb 0x7ff8bc0f6010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36792/0 (socket says 172.21.15.44:36792) 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.908+0000 7ff8ca000640 1 -- 172.21.15.44:0/2197733855 learned_addr learned my addr 172.21.15.44:0/2197733855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8cb400640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8caa00640 1 -- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 msgr2=0x558dfd2a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8caa00640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8caa00640 1 -- 172.21.15.44:0/2197733855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 msgr2=0x558dfd2a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8caa00640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8caa00640 1 -- 172.21.15.44:0/2197733855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfd321390 con 0x558dfd2a2380 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8ca000640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8caa00640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2a56c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0bb470 tx=0x7ff8bc009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4406211e4cb9390b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8b8c00640 1 -- 172.21.15.44:0/2197733855 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0be450 con 0x558dfd2a2380 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8b8c00640 1 -- 172.21.15.44:0/2197733855 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff8bc0f0380 con 0x558dfd2a2380 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.909+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8bc0beaf0 con 0x558dfd2a2380 2024-04-20T11:26:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8b8c00640 1 -- 172.21.15.44:0/2197733855 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff8bc0be8c0 con 0x558dfd2a2380 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 10 client.26398 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558dfd321390 con 0x558dfd2a2380 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8b8c00640 1 -- 172.21.15.44:0/2197733855 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8bc0beaf0 con 0x558dfd2a2380 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8b8c00640 1 -- 172.21.15.44:0/2197733855 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7ff8bc0bdf40 con 0x558dfd2a2380 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 10 client.26398 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 10 client.26398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 20 client.26398 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd322a10 0x558dfd324e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 4 client.26398 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.910+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dfd20f9e0 con 0x558dfd322a10 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.911+0000 7ff8ca000640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd322a10 0x558dfd324e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.911+0000 7ff8ca000640 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd322a10 0x558dfd324e30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2313f04cd7bb42e4 server_cookie=c482cb1cbbc1a5cc in_seq=0 out_seq=0 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.911+0000 7ff8b8c00640 10 client.26398 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.911+0000 7ff8b8c00640 1 -- 172.21.15.44:0/2197733855 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558dfd20f9e0 con 0x558dfd322a10 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.911+0000 7ff8b8c00640 10 client.26398 handle_command_reply: tid=0 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.911+0000 7ff8d49c1740 1 client.26398 shutdown 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 msgr2=0x558dfd2a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2a56c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0bb470 tx=0x7ff8bc009610 comp rx=0 tx=0).stop 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 shutdown_connections 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558dfd322a10 0x558dfd324e30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dfd2ab880 0x558dfd2a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dfd2a2ca0 0x558dfd2a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8b8c00640 0 client.26398 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 --2- 172.21.15.44:0/2197733855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dfd2a2380 0x558dfd2a56c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 >> 172.21.15.44:0/2197733855 conn(0x558dfd29cab0 msgr2=0x558dfd29bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 shutdown_connections 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 1 -- 172.21.15.44:0/2197733855 wait complete. 2024-04-20T11:26:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.912+0000 7ff8d49c1740 20 client.26398 trim_cache size 0 max 16384 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "7f22ba9d-246e-47c5-aa8c-ad5b24dee873": { 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "7f22ba9d-246e-47c5-aa8c-ad5b24dee873", 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:26:08.916 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:08.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff8b00a3d20 msgr2=0x7ff8b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff8b00a3d20 0x7ff8b00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009fd0 tx=0x7ff8c4005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:08.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 msgr2=0x7ff8cc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:08.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc101d60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006010 tx=0x7ff8bc0346f0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 shutdown_connections 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff8b00a3d20 0x7ff8b00a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8cc10e640 0x7ff8cc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc074cb0 0x7ff8cc1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 --2- 172.21.15.44:0/4093371101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8cc074390 0x7ff8cc101d60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 >> 172.21.15.44:0/4093371101 conn(0x7ff8cc0fd930 msgr2=0x7ff8cc0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 shutdown_connections 2024-04-20T11:26:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:08.913+0000 7ff8cbe00640 1 -- 172.21.15.44:0/4093371101 wait complete. 2024-04-20T11:26:08.929 INFO:tasks.cephfs.filesystem:scrub status for tag:7f22ba9d-246e-47c5-aa8c-ad5b24dee873 - {'path': '/', 'tag': '7f22ba9d-246e-47c5-aa8c-ad5b24dee873', 'options': 'recursive,force'} 2024-04-20T11:26:09.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:09 smithi078 ceph-mon[21464]: pgmap v109: 97 pgs: 97 active+clean; 267 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 292 op/s 2024-04-20T11:26:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:09 smithi044 ceph-mon[16855]: pgmap v109: 97 pgs: 97 active+clean; 267 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 292 op/s 2024-04-20T11:26:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:09 smithi165 ceph-mon[21975]: pgmap v109: 97 pgs: 97 active+clean; 267 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 292 op/s 2024-04-20T11:26:10.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:10 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:26:10.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:10 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:10.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:10 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:26:10.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:10 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:10.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:10 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:26:10.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:10 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:10.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:10 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:26:10.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:10 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:10.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:10 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:26:10.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:10 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:10.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:10 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:26:10.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:10 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:11.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:11 smithi078 ceph-mon[21464]: pgmap v110: 97 pgs: 97 active+clean; 267 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 814 KiB/s wr, 207 op/s 2024-04-20T11:26:11.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:11 smithi044 ceph-mon[16855]: pgmap v110: 97 pgs: 97 active+clean; 267 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 814 KiB/s wr, 207 op/s 2024-04-20T11:26:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:11 smithi165 ceph-mon[21975]: pgmap v110: 97 pgs: 97 active+clean; 267 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 814 KiB/s wr, 207 op/s 2024-04-20T11:26:13.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:13 smithi078 ceph-mon[21464]: pgmap v111: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 829 KiB/s wr, 215 op/s 2024-04-20T11:26:13.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:13 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:26:13.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:13 smithi044 ceph-mon[16855]: pgmap v111: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 829 KiB/s wr, 215 op/s 2024-04-20T11:26:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:13 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:26:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:13 smithi165 ceph-mon[21975]: pgmap v111: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 829 KiB/s wr, 215 op/s 2024-04-20T11:26:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:13 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:26:15.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:15 smithi044 ceph-mon[16855]: pgmap v112: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 404 KiB/s wr, 111 op/s 2024-04-20T11:26:15.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:15 smithi165 ceph-mon[21975]: pgmap v112: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 404 KiB/s wr, 111 op/s 2024-04-20T11:26:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:15 smithi078 ceph-mon[21464]: pgmap v112: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 404 KiB/s wr, 111 op/s 2024-04-20T11:26:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:17 smithi165 ceph-mon[21975]: pgmap v113: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 404 KiB/s wr, 111 op/s 2024-04-20T11:26:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:17 smithi078 ceph-mon[21464]: pgmap v113: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 404 KiB/s wr, 111 op/s 2024-04-20T11:26:17.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:17 smithi044 ceph-mon[16855]: pgmap v113: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 404 KiB/s wr, 111 op/s 2024-04-20T11:26:19.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:19 smithi165 ceph-mon[21975]: pgmap v114: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 408 KiB/s wr, 116 op/s 2024-04-20T11:26:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:19 smithi078 ceph-mon[21464]: pgmap v114: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 408 KiB/s wr, 116 op/s 2024-04-20T11:26:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:19 smithi044 ceph-mon[16855]: pgmap v114: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 408 KiB/s wr, 116 op/s 2024-04-20T11:26:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:21 smithi165 ceph-mon[21975]: pgmap v115: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-20T11:26:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:21 smithi078 ceph-mon[21464]: pgmap v115: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-20T11:26:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:21 smithi044 ceph-mon[16855]: pgmap v115: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-04-20T11:26:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:23 smithi165 ceph-mon[21975]: pgmap v116: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-20T11:26:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:23 smithi078 ceph-mon[21464]: pgmap v116: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-20T11:26:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:23 smithi044 ceph-mon[16855]: pgmap v116: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-20T11:26:25.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:25 smithi165 ceph-mon[21975]: pgmap v117: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-20T11:26:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:25 smithi078 ceph-mon[21464]: pgmap v117: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-20T11:26:25.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:25 smithi044 ceph-mon[16855]: pgmap v117: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-20T11:26:27.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:27 smithi165 ceph-mon[21975]: pgmap v118: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-20T11:26:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:27 smithi078 ceph-mon[21464]: pgmap v118: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-20T11:26:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:27 smithi044 ceph-mon[16855]: pgmap v118: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-20T11:26:29.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:29 smithi165 ceph-mon[21975]: pgmap v119: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-04-20T11:26:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:29 smithi078 ceph-mon[21464]: pgmap v119: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-04-20T11:26:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:29 smithi044 ceph-mon[16855]: pgmap v119: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-04-20T11:26:31.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:31 smithi078 ceph-mon[21464]: pgmap v120: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T11:26:31.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:31 smithi044 ceph-mon[16855]: pgmap v120: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T11:26:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:31 smithi165 ceph-mon[21975]: pgmap v120: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T11:26:33.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:33 smithi078 ceph-mon[21464]: pgmap v121: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-20T11:26:33.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:33 smithi044 ceph-mon[16855]: pgmap v121: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-20T11:26:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:33 smithi165 ceph-mon[21975]: pgmap v121: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-20T11:26:35.748 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:35 smithi044 ceph-mon[16855]: pgmap v122: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-20T11:26:35.782 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:35 smithi165 ceph-mon[21975]: pgmap v122: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-20T11:26:35.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:35 smithi078 ceph-mon[21464]: pgmap v122: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-20T11:26:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:37 smithi078 ceph-mon[21464]: pgmap v123: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-20T11:26:37.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:37 smithi044 ceph-mon[16855]: pgmap v123: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-20T11:26:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:37 smithi165 ceph-mon[21975]: pgmap v123: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-20T11:26:38.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:26:38.930 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 -- 172.21.15.44:0/3118796060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1043b0 msgr2=0x7fcc1c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 --2- 172.21.15.44:0/3118796060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1043b0 0x7fcc1c104790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009900 tx=0x7fcc0c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 -- 172.21.15.44:0/3118796060 shutdown_connections 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 --2- 172.21.15.44:0/3118796060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc1c109480 0x7fcc1c111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 --2- 172.21.15.44:0/3118796060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc1c104d60 0x7fcc1c108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 --2- 172.21.15.44:0/3118796060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1043b0 0x7fcc1c104790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 -- 172.21.15.44:0/3118796060 >> 172.21.15.44:0/3118796060 conn(0x7fcc1c0ffbd0 msgr2=0x7fcc1c101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 -- 172.21.15.44:0/3118796060 shutdown_connections 2024-04-20T11:26:39.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.160+0000 7fcc20a00640 1 -- 172.21.15.44:0/3118796060 wait complete. 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.161+0000 7fcc20a00640 1 Processor -- start 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.161+0000 7fcc20a00640 1 -- start start 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc20a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc1c1043b0 0x7fcc1c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc20a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 0x7fcc1c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc20a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc1c19a9c0 0x7fcc1c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc20a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c114620 con 0x7fcc1c109480 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc20a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1144a0 con 0x7fcc1c1043b0 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc20a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1147a0 con 0x7fcc1c19a9c0 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 0x7fcc1c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 0x7fcc1c1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35740/0 (socket says 172.21.15.44:35740) 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 -- 172.21.15.44:0/1095397510 learned_addr learned my addr 172.21.15.44:0/1095397510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc1a800640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc1c19a9c0 0x7fcc1c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19e00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc1c1043b0 0x7fcc1c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 -- 172.21.15.44:0/1095397510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc1c19a9c0 msgr2=0x7fcc1c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc1c19a9c0 0x7fcc1c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 -- 172.21.15.44:0/1095397510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc1c1043b0 msgr2=0x7fcc1c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc1c1043b0 0x7fcc1c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.162+0000 7fcc19400640 1 -- 172.21.15.44:0/1095397510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c19b4e0 con 0x7fcc1c109480 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.163+0000 7fcc19400640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 0x7fcc1c1a0d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e9f0 tx=0x7fcc0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.163+0000 7fcc12a00640 1 -- 172.21.15.44:0/1095397510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0800ccc0 con 0x7fcc1c109480 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.163+0000 7fcc12a00640 1 -- 172.21.15.44:0/1095397510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc08005520 con 0x7fcc1c109480 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.163+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c19b7d0 con 0x7fcc1c109480 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.163+0000 7fcc12a00640 1 -- 172.21.15.44:0/1095397510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08010690 con 0x7fcc1c109480 2024-04-20T11:26:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.163+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1a79d0 con 0x7fcc1c109480 2024-04-20T11:26:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 Processor -- start 2024-04-20T11:26:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc12a00640 1 -- 172.21.15.44:0/1095397510 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcc08004f70 con 0x7fcc1c109480 2024-04-20T11:26:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 -- start start 2024-04-20T11:26:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f99b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55616f901150 con 0x55616f99b1a0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55616f8a7370 con 0x55616f99bac0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.167+0000 7fcc2382b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55616f8fd110 con 0x55616f9a46a0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f99b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f99b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35748/0 (socket says 172.21.15.44:35748) 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 -- 172.21.15.44:0/1486976507 learned_addr learned my addr 172.21.15.44:0/1486976507 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc19e00640 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc19400640 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 -- 172.21.15.44:0/1486976507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 msgr2=0x55616f9a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 -- 172.21.15.44:0/1486976507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 msgr2=0x55616f9a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 -- 172.21.15.44:0/1486976507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616f6dd7d0 con 0x55616f99b1a0 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc19e00640 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.168+0000 7fcc1a800640 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f99b580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009f60 tx=0x7fcc14009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd272b37b8ad6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.169+0000 7fcc12a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc000a3d20 0x7fcc000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.169+0000 7fcc07000640 1 -- 172.21.15.44:0/1486976507 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1401e070 con 0x55616f99b1a0 2024-04-20T11:26:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.169+0000 7fcc07000640 1 -- 172.21.15.44:0/1486976507 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc14003c30 con 0x55616f99b1a0 2024-04-20T11:26:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.169+0000 7fcc12a00640 1 -- 172.21.15.44:0/1095397510 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc080d0b00 con 0x7fcc1c109480 2024-04-20T11:26:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.169+0000 7fcc19e00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc000a3d20 0x7fcc000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.169+0000 7fcc07000640 1 -- 172.21.15.44:0/1486976507 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400fa10 con 0x55616f99b1a0 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 -- 172.21.15.44:0/1486976507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 msgr2=0x55616f99b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f99b580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009f60 tx=0x7fcc14009f90 comp rx=0 tx=0).stop 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 -- 172.21.15.44:0/1486976507 shutdown_connections 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 --2- 172.21.15.44:0/1486976507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f99b580 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 -- 172.21.15.44:0/1486976507 >> 172.21.15.44:0/1486976507 conn(0x55616f9958d0 msgr2=0x55616f995cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc19e00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc000a3d20 0x7fcc000a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c19bb00 tx=0x7fcc0c005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 -- 172.21.15.44:0/1486976507 shutdown_connections 2024-04-20T11:26:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 -- 172.21.15.44:0/1486976507 wait complete. 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.170+0000 7fcc2382b740 1 -- start start 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 Processor -- start 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55616f9a74e0 con 0x55616f99b1a0 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55616f8ba740 con 0x55616f99bac0 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55616f9a7660 con 0x55616f9a46a0 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc2382b740 10 client.0 _ll_register_callbacks cb 0x55616f8b5b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.171+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35764/0 (socket says 172.21.15.44:35764) 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 -- 172.21.15.44:0/3009406527 learned_addr learned my addr 172.21.15.44:0/3009406527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19400640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc1a800640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 -- 172.21.15.44:0/3009406527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 msgr2=0x55616f9a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 -- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 msgr2=0x55616f9a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 -- 172.21.15.44:0/3009406527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616f6133f0 con 0x55616f99b1a0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcc19e00640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a1790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c035000 tx=0x7fcc0c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bf1ec700027959 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.172+0000 7fcbfb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcbfb400640 1 -- 172.21.15.44:0/3009406527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c005600 con 0x55616f99b1a0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcbfb400640 1 -- 172.21.15.44:0/3009406527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc0c0046c0 con 0x55616f99b1a0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc14003c30 con 0x55616f99b1a0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcbfb400640 1 -- 172.21.15.44:0/3009406527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcc0c059050 con 0x55616f99b1a0 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcc2382b740 10 client.16281 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55616f5d30f0 con 0x55616f99b1a0 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcbfb400640 1 -- 172.21.15.44:0/3009406527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc14003c30 con 0x55616f99b1a0 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.173+0000 7fcbfb400640 1 -- 172.21.15.44:0/3009406527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fcc0c003b90 con 0x55616f99b1a0 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc2382b740 10 client.16281 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc2382b740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc2382b740 20 client.16281 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616f9b4bd0 0x55616f9b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc2382b740 4 client.16281 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55616f90eb70 con 0x55616f9b4bd0 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.174+0000 7fcc19400640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616f9b4bd0 0x55616f9b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.175+0000 7fcc19400640 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616f9b4bd0 0x55616f9b7050 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a4c5bc2ad21928 server_cookie=8cc74616aa7a21f3 in_seq=0 out_seq=0 2024-04-20T11:26:39.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.175+0000 7fcbfb400640 10 client.16281 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.175+0000 7fcbfb400640 1 -- 172.21.15.44:0/3009406527 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55616f90eb70 con 0x55616f9b4bd0 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.175+0000 7fcbfb400640 10 client.16281 handle_command_reply: tid=0 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.175+0000 7fcc2382b740 1 client.16281 shutdown 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 msgr2=0x55616f9a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a1790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c035000 tx=0x7fcc0c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 shutdown_connections 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616f9b4bd0 0x55616f9b7050 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3009406527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a1790 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcbfb400640 0 client.16281 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 >> 172.21.15.44:0/3009406527 conn(0x55616f9958d0 msgr2=0x55616f997620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 shutdown_connections 2024-04-20T11:26:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 1 -- 172.21.15.44:0/3009406527 wait complete. 2024-04-20T11:26:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.176+0000 7fcc2382b740 20 client.16281 trim_cache size 0 max 16384 2024-04-20T11:26:39.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.195+0000 7fcc2382b740 1 Processor -- start 2024-04-20T11:26:39.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.195+0000 7fcc2382b740 1 -- start start 2024-04-20T11:26:39.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc2382b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1afed0 con 0x55616f99b1a0 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc2382b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1b1bf0 con 0x55616f99bac0 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc2382b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1b05a0 con 0x55616f9a46a0 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35780/0 (socket says 172.21.15.44:35780) 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 -- 172.21.15.44:0/869362339 learned_addr learned my addr 172.21.15.44:0/869362339 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19400640 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 -- 172.21.15.44:0/869362339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 msgr2=0x55616f9b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 -- 172.21.15.44:0/869362339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 msgr2=0x55616f9a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.196+0000 7fcc19e00640 1 -- 172.21.15.44:0/869362339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616f9a6e00 con 0x55616f99b1a0 2024-04-20T11:26:39.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.197+0000 7fcc19e00640 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a3ec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c036040 tx=0x7fcc0c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86498c0573eb9f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.197+0000 7fcc06600640 1 -- 172.21.15.44:0/869362339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c061020 con 0x55616f99b1a0 2024-04-20T11:26:39.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.197+0000 7fcc06600640 1 -- 172.21.15.44:0/869362339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc0c050070 con 0x55616f99b1a0 2024-04-20T11:26:39.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.197+0000 7fcc06600640 1 -- 172.21.15.44:0/869362339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c003b90 con 0x55616f99b1a0 2024-04-20T11:26:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 -- 172.21.15.44:0/869362339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 msgr2=0x55616f9a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a3ec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c036040 tx=0x7fcc0c004610 comp rx=0 tx=0).stop 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 -- 172.21.15.44:0/869362339 shutdown_connections 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f9a46a0 0x55616f9b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99bac0 0x55616f9a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 --2- 172.21.15.44:0/869362339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f99b1a0 0x55616f9a3ec0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 -- 172.21.15.44:0/869362339 >> 172.21.15.44:0/869362339 conn(0x55616f9958d0 msgr2=0x55616f994030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 -- 172.21.15.44:0/869362339 shutdown_connections 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 -- 172.21.15.44:0/869362339 wait complete. 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 -- start start 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.198+0000 7fcc2382b740 1 Processor -- start 2024-04-20T11:26:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 0x55616f9acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f99bac0 0x55616f99e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f9a46a0 0x55616f99e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1b1460 con 0x55616f9a46a0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1b0d00 con 0x55616f99b1a0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1ade90 con 0x55616f99bac0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc2382b740 10 client.0 _ll_register_callbacks cb 0x7fcc0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 0x55616f9acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc1a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f9a46a0 0x55616f99e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 0x55616f9acbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60484/0 (socket says 172.21.15.44:60484) 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc19e00640 1 -- 172.21.15.44:0/3716630137 learned_addr learned my addr 172.21.15.44:0/3716630137 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.199+0000 7fcc19400640 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f99bac0 0x55616f99e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc19e00640 1 -- 172.21.15.44:0/3716630137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f99bac0 msgr2=0x55616f99e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc19e00640 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f99bac0 0x55616f99e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc19e00640 1 -- 172.21.15.44:0/3716630137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f9a46a0 msgr2=0x55616f99e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc19e00640 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f9a46a0 0x55616f99e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc19e00640 1 -- 172.21.15.44:0/3716630137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616f99ed50 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc19e00640 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 0x55616f9acbf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005600 tx=0x7fcc0c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781bf6118701d901 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcbfb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcbfb400640 1 -- 172.21.15.44:0/3716630137 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c061040 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcbfb400640 1 -- 172.21.15.44:0/3716630137 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcc0c0046c0 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.200+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0c050070 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcbfb400640 1 -- 172.21.15.44:0/3716630137 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcc0c0048c0 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcc2382b740 10 client.26417 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55616fa1b160 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcbfb400640 1 -- 172.21.15.44:0/3716630137 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc0c050070 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcbfb400640 1 -- 172.21.15.44:0/3716630137 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fcc0c06e020 con 0x55616f99b1a0 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcc2382b740 10 client.26417 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcc2382b740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcc2382b740 20 client.26417 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.201+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616fa1b7e0 0x55616fa1dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.202+0000 7fcc2382b740 4 client.26417 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.202+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55616f9089e0 con 0x55616fa1b7e0 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.202+0000 7fcc19400640 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616fa1b7e0 0x55616fa1dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.202+0000 7fcc19400640 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616fa1b7e0 0x55616fa1dc00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a93c3cf6beb7e0c server_cookie=8e48ba8a587d2907 in_seq=0 out_seq=0 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.202+0000 7fcbfb400640 10 client.26417 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcbfb400640 1 -- 172.21.15.44:0/3716630137 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55616f9089e0 con 0x55616fa1b7e0 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcbfb400640 10 client.26417 handle_command_reply: tid=0 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 client.26417 shutdown 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 msgr2=0x55616f9acbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 0x55616f9acbf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005600 tx=0x7fcc0c0035f0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 shutdown_connections 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55616fa1b7e0 0x55616fa1dc00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55616f9a46a0 0x55616f99e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55616f99bac0 0x55616f99e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 --2- 172.21.15.44:0/3716630137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55616f99b1a0 0x55616f9acbf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcbfb400640 0 client.26417 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 >> 172.21.15.44:0/3716630137 conn(0x55616f9958d0 msgr2=0x55616f9a1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 shutdown_connections 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 1 -- 172.21.15.44:0/3716630137 wait complete. 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.203+0000 7fcc2382b740 20 client.26417 trim_cache size 0 max 16384 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:39.207 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:26:39.208 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:26:39.208 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:39.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc000a3d20 msgr2=0x7fcc000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc000a3d20 0x7fcc000a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c19bb00 tx=0x7fcc0c005fd0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 msgr2=0x7fcc1c1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 0x7fcc1c1a0d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e9f0 tx=0x7fcc0800edf0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 shutdown_connections 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc1c19a9c0 0x7fcc1c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcc000a3d20 0x7fcc000a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c109480 0x7fcc1c1a0d50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 --2- 172.21.15.44:0/1095397510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc1c1043b0 0x7fcc1c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 >> 172.21.15.44:0/1095397510 conn(0x7fcc1c0ffbd0 msgr2=0x7fcc1c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 shutdown_connections 2024-04-20T11:26:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.204+0000 7fcc20a00640 1 -- 172.21.15.44:0/1095397510 wait complete. 2024-04-20T11:26:39.221 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:26:39.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 -- 172.21.15.44:0/1999908872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c108da0 msgr2=0x7f7c9c115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/1999908872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c108da0 0x7f7c9c115210 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009ca0 tx=0x7f7c9402c180 comp rx=0 tx=0).stop 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 -- 172.21.15.44:0/1999908872 shutdown_connections 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/1999908872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c108da0 0x7f7c9c115210 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/1999908872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108400 0x7f7c9c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/1999908872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c10e470 0x7f7c9c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 -- 172.21.15.44:0/1999908872 >> 172.21.15.44:0/1999908872 conn(0x7f7c9c077ae0 msgr2=0x7f7c9c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 -- 172.21.15.44:0/1999908872 shutdown_connections 2024-04-20T11:26:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.427+0000 7f7ca0a00640 1 -- 172.21.15.44:0/1999908872 wait complete. 2024-04-20T11:26:39.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 Processor -- start 2024-04-20T11:26:39.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 -- start start 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 0x7f7c9c1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 0x7f7c9c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c10e470 0x7f7c9c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c1188b0 con 0x7f7c9c108da0 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c118730 con 0x7f7c9c10e470 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.428+0000 7f7ca0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c118a30 con 0x7f7c9c108400 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c99e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 0x7f7c9c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 0x7f7c9c1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 0x7f7c9c1a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53760/0 (socket says 172.21.15.44:53760) 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c99e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 0x7f7c9c1a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35812/0 (socket says 172.21.15.44:35812) 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 -- 172.21.15.44:0/3186554730 learned_addr learned my addr 172.21.15.44:0/3186554730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9b200640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c10e470 0x7f7c9c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 -- 172.21.15.44:0/3186554730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c10e470 msgr2=0x7f7c9c19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c10e470 0x7f7c9c19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 -- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 msgr2=0x7f7c9c1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 0x7f7c9c1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c9a800640 1 -- 172.21.15.44:0/3186554730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c19f210 con 0x7f7c9c108400 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.429+0000 7f7c99e00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 0x7f7c9c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.430+0000 7f7c9a800640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 0x7f7c9c1a48e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00e9f0 tx=0x7f7c8c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.430+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00cd80 con 0x7f7c9c108400 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.430+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c19f500 con 0x7f7c9c108400 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.430+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7c8c005520 con 0x7f7c9c108400 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.430+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c010690 con 0x7f7c9c108400 2024-04-20T11:26:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.430+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c1ab7a0 con 0x7f7c9c108400 2024-04-20T11:26:39.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.431+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c0690f0 con 0x7f7c9c108400 2024-04-20T11:26:39.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.433+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7c8c0046c0 con 0x7f7c9c108400 2024-04-20T11:26:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.435+0000 7f7c93400640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7c800a3da0 0x7f7c800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.435+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7c8c0d0b40 con 0x7f7c9c108400 2024-04-20T11:26:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.435+0000 7f7c99e00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7c800a3da0 0x7f7c800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.436+0000 7f7c99e00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7c800a3da0 0x7f7c800a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c10da20 tx=0x7f7c88005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.437+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7c8c09d1f0 con 0x7f7c9c108400 2024-04-20T11:26:39.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.684+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c9c19ffc0 con 0x7f7c9c108400 2024-04-20T11:26:39.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.686+0000 7f7c93400640 1 -- 172.21.15.44:0/3186554730 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11970 (secure 0 0 0) 0x7f7c8c09caf0 con 0x7f7c9c108400 2024-04-20T11:26:39.685 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:26:39.685 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:26:00.778182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:26:39.686 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 39 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7c800a3da0 msgr2=0x7f7c800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7c800a3da0 0x7f7c800a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c10da20 tx=0x7f7c88005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 msgr2=0x7f7c9c1a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 0x7f7c9c1a48e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00e9f0 tx=0x7f7c8c00edf0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 shutdown_connections 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c9c10e470 0x7f7c9c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7c800a3da0 0x7f7c800a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c9c108da0 0x7f7c9c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 --2- 172.21.15.44:0/3186554730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c9c108400 0x7f7c9c1a48e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 >> 172.21.15.44:0/3186554730 conn(0x7f7c9c077ae0 msgr2=0x7f7c9c079020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.688+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 shutdown_connections 2024-04-20T11:26:39.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.689+0000 7f7ca0a00640 1 -- 172.21.15.44:0/3186554730 wait complete. 2024-04-20T11:26:39.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:39 smithi044 ceph-mon[16855]: pgmap v124: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-20T11:26:39.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:26:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 -- 172.21.15.44:0/2095219409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 msgr2=0x7f2f3410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 --2- 172.21.15.44:0/2095219409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 0x7f2f3410e880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009900 tx=0x7f2f2c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 -- 172.21.15.44:0/2095219409 shutdown_connections 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 --2- 172.21.15.44:0/2095219409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 0x7f2f3410e880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 --2- 172.21.15.44:0/2095219409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f3410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 --2- 172.21.15.44:0/2095219409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 0x7f2f341033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 -- 172.21.15.44:0/2095219409 >> 172.21.15.44:0/2095219409 conn(0x7f2f340fc7c0 msgr2=0x7f2f340febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 -- 172.21.15.44:0/2095219409 shutdown_connections 2024-04-20T11:26:39.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.926+0000 7f2f39800640 1 -- 172.21.15.44:0/2095219409 wait complete. 2024-04-20T11:26:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 Processor -- start 2024-04-20T11:26:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 -- start start 2024-04-20T11:26:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 0x7f2f341a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f341a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 0x7f2f3419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341122b0 con 0x7f2f341038e0 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34112130 con 0x7f2f3410c430 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.928+0000 7f2f39800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34112430 con 0x7f2f34102fc0 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f341a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 0x7f2f341a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 0x7f2f341a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53784/0 (socket says 172.21.15.44:53784) 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f341a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35830/0 (socket says 172.21.15.44:35830) 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33e00640 1 -- 172.21.15.44:0/2732999945 learned_addr learned my addr 172.21.15.44:0/2732999945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f38e00640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 0x7f2f3419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 -- 172.21.15.44:0/2732999945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 msgr2=0x7f2f341a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 0x7f2f341a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 -- 172.21.15.44:0/2732999945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 msgr2=0x7f2f3419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 0x7f2f3419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 -- 172.21.15.44:0/2732999945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3419b030 con 0x7f2f341038e0 2024-04-20T11:26:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.929+0000 7f2f33400640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f341a0bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e9c0 tx=0x7f2f1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.930+0000 7f2f30c00640 1 -- 172.21.15.44:0/2732999945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00ccc0 con 0x7f2f341038e0 2024-04-20T11:26:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.930+0000 7f2f30c00640 1 -- 172.21.15.44:0/2732999945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2f1c0035f0 con 0x7f2f341038e0 2024-04-20T11:26:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.930+0000 7f2f30c00640 1 -- 172.21.15.44:0/2732999945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00ce60 con 0x7f2f341038e0 2024-04-20T11:26:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.930+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3419b320 con 0x7f2f341038e0 2024-04-20T11:26:39.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.930+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f341a7630 con 0x7f2f341038e0 2024-04-20T11:26:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.933+0000 7f2f3dbe5740 1 Processor -- start 2024-04-20T11:26:39.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 -- start start 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 0x562d5a44d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44da90 0x562d5a456130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a458a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d5a3b2050 con 0x562d5a44d170 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d5a358490 con 0x562d5a44da90 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562d5a3ae010 con 0x562d5a456670 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f30c00640 1 -- 172.21.15.44:0/2732999945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2f1c0037a0 con 0x7f2f341038e0 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f38e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 0x562d5a44d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f38e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 0x562d5a44d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35836/0 (socket says 172.21.15.44:35836) 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f38e00640 1 -- 172.21.15.44:0/3914631916 learned_addr learned my addr 172.21.15.44:0/3914631916 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.934+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44da90 0x562d5a456130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f33400640 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a458a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f38e00640 1 -- 172.21.15.44:0/3914631916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 msgr2=0x562d5a458a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f38e00640 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a458a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f38e00640 1 -- 172.21.15.44:0/3914631916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44da90 msgr2=0x562d5a456130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f38e00640 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44da90 0x562d5a456130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f38e00640 1 -- 172.21.15.44:0/3914631916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5a360070 con 0x562d5a44d170 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44da90 0x562d5a456130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:39.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.935+0000 7f2f38e00640 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 0x562d5a44d550 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009230 tx=0x7f2f2c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262834bc9a9c5e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.936+0000 7f2f29600640 1 -- 172.21.15.44:0/3914631916 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c02c650 con 0x562d5a44d170 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.936+0000 7f2f29600640 1 -- 172.21.15.44:0/3914631916 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2f2c02c7f0 con 0x562d5a44d170 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.936+0000 7f2f29600640 1 -- 172.21.15.44:0/3914631916 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c005500 con 0x562d5a44d170 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.936+0000 7f2f30c00640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2f180a3d20 0x7f2f180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.936+0000 7f2f30c00640 1 -- 172.21.15.44:0/2732999945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2f1c0d0bb0 con 0x7f2f341038e0 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.936+0000 7f2f33e00640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2f180a3d20 0x7f2f180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3914631916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 msgr2=0x562d5a44d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 0x562d5a44d550 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009230 tx=0x7f2f2c009560 comp rx=0 tx=0).stop 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3914631916 shutdown_connections 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a458a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44da90 0x562d5a456130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3914631916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44d170 0x562d5a44d550 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3914631916 >> 172.21.15.44:0/3914631916 conn(0x562d5a4478a0 msgr2=0x562d5a447c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3914631916 shutdown_connections 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3914631916 wait complete. 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f33e00640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2f180a3d20 0x7f2f180a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562d5a44cc40 tx=0x7f2f24009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.937+0000 7f2f3dbe5740 1 -- start start 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 Processor -- start 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a4536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a44da90 0x562d5a453c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a456670 0x562d5a454170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d5a459400 con 0x562d5a456670 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d5a36b860 con 0x562d5a44d170 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562d5a459580 con 0x562d5a44da90 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f3dbe5740 10 client.0 _ll_register_callbacks cb 0x562d5a339fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a4536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f38e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a456670 0x562d5a454170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a4536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60554/0 (socket says 172.21.15.44:60554) 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.938+0000 7f2f33e00640 1 -- 172.21.15.44:0/3721248917 learned_addr learned my addr 172.21.15.44:0/3721248917 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33400640 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a44da90 0x562d5a453c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33e00640 1 -- 172.21.15.44:0/3721248917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a44da90 msgr2=0x562d5a453c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a44da90 0x562d5a453c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33e00640 1 -- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a456670 msgr2=0x562d5a454170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a456670 0x562d5a454170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33e00640 1 -- 172.21.15.44:0/3721248917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5a36f950 con 0x562d5a44d170 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a4536f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400fc60 tx=0x7f2f24004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1200004bb4270c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f22000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f22000640 1 -- 172.21.15.44:0/3721248917 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24004b20 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2c02c7f0 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.939+0000 7f2f22000640 1 -- 172.21.15.44:0/3721248917 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2f24018030 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f22000640 1 -- 172.21.15.44:0/3721248917 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2f24004cc0 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 10 client.26435 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562d5a33d820 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f22000640 1 -- 172.21.15.44:0/3721248917 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2f2c02c7f0 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f22000640 1 -- 172.21.15.44:0/3721248917 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f2f24031030 con 0x562d5a44d170 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 10 client.26435 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 10 client.26435 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 20 client.26435 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a466780 0x562d5a468c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 4 client.26435 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.940+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d5a3bfa70 con 0x562d5a466780 2024-04-20T11:26:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.941+0000 7f2f33400640 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a466780 0x562d5a468c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.944 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:39 smithi078 ceph-mon[21464]: pgmap v124: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-20T11:26:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.941+0000 7f2f33400640 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a466780 0x562d5a468c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43d31ae5a1ca8302 server_cookie=805f195f457b41ae in_seq=0 out_seq=0 2024-04-20T11:26:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.941+0000 7f2f22000640 10 client.26435 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f22000640 1 -- 172.21.15.44:0/3721248917 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d5a3bfa70 con 0x562d5a466780 2024-04-20T11:26:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f22000640 10 client.26435 handle_command_reply: tid=0 2024-04-20T11:26:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 client.26435 shutdown 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 msgr2=0x562d5a4536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a4536f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400fc60 tx=0x7f2f24004ab0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 shutdown_connections 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a456670 0x562d5a454170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a44da90 0x562d5a453c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a466780 0x562d5a468c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3721248917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a4536f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f22000640 0 client.26435 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.942+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 >> 172.21.15.44:0/3721248917 conn(0x562d5a4478a0 msgr2=0x562d5a4495f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.943+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 shutdown_connections 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.943+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3721248917 wait complete. 2024-04-20T11:26:39.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.943+0000 7f2f3dbe5740 20 client.26435 trim_cache size 0 max 16384 2024-04-20T11:26:39.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 Processor -- start 2024-04-20T11:26:39.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 -- start start 2024-04-20T11:26:39.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a455e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a46e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341afc00 con 0x562d5a44da90 2024-04-20T11:26:39.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341b1920 con 0x562d5a44d170 2024-04-20T11:26:39.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341b02d0 con 0x562d5a456670 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f33400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f33400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35844/0 (socket says 172.21.15.44:35844) 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a455e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.958+0000 7f2f33400640 1 -- 172.21.15.44:0/3502304237 learned_addr learned my addr 172.21.15.44:0/3502304237 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f38e00640 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a46e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33e00640 1 -- 172.21.15.44:0/3502304237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 msgr2=0x562d5a46e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a46e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33e00640 1 -- 172.21.15.44:0/3502304237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 msgr2=0x562d5a4622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33e00640 1 -- 172.21.15.44:0/3502304237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5a44d8c0 con 0x562d5a44d170 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33400640 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.959+0000 7f2f33e00640 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a455e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2f24029d70 tx=0x7f2f240093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8457ba15279675 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f28c00640 1 -- 172.21.15.44:0/3502304237 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2401c3a0 con 0x562d5a44d170 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f28c00640 1 -- 172.21.15.44:0/3502304237 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2f240298f0 con 0x562d5a44d170 2024-04-20T11:26:39.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f28c00640 1 -- 172.21.15.44:0/3502304237 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2401c670 con 0x562d5a44d170 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3502304237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 msgr2=0x562d5a455e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a455e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2f24029d70 tx=0x7f2f240093a0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3502304237 shutdown_connections 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a46e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/3502304237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a455e20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3502304237 >> 172.21.15.44:0/3502304237 conn(0x562d5a4478a0 msgr2=0x562d5a449560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.960+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3502304237 shutdown_connections 2024-04-20T11:26:39.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.961+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/3502304237 wait complete. 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.961+0000 7f2f3dbe5740 1 -- start start 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.961+0000 7f2f3dbe5740 1 Processor -- start 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.961+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a450470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f3dbe5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a450ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341b1190 con 0x562d5a44da90 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341b0a30 con 0x562d5a44d170 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f3dbe5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341adb40 con 0x562d5a456670 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f3dbe5740 10 client.0 _ll_register_callbacks cb 0x7f2f2400faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a450470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a450470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60584/0 (socket says 172.21.15.44:60584) 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 -- 172.21.15.44:0/2864163060 learned_addr learned my addr 172.21.15.44:0/2864163060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f38e00640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a450ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 -- 172.21.15.44:0/2864163060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 msgr2=0x562d5a450ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a450ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 -- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 msgr2=0x562d5a4509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33e00640 1 -- 172.21.15.44:0/2864163060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5a0c43f0 con 0x562d5a44d170 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.962+0000 7f2f33400640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f33e00640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a450470 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2f24029cf0 tx=0x7f2f24005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87544fd3436221e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:39.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f22000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:39.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f22000640 1 -- 172.21.15.44:0/2864163060 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2401c3a0 con 0x562d5a44d170 2024-04-20T11:26:39.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f240298f0 con 0x562d5a44d170 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f22000640 1 -- 172.21.15.44:0/2864163060 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f2f240293f0 con 0x562d5a44d170 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f22000640 1 -- 172.21.15.44:0/2864163060 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2f24029630 con 0x562d5a44d170 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f3dbe5740 10 client.26441 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.963+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562d5a18e7d0 con 0x562d5a44d170 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f22000640 1 -- 172.21.15.44:0/2864163060 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2f240298f0 con 0x562d5a44d170 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f22000640 1 -- 172.21.15.44:0/2864163060 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f2f2404e050 con 0x562d5a44d170 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f3dbe5740 10 client.26441 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f3dbe5740 10 client.26441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f3dbe5740 20 client.26441 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a4cd2f0 0x562d5a4cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f3dbe5740 4 client.26441 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d5a3b98e0 con 0x562d5a4cd2f0 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.964+0000 7f2f33400640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a4cd2f0 0x562d5a4cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f33400640 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a4cd2f0 0x562d5a4cf710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16b936e3d8a0e2de server_cookie=6c13bad55650f952 in_seq=0 out_seq=0 2024-04-20T11:26:39.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f22000640 10 client.26441 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f22000640 1 -- 172.21.15.44:0/2864163060 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d5a3b98e0 con 0x562d5a4cd2f0 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f22000640 10 client.26441 handle_command_reply: tid=0 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 client.26441 shutdown 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 msgr2=0x562d5a450470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a450470 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2f24029cf0 tx=0x7f2f24005bd0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 shutdown_connections 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d5a456670 0x562d5a450ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d5a44da90 0x562d5a4509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x562d5a4cd2f0 0x562d5a4cf710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 --2- 172.21.15.44:0/2864163060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d5a44d170 0x562d5a450470 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f22000640 0 client.26441 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.965+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 >> 172.21.15.44:0/2864163060 conn(0x562d5a4478a0 msgr2=0x562d5a453690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 shutdown_connections 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f3dbe5740 1 -- 172.21.15.44:0/2864163060 wait complete. 2024-04-20T11:26:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f3dbe5740 20 client.26441 trim_cache size 0 max 16384 2024-04-20T11:26:39.969 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:39.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2f180a3d20 msgr2=0x7f2f180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f39800640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2f180a3d20 0x7f2f180a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562d5a44cc40 tx=0x7f2f24009500 comp rx=0 tx=0).stop 2024-04-20T11:26:39.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 msgr2=0x7f2f341a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.966+0000 7f2f39800640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f341a0bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e9c0 tx=0x7f2f1c00ee80 comp rx=0 tx=0).stop 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 shutdown_connections 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3410c430 0x7f2f3419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2f180a3d20 0x7f2f180a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f341038e0 0x7f2f341a0bf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 --2- 172.21.15.44:0/2732999945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f34102fc0 0x7f2f341a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 >> 172.21.15.44:0/2732999945 conn(0x7f2f340fc7c0 msgr2=0x7f2f34100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 shutdown_connections 2024-04-20T11:26:39.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:39.967+0000 7f2f39800640 1 -- 172.21.15.44:0/2732999945 wait complete. 2024-04-20T11:26:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:39 smithi165 ceph-mon[21975]: pgmap v124: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-20T11:26:39.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:26:40.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 -- 172.21.15.44:0/2972930761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108f50 msgr2=0x7f8c501153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 --2- 172.21.15.44:0/2972930761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108f50 0x7f8c501153c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009c10 tx=0x7f8c4802bef0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 -- 172.21.15.44:0/2972930761 shutdown_connections 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 --2- 172.21.15.44:0/2972930761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108f50 0x7f8c501153c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 --2- 172.21.15.44:0/2972930761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108590 0x7f8c50108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 --2- 172.21.15.44:0/2972930761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 0x7f8c5010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.190+0000 7f8c55000640 1 -- 172.21.15.44:0/2972930761 >> 172.21.15.44:0/2972930761 conn(0x7f8c500764a0 msgr2=0x7f8c500788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.191+0000 7f8c55000640 1 -- 172.21.15.44:0/2972930761 shutdown_connections 2024-04-20T11:26:40.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.191+0000 7f8c55000640 1 -- 172.21.15.44:0/2972930761 wait complete. 2024-04-20T11:26:40.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 Processor -- start 2024-04-20T11:26:40.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 -- start start 2024-04-20T11:26:40.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108590 0x7f8c5019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 0x7f8c5019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 0x7f8c501a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50118a20 con 0x7f8c5010e600 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501188a0 con 0x7f8c50108f50 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c55000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50118ba0 con 0x7f8c50108590 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108590 0x7f8c5019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 0x7f8c5019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.192+0000 7f8c4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 0x7f8c501a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 0x7f8c5019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60594/0 (socket says 172.21.15.44:60594) 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4d400640 1 -- 172.21.15.44:0/356470271 learned_addr learned my addr 172.21.15.44:0/356470271 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4e800640 1 -- 172.21.15.44:0/356470271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108590 msgr2=0x7f8c5019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4e800640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108590 0x7f8c5019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4e800640 1 -- 172.21.15.44:0/356470271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 msgr2=0x7f8c5019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4e800640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 0x7f8c5019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4e800640 1 -- 172.21.15.44:0/356470271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c501a3e40 con 0x7f8c5010e600 2024-04-20T11:26:40.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4d400640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 0x7f8c5019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.193+0000 7f8c4e800640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 0x7f8c501a3710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802c430 tx=0x7f8c48033720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.194+0000 7f8c46a00640 1 -- 172.21.15.44:0/356470271 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48043020 con 0x7f8c5010e600 2024-04-20T11:26:40.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.194+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c501a40d0 con 0x7f8c5010e600 2024-04-20T11:26:40.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.194+0000 7f8c46a00640 1 -- 172.21.15.44:0/356470271 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c4802ca60 con 0x7f8c5010e600 2024-04-20T11:26:40.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.194+0000 7f8c46a00640 1 -- 172.21.15.44:0/356470271 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4803e6b0 con 0x7f8c5010e600 2024-04-20T11:26:40.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.194+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501ab9c0 con 0x7f8c5010e600 2024-04-20T11:26:40.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.197+0000 7f8c57d93740 1 Processor -- start 2024-04-20T11:26:40.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.197+0000 7f8c46a00640 1 -- 172.21.15.44:0/356470271 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8c48046020 con 0x7f8c5010e600 2024-04-20T11:26:40.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 -- start start 2024-04-20T11:26:40.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9a9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55762f90e050 con 0x55762f9b24b0 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55762f8b4490 con 0x55762f9a98d0 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c57d93740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55762f90a010 con 0x55762f9a8fb0 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.198+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9a9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53852/0 (socket says 172.21.15.44:53852) 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 -- 172.21.15.44:0/3913134762 learned_addr learned my addr 172.21.15.44:0/3913134762 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4de00640 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 -- 172.21.15.44:0/3913134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 msgr2=0x55762f9b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 -- 172.21.15.44:0/3913134762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 msgr2=0x55762f9b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 -- 172.21.15.44:0/3913134762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762f8bc070 con 0x55762f9a8fb0 2024-04-20T11:26:40.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c4d400640 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9a9390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c38010e20 tx=0x7f8c38005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6fedaa2cf8f6504 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c3f400640 1 -- 172.21.15.44:0/3913134762 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38016070 con 0x55762f9a8fb0 2024-04-20T11:26:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c3f400640 1 -- 172.21.15.44:0/3913134762 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c3800fe50 con 0x55762f9a8fb0 2024-04-20T11:26:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.199+0000 7f8c3f400640 1 -- 172.21.15.44:0/3913134762 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38019040 con 0x55762f9a8fb0 2024-04-20T11:26:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c46a00640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c340a3d20 0x7f8c340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c46a00640 1 -- 172.21.15.44:0/356470271 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8c480f2c30 con 0x7f8c5010e600 2024-04-20T11:26:40.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c4de00640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c340a3d20 0x7f8c340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 -- 172.21.15.44:0/3913134762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 msgr2=0x55762f9a9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9a9390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c38010e20 tx=0x7f8c38005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 -- 172.21.15.44:0/3913134762 shutdown_connections 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3913134762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9a9390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.200+0000 7f8c57d93740 1 -- 172.21.15.44:0/3913134762 >> 172.21.15.44:0/3913134762 conn(0x55762f9a36e0 msgr2=0x55762f9a3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.201+0000 7f8c57d93740 1 -- 172.21.15.44:0/3913134762 shutdown_connections 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.201+0000 7f8c57d93740 1 -- 172.21.15.44:0/3913134762 wait complete. 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.201+0000 7f8c4de00640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c340a3d20 0x7f8c340a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55762f9a8a80 tx=0x7f8c40009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.201+0000 7f8c57d93740 1 -- start start 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.201+0000 7f8c57d93740 1 Processor -- start 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 0x55762f9af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a98d0 0x55762f9afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55762f9b52a0 con 0x55762f9b24b0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55762f8c7860 con 0x55762f9a8fb0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55762f9b5420 con 0x55762f9a98d0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c57d93740 10 client.0 _ll_register_callbacks cb 0x55762f895fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 0x55762f9af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 0x55762f9af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60620/0 (socket says 172.21.15.44:60620) 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c4de00640 1 -- 172.21.15.44:0/1168036785 learned_addr learned my addr 172.21.15.44:0/1168036785 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c4e800640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.202+0000 7f8c4d400640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a98d0 0x55762f9afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c4de00640 1 -- 172.21.15.44:0/1168036785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a98d0 msgr2=0x55762f9afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c4de00640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a98d0 0x55762f9afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c4de00640 1 -- 172.21.15.44:0/1168036785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 msgr2=0x55762f9afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c4de00640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c4de00640 1 -- 172.21.15.44:0/1168036785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762f8cb950 con 0x55762f9a8fb0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c4de00640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 0x55762f9af570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8c4001e9d0 tx=0x7f8c40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e554840b8a03cb32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c33e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c33e00640 1 -- 172.21.15.44:0/1168036785 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004020 con 0x55762f9a8fb0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3800fe50 con 0x55762f9a8fb0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.203+0000 7f8c33e00640 1 -- 172.21.15.44:0/1168036785 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8c40018030 con 0x55762f9a8fb0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.204+0000 7f8c33e00640 1 -- 172.21.15.44:0/1168036785 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c400041c0 con 0x55762f9a8fb0 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.204+0000 7f8c57d93740 10 client.26453 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:40.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.204+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55762f899820 con 0x55762f9a8fb0 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.204+0000 7f8c33e00640 1 -- 172.21.15.44:0/1168036785 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c3800fe50 con 0x55762f9a8fb0 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.204+0000 7f8c33e00640 1 -- 172.21.15.44:0/1168036785 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f8c40031030 con 0x55762f9a8fb0 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c57d93740 10 client.26453 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c57d93740 10 client.26453 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c57d93740 20 client.26453 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c57d93740 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762f9c25b0 0x55762f9c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c57d93740 4 client.26453 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55762f91ba70 con 0x55762f9c25b0 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.205+0000 7f8c4d400640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762f9c25b0 0x55762f9c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.206+0000 7f8c4d400640 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762f9c25b0 0x55762f9c4a70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9489500d33db299 server_cookie=773d643662089dcf in_seq=0 out_seq=0 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.206+0000 7f8c33e00640 10 client.26453 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.206+0000 7f8c33e00640 1 -- 172.21.15.44:0/1168036785 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55762f91ba70 con 0x55762f9c25b0 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.206+0000 7f8c33e00640 10 client.26453 handle_command_reply: tid=0 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.206+0000 7f8c57d93740 1 client.26453 shutdown 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 msgr2=0x55762f9af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 0x55762f9af570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8c4001e9d0 tx=0x7f8c40004730 comp rx=0 tx=0).stop 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 shutdown_connections 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762f9c25b0 0x55762f9c4a70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762f9afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a98d0 0x55762f9afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c33e00640 0 client.26453 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 --2- 172.21.15.44:0/1168036785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a8fb0 0x55762f9af570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 >> 172.21.15.44:0/1168036785 conn(0x55762f9a36e0 msgr2=0x55762f9a5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 shutdown_connections 2024-04-20T11:26:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 1 -- 172.21.15.44:0/1168036785 wait complete. 2024-04-20T11:26:40.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.207+0000 7f8c57d93740 20 client.26453 trim_cache size 0 max 16384 2024-04-20T11:26:40.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 Processor -- start 2024-04-20T11:26:40.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 -- start start 2024-04-20T11:26:40.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9a98d0 0x55762f9be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9b24b0 0x55762f9a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b4250 con 0x55762f9a98d0 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b5f70 con 0x55762f9b24b0 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.222+0000 7f8c57d93740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b4920 con 0x55762f9a8fb0 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9a98d0 0x55762f9be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53868/0 (socket says 172.21.15.44:53868) 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 -- 172.21.15.44:0/4244154617 learned_addr learned my addr 172.21.15.44:0/4244154617 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4e800640 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9b24b0 0x55762f9a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 -- 172.21.15.44:0/4244154617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9b24b0 msgr2=0x55762f9a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9b24b0 0x55762f9a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 -- 172.21.15.44:0/4244154617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9a98d0 msgr2=0x55762f9be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9a98d0 0x55762f9be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.223+0000 7f8c4de00640 1 -- 172.21.15.44:0/4244154617 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762f6203f0 con 0x55762f9a8fb0 2024-04-20T11:26:40.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.224+0000 7f8c4de00640 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b1ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c40029700 tx=0x7f8c400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891a6c1826f5df1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.224+0000 7f8c3ea00640 1 -- 172.21.15.44:0/4244154617 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004360 con 0x55762f9a8fb0 2024-04-20T11:26:40.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.224+0000 7f8c3ea00640 1 -- 172.21.15.44:0/4244154617 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c40005520 con 0x55762f9a8fb0 2024-04-20T11:26:40.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.224+0000 7f8c3ea00640 1 -- 172.21.15.44:0/4244154617 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004c30 con 0x55762f9a8fb0 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 -- 172.21.15.44:0/4244154617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 msgr2=0x55762f9b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b1ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c40029700 tx=0x7f8c400046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 -- 172.21.15.44:0/4244154617 shutdown_connections 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9b24b0 0x55762f9a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9a98d0 0x55762f9be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 --2- 172.21.15.44:0/4244154617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b1ca0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 -- 172.21.15.44:0/4244154617 >> 172.21.15.44:0/4244154617 conn(0x55762f9a36e0 msgr2=0x55762f9a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 -- 172.21.15.44:0/4244154617 shutdown_connections 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 -- 172.21.15.44:0/4244154617 wait complete. 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.225+0000 7f8c57d93740 1 -- start start 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 Processor -- start 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762fa27bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b57e0 con 0x55762f9b24b0 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b5080 con 0x55762f9a98d0 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b2190 con 0x55762f9a8fb0 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c57d93740 10 client.0 _ll_register_callbacks cb 0x7f8c4001e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762fa27bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c4de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c4d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60634/0 (socket says 172.21.15.44:60634) 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.226+0000 7f8c4d400640 1 -- 172.21.15.44:0/3247348125 learned_addr learned my addr 172.21.15.44:0/3247348125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4d400640 1 -- 172.21.15.44:0/3247348125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 msgr2=0x55762f9b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4d400640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4d400640 1 -- 172.21.15.44:0/3247348125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 msgr2=0x55762fa27bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4d400640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762fa27bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4d400640 1 -- 172.21.15.44:0/3247348125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762f6ea7d0 con 0x55762f9a98d0 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4de00640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4e800640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762fa27bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c4d400640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b69e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8c38004b70 tx=0x7f8c38009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4be10b92182dac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c33e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c33e00640 1 -- 172.21.15.44:0/3247348125 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38016040 con 0x55762f9a98d0 2024-04-20T11:26:40.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c33e00640 1 -- 172.21.15.44:0/3247348125 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8c3800fc20 con 0x55762f9a98d0 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.227+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c40005520 con 0x55762f9a98d0 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c33e00640 1 -- 172.21.15.44:0/3247348125 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c3800fe40 con 0x55762f9a98d0 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 10 client.26459 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55762fa290c0 con 0x55762f9a98d0 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c33e00640 1 -- 172.21.15.44:0/3247348125 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8c40005520 con 0x55762f9a98d0 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c33e00640 1 -- 172.21.15.44:0/3247348125 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f8c3803f020 con 0x55762f9a98d0 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 10 client.26459 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 10 client.26459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 20 client.26459 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762fa29530 0x55762fa2b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 4 client.26459 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.228+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55762f9158e0 con 0x55762fa29530 2024-04-20T11:26:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c4de00640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762fa29530 0x55762fa2b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c4de00640 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762fa29530 0x55762fa2b950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=84b0cf61553ce8b0 server_cookie=7138664316a394bf in_seq=0 out_seq=0 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c33e00640 10 client.26459 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c33e00640 1 -- 172.21.15.44:0/3247348125 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55762f9158e0 con 0x55762fa29530 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c33e00640 10 client.26459 handle_command_reply: tid=0 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c57d93740 1 client.26459 shutdown 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.229+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 msgr2=0x55762f9b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b69e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8c38004b70 tx=0x7f8c38009530 comp rx=0 tx=0).stop 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 shutdown_connections 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55762fa29530 0x55762fa2b950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55762f9b24b0 0x55762fa27bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55762f9a98d0 0x55762f9b69e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 --2- 172.21.15.44:0/3247348125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55762f9a8fb0 0x55762f9b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c33e00640 0 client.26459 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 >> 172.21.15.44:0/3247348125 conn(0x55762f9a36e0 msgr2=0x55762f9af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 shutdown_connections 2024-04-20T11:26:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 1 -- 172.21.15.44:0/3247348125 wait complete. 2024-04-20T11:26:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c57d93740 20 client.26459 trim_cache size 0 max 16384 2024-04-20T11:26:40.234 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c340a3d20 msgr2=0x7f8c340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c55000640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c340a3d20 0x7f8c340a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55762f9a8a80 tx=0x7f8c40009260 comp rx=0 tx=0).stop 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 msgr2=0x7f8c501a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.230+0000 7f8c55000640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 0x7f8c501a3710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802c430 tx=0x7f8c48033720 comp rx=0 tx=0).stop 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 shutdown_connections 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8c340a3d20 0x7f8c340a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c5010e600 0x7f8c501a3710 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c50108f50 0x7f8c5019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 --2- 172.21.15.44:0/356470271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c50108590 0x7f8c5019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 >> 172.21.15.44:0/356470271 conn(0x7f8c500764a0 msgr2=0x7f8c50077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 shutdown_connections 2024-04-20T11:26:40.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.231+0000 7f8c55000640 1 -- 172.21.15.44:0/356470271 wait complete. 2024-04-20T11:26:40.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:26:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.448+0000 7fbd86800640 1 -- 172.21.15.44:0/3516162416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 msgr2=0x7fbd801047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.448+0000 7fbd86800640 1 --2- 172.21.15.44:0/3516162416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801047f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ca0 tx=0x7fbd7002c100 comp rx=0 tx=0).stop 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 -- 172.21.15.44:0/3516162416 shutdown_connections 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 --2- 172.21.15.44:0/3516162416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd801094e0 0x7fbd801110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 --2- 172.21.15.44:0/3516162416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd80104dc0 0x7fbd80108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 --2- 172.21.15.44:0/3516162416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801047f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 -- 172.21.15.44:0/3516162416 >> 172.21.15.44:0/3516162416 conn(0x7fbd800ffcf0 msgr2=0x7fbd80102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 -- 172.21.15.44:0/3516162416 shutdown_connections 2024-04-20T11:26:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.449+0000 7fbd86800640 1 -- 172.21.15.44:0/3516162416 wait complete. 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 Processor -- start 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 -- start start 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd801094e0 0x7fbd801a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 0x7fbd8019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801146b0 con 0x7fbd801094e0 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80114530 con 0x7fbd80104410 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.450+0000 7fbd86800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80114830 con 0x7fbd8019a9f0 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 0x7fbd8019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 0x7fbd8019aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53890/0 (socket says 172.21.15.44:53890) 2024-04-20T11:26:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7fe00640 1 -- 172.21.15.44:0/4219656108 learned_addr learned my addr 172.21.15.44:0/4219656108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7f400640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd76a00640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd801094e0 0x7fbd801a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7f400640 1 -- 172.21.15.44:0/4219656108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 msgr2=0x7fbd8019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7f400640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 0x7fbd8019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7f400640 1 -- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd801094e0 msgr2=0x7fbd801a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7f400640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd801094e0 0x7fbd801a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7f400640 1 -- 172.21.15.44:0/4219656108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8019b510 con 0x7fbd80104410 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.451+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 0x7fbd8019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.452+0000 7fbd7f400640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801a0840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002c640 tx=0x7fbd70034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.452+0000 7fbd7cc00640 1 -- 172.21.15.44:0/4219656108 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70039040 con 0x7fbd80104410 2024-04-20T11:26:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.452+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8019b7a0 con 0x7fbd80104410 2024-04-20T11:26:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.452+0000 7fbd7cc00640 1 -- 172.21.15.44:0/4219656108 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd70044070 con 0x7fbd80104410 2024-04-20T11:26:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.452+0000 7fbd7cc00640 1 -- 172.21.15.44:0/4219656108 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700043a0 con 0x7fbd80104410 2024-04-20T11:26:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.452+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801a7990 con 0x7fbd80104410 2024-04-20T11:26:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.455+0000 7fbd89617740 1 Processor -- start 2024-04-20T11:26:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.455+0000 7fbd89617740 1 -- start start 2024-04-20T11:26:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.455+0000 7fbd7cc00640 1 -- 172.21.15.44:0/4219656108 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbd70032840 con 0x7fbd80104410 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd89617740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd89617740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfbaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd89617740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfbb4b0 0x55e31dfbd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd89617740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e31df17050 con 0x55e31dfbb4b0 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd89617740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e31debd490 con 0x55e31dfb1fb0 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd89617740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e31df13010 con 0x55e31dfb28d0 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfbb4b0 0x55e31dfbd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60678/0 (socket says 172.21.15.44:60678) 2024-04-20T11:26:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd7fe00640 1 -- 172.21.15.44:0/3034922482 learned_addr learned my addr 172.21.15.44:0/3034922482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd7f400640 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfbaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd76a00640 1 -- 172.21.15.44:0/3034922482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 msgr2=0x55e31dfbaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd76a00640 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfbaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.456+0000 7fbd76a00640 1 -- 172.21.15.44:0/3034922482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 msgr2=0x55e31dfb2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd76a00640 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd76a00640 1 -- 172.21.15.44:0/3034922482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31dec5070 con 0x55e31dfbb4b0 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd76a00640 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfbb4b0 0x55e31dfbd8a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd8019c1e0 tx=0x7fbd680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6b944e61fe104 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd7cc00640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd640a3d20 0x7fbd640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd74c00640 1 -- 172.21.15.44:0/3034922482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801c480 con 0x55e31dfbb4b0 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd74c00640 1 -- 172.21.15.44:0/3034922482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd6801cb20 con 0x55e31dfbb4b0 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd74c00640 1 -- 172.21.15.44:0/3034922482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011c50 con 0x55e31dfbb4b0 2024-04-20T11:26:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.457+0000 7fbd7cc00640 1 -- 172.21.15.44:0/4219656108 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbd700f2b90 con 0x7fbd80104410 2024-04-20T11:26:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd640a3d20 0x7fbd640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 -- 172.21.15.44:0/3034922482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfbb4b0 msgr2=0x55e31dfbd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfbb4b0 0x55e31dfbd8a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd8019c1e0 tx=0x7fbd680107a0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 -- 172.21.15.44:0/3034922482 shutdown_connections 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfbb4b0 0x55e31dfbd8a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfbaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 --2- 172.21.15.44:0/3034922482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 -- 172.21.15.44:0/3034922482 >> 172.21.15.44:0/3034922482 conn(0x55e31dfac6e0 msgr2=0x55e31dfacac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 -- 172.21.15.44:0/3034922482 shutdown_connections 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd640a3d20 0x7fbd640a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd78012940 tx=0x7fbd78005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.458+0000 7fbd89617740 1 -- 172.21.15.44:0/3034922482 wait complete. 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.459+0000 7fbd89617740 1 -- start start 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.459+0000 7fbd89617740 1 Processor -- start 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.459+0000 7fbd89617740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.459+0000 7fbd89617740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfb8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.459+0000 7fbd89617740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfb8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.459+0000 7fbd89617740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e31dfbe2a0 con 0x55e31dfb1fb0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd89617740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e31ded0860 con 0x55e31dfbb4b0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd89617740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e31dfbe420 con 0x55e31dfb28d0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd89617740 10 client.0 _ll_register_callbacks cb 0x55e31de9efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfb8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35926/0 (socket says 172.21.15.44:35926) 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7f400640 1 -- 172.21.15.44:0/676302726 learned_addr learned my addr 172.21.15.44:0/676302726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd76a00640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfb8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7fe00640 1 -- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 msgr2=0x55e31dfb8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfb8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7fe00640 1 -- 172.21.15.44:0/676302726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 msgr2=0x55e31dfb8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7fe00640 1 -- 172.21.15.44:0/676302726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31ded4950 con 0x55e31dfbb4b0 2024-04-20T11:26:40.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.460+0000 7fbd7f400640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfb8fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800fed0 tx=0x7fbd78004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf67637e7c09a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd6d600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd6d600640 1 -- 172.21.15.44:0/676302726 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78009d20 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6801cb20 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd6d600640 1 -- 172.21.15.44:0/676302726 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbd78018030 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd6d600640 1 -- 172.21.15.44:0/676302726 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbd7801e920 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd89617740 10 client.26471 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.461+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e31dea2820 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd6d600640 1 -- 172.21.15.44:0/676302726 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd6801cb20 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd6d600640 1 -- 172.21.15.44:0/676302726 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fbd78031030 con 0x55e31dfbb4b0 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd89617740 10 client.26471 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd89617740 10 client.26471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd89617740 20 client.26471 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd89617740 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31dfcb5f0 0x55e31dfcda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd89617740 4 client.26471 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e31df24a70 con 0x55e31dfcb5f0 2024-04-20T11:26:40.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.462+0000 7fbd7f400640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31dfcb5f0 0x55e31dfcda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.463+0000 7fbd7f400640 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31dfcb5f0 0x55e31dfcda50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=193923108775efcb server_cookie=a7113b6b9c65e822 in_seq=0 out_seq=0 2024-04-20T11:26:40.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.463+0000 7fbd6d600640 10 client.26471 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd6d600640 1 -- 172.21.15.44:0/676302726 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e31df24a70 con 0x55e31dfcb5f0 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd6d600640 10 client.26471 handle_command_reply: tid=0 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 client.26471 shutdown 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 msgr2=0x55e31dfb8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfb8fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800fed0 tx=0x7fbd78004730 comp rx=0 tx=0).stop 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 shutdown_connections 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31dfcb5f0 0x55e31dfcda50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfb8fa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb28d0 0x55e31dfb8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 --2- 172.21.15.44:0/676302726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 >> 172.21.15.44:0/676302726 conn(0x55e31dfac6e0 msgr2=0x55e31dfae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd6d600640 0 client.26471 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:40.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 shutdown_connections 2024-04-20T11:26:40.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 1 -- 172.21.15.44:0/676302726 wait complete. 2024-04-20T11:26:40.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.464+0000 7fbd89617740 20 client.26471 trim_cache size 0 max 16384 2024-04-20T11:26:40.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.479+0000 7fbd89617740 1 Processor -- start 2024-04-20T11:26:40.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.479+0000 7fbd89617740 1 -- start start 2024-04-20T11:26:40.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd89617740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb1fb0 0x55e31dfbac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd89617740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 0x55e31dfc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd89617740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd89617740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801afea0 con 0x55e31dfb28d0 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd89617740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b1bc0 con 0x55e31dfbb4b0 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd89617740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b0570 con 0x55e31dfb1fb0 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 0x55e31dfc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 0x55e31dfc7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35938/0 (socket says 172.21.15.44:35938) 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 -- 172.21.15.44:0/1962356194 learned_addr learned my addr 172.21.15.44:0/1962356194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd7f400640 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb1fb0 0x55e31dfbac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 -- 172.21.15.44:0/1962356194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb1fb0 msgr2=0x55e31dfbac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb1fb0 0x55e31dfbac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 -- 172.21.15.44:0/1962356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 msgr2=0x55e31dfab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.480+0000 7fbd76a00640 1 -- 172.21.15.44:0/1962356194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31dc293f0 con 0x55e31dfb28d0 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.481+0000 7fbd76a00640 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 0x55e31dfc7160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55e31dfad870 tx=0x7fbd68010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c674629aecb2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.481+0000 7fbd6fe00640 1 -- 172.21.15.44:0/1962356194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6802b070 con 0x55e31dfb28d0 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.481+0000 7fbd6fe00640 1 -- 172.21.15.44:0/1962356194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd68011de0 con 0x55e31dfb28d0 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.481+0000 7fbd6fe00640 1 -- 172.21.15.44:0/1962356194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801c480 con 0x55e31dfb28d0 2024-04-20T11:26:40.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 -- 172.21.15.44:0/1962356194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 msgr2=0x55e31dfc7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 0x55e31dfc7160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55e31dfad870 tx=0x7fbd68010f40 comp rx=0 tx=0).stop 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 -- 172.21.15.44:0/1962356194 shutdown_connections 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfbb4b0 0x55e31dfab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb28d0 0x55e31dfc7160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 --2- 172.21.15.44:0/1962356194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfb1fb0 0x55e31dfbac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 -- 172.21.15.44:0/1962356194 >> 172.21.15.44:0/1962356194 conn(0x55e31dfac6e0 msgr2=0x55e31dfaaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 -- 172.21.15.44:0/1962356194 shutdown_connections 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 -- 172.21.15.44:0/1962356194 wait complete. 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 -- start start 2024-04-20T11:26:40.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.482+0000 7fbd89617740 1 Processor -- start 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb28d0 0x55e31dfb5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfbb4b0 0x55e31e030b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b1430 con 0x55e31dfb1fb0 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b0cd0 con 0x55e31dfb28d0 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801ade00 con 0x55e31dfbb4b0 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd89617740 10 client.0 _ll_register_callbacks cb 0x7fbd7800fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35942/0 (socket says 172.21.15.44:35942) 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd7f400640 1 -- 172.21.15.44:0/1216910571 learned_addr learned my addr 172.21.15.44:0/1216910571 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd76a00640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb28d0 0x55e31dfb5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.483+0000 7fbd7fe00640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfbb4b0 0x55e31e030b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd7f400640 1 -- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfbb4b0 msgr2=0x55e31e030b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd7f400640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfbb4b0 0x55e31e030b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd7f400640 1 -- 172.21.15.44:0/1216910571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb28d0 msgr2=0x55e31dfb5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd7f400640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb28d0 0x55e31dfb5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd7f400640 1 -- 172.21.15.44:0/1216910571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31dcf37d0 con 0x55e31dfb1fb0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd7f400640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb5980 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd700bb0b0 tx=0x7fbd7002c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a99bc441c0126b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd6d600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd6d600640 1 -- 172.21.15.44:0/1216910571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700f7040 con 0x55e31dfb1fb0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd6d600640 1 -- 172.21.15.44:0/1216910571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbd700bcf50 con 0x55e31dfb1fb0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd68011de0 con 0x55e31dfb1fb0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd6d600640 1 -- 172.21.15.44:0/1216910571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbd700bec50 con 0x55e31dfb1fb0 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd89617740 10 client.16347 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:40.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.484+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e31e032290 con 0x55e31dfb1fb0 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd6d600640 1 -- 172.21.15.44:0/1216910571 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd68011de0 con 0x55e31dfb1fb0 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd6d600640 1 -- 172.21.15.44:0/1216910571 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fbd7003c030 con 0x55e31dfb1fb0 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd89617740 10 client.16347 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd89617740 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd89617740 20 client.16347 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd89617740 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31e032a50 0x55e31e034e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd89617740 4 client.16347 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e31df1e8e0 con 0x55e31e032a50 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.485+0000 7fbd76a00640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31e032a50 0x55e31e034e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd76a00640 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31e032a50 0x55e31e034e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c9c0a2e1e511bf1 server_cookie=fc7b744da8375dab in_seq=0 out_seq=0 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd6d600640 10 client.16347 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd6d600640 1 -- 172.21.15.44:0/1216910571 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e31df1e8e0 con 0x55e31e032a50 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd6d600640 10 client.16347 handle_command_reply: tid=0 2024-04-20T11:26:40.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd89617740 1 client.16347 shutdown 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 msgr2=0x55e31dfb5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd89617740 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb5980 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd700bb0b0 tx=0x7fbd7002c100 comp rx=0 tx=0).stop 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 shutdown_connections 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd89617740 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55e31e032a50 0x55e31e034e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.486+0000 7fbd89617740 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e31dfbb4b0 0x55e31e030b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd89617740 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e31dfb28d0 0x55e31dfb5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd89617740 1 --2- 172.21.15.44:0/1216910571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e31dfb1fb0 0x55e31dfb5980 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd6d600640 0 client.16347 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 >> 172.21.15.44:0/1216910571 conn(0x55e31dfac6e0 msgr2=0x55e31dfab740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 shutdown_connections 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd89617740 1 -- 172.21.15.44:0/1216910571 wait complete. 2024-04-20T11:26:40.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd89617740 20 client.16347 trim_cache size 0 max 16384 2024-04-20T11:26:40.491 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:40.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd640a3d20 msgr2=0x7fbd640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd86800640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd640a3d20 0x7fbd640a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd78012940 tx=0x7fbd78005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 msgr2=0x7fbd801a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.487+0000 7fbd86800640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801a0840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002c640 tx=0x7fbd70034670 comp rx=0 tx=0).stop 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 shutdown_connections 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd8019a9f0 0x7fbd8019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd640a3d20 0x7fbd640a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd801094e0 0x7fbd801a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 --2- 172.21.15.44:0/4219656108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd80104410 0x7fbd801a0840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 >> 172.21.15.44:0/4219656108 conn(0x7fbd800ffcf0 msgr2=0x7fbd801010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 shutdown_connections 2024-04-20T11:26:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.488+0000 7fbd86800640 1 -- 172.21.15.44:0/4219656108 wait complete. 2024-04-20T11:26:40.507 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:26:40.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.723+0000 7ff533e00640 1 -- 172.21.15.44:0/2914166969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff534107140 msgr2=0x7ff53410ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.723+0000 7ff533e00640 1 --2- 172.21.15.44:0/2914166969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff534107140 0x7ff53410ed10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009ca0 tx=0x7ff52c02c100 comp rx=0 tx=0).stop 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 -- 172.21.15.44:0/2914166969 shutdown_connections 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 --2- 172.21.15.44:0/2914166969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff534107140 0x7ff53410ed10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 --2- 172.21.15.44:0/2914166969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102a20 0x7ff534106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 --2- 172.21.15.44:0/2914166969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534102070 0x7ff534102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 -- 172.21.15.44:0/2914166969 >> 172.21.15.44:0/2914166969 conn(0x7ff5340fd930 msgr2=0x7ff5340ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 -- 172.21.15.44:0/2914166969 shutdown_connections 2024-04-20T11:26:40.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.724+0000 7ff533e00640 1 -- 172.21.15.44:0/2914166969 wait complete. 2024-04-20T11:26:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 Processor -- start 2024-04-20T11:26:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 -- start start 2024-04-20T11:26:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102070 0x7ff5341a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 0x7ff5341a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff53419a7c0 0x7ff53419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff534112330 con 0x7ff534107140 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341121b0 con 0x7ff534102070 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.725+0000 7ff533e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341124b0 con 0x7ff53419a7c0 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 0x7ff5341a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff533400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff53419a7c0 0x7ff53419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 0x7ff5341a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35950/0 (socket says 172.21.15.44:35950) 2024-04-20T11:26:40.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff532a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102070 0x7ff5341a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff52be00640 1 -- 172.21.15.44:0/4215282546 learned_addr learned my addr 172.21.15.44:0/4215282546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff533400640 1 -- 172.21.15.44:0/4215282546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102070 msgr2=0x7ff5341a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff533400640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102070 0x7ff5341a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff533400640 1 -- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 msgr2=0x7ff5341a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff533400640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 0x7ff5341a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff533400640 1 -- 172.21.15.44:0/4215282546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53419b350 con 0x7ff53419a7c0 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff52be00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 0x7ff5341a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.726+0000 7ff532a00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102070 0x7ff5341a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.727+0000 7ff533400640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff53419a7c0 0x7ff53419ac70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009bf0 tx=0x7ff52c032f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.727+0000 7ff538a00640 1 -- 172.21.15.44:0/4215282546 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c039040 con 0x7ff53419a7c0 2024-04-20T11:26:40.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.727+0000 7ff538a00640 1 -- 172.21.15.44:0/4215282546 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff52c044070 con 0x7ff53419a7c0 2024-04-20T11:26:40.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.727+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53419b5e0 con 0x7ff53419a7c0 2024-04-20T11:26:40.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.727+0000 7ff538a00640 1 -- 172.21.15.44:0/4215282546 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0042a0 con 0x7ff53419a7c0 2024-04-20T11:26:40.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.727+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5341a7730 con 0x7ff53419a7c0 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.730+0000 7ff538a00640 1 -- 172.21.15.44:0/4215282546 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff52c003970 con 0x7ff53419a7c0 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.730+0000 7ff53cc12740 1 Processor -- start 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.730+0000 7ff53cc12740 1 -- start start 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d1f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1fa90 0x563b65d28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff53cc12740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563b65c84050 con 0x563b65d1f170 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff53cc12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b65c2a490 con 0x563b65d28670 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff53cc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b65c80010 con 0x563b65d1fa90 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d1f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d1f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35962/0 (socket says 172.21.15.44:35962) 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff52be00640 1 -- 172.21.15.44:0/4288766845 learned_addr learned my addr 172.21.15.44:0/4288766845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff532a00640 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1fa90 0x563b65d28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.731+0000 7ff52be00640 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff532a00640 1 -- 172.21.15.44:0/4288766845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 msgr2=0x563b65d2aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff532a00640 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d2aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff532a00640 1 -- 172.21.15.44:0/4288766845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 msgr2=0x563b65d1f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff532a00640 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d1f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff532a00640 1 -- 172.21.15.44:0/4288766845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b65c32070 con 0x563b65d1fa90 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff532a00640 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1fa90 0x563b65d28130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff524009fd0 tx=0x7ff5240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2540f7ed03ddff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff538a00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff5180a3d20 0x7ff5180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff523e00640 1 -- 172.21.15.44:0/4288766845 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524004030 con 0x563b65d1fa90 2024-04-20T11:26:40.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff523e00640 1 -- 172.21.15.44:0/4288766845 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff524005520 con 0x563b65d1fa90 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff523e00640 1 -- 172.21.15.44:0/4288766845 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524004da0 con 0x563b65d1fa90 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff538a00640 1 -- 172.21.15.44:0/4215282546 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff52c0f2b60 con 0x7ff53419a7c0 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.732+0000 7ff52be00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff5180a3d20 0x7ff5180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 -- 172.21.15.44:0/4288766845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1fa90 msgr2=0x563b65d28130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1fa90 0x563b65d28130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff524009fd0 tx=0x7ff5240118d0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 -- 172.21.15.44:0/4288766845 shutdown_connections 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff52be00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff5180a3d20 0x7ff5180a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x563b65d1aa30 tx=0x7ff51c005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1fa90 0x563b65d28130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 --2- 172.21.15.44:0/4288766845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d1f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 -- 172.21.15.44:0/4288766845 >> 172.21.15.44:0/4288766845 conn(0x563b65d198a0 msgr2=0x563b65d19c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 -- 172.21.15.44:0/4288766845 shutdown_connections 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.733+0000 7ff53cc12740 1 -- 172.21.15.44:0/4288766845 wait complete. 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 -- start start 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 Processor -- start 2024-04-20T11:26:40.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1f170 0x563b65d256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1fa90 0x563b65d25be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563b65d2b460 con 0x563b65d1fa90 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b65c3d860 con 0x563b65d28670 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.734+0000 7ff53cc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b65d2b5e0 con 0x563b65d1f170 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff53cc12740 10 client.0 _ll_register_callbacks cb 0x563b65c0bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1fa90 0x563b65d25be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff52be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1fa90 0x563b65d25be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35972/0 (socket says 172.21.15.44:35972) 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff532a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1f170 0x563b65d256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff52be00640 1 -- 172.21.15.44:0/3057893417 learned_addr learned my addr 172.21.15.44:0/3057893417 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff533400640 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff532a00640 1 -- 172.21.15.44:0/3057893417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 msgr2=0x563b65d26120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff532a00640 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d26120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff532a00640 1 -- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1fa90 msgr2=0x563b65d25be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff532a00640 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1fa90 0x563b65d25be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.735+0000 7ff532a00640 1 -- 172.21.15.44:0/3057893417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b65c41950 con 0x563b65d1f170 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff532a00640 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1f170 0x563b65d256a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff524011dd0 tx=0x7ff524005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2082df085e20c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff520c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff520c00640 1 -- 172.21.15.44:0/3057893417 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524029020 con 0x563b65d1f170 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff520c00640 1 -- 172.21.15.44:0/3057893417 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff52400f5a0 con 0x563b65d1f170 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff524005520 con 0x563b65d1f170 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff520c00640 1 -- 172.21.15.44:0/3057893417 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff524034050 con 0x563b65d1f170 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff53cc12740 10 client.26485 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:40.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.736+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563b65c0f820 con 0x563b65d1f170 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff520c00640 1 -- 172.21.15.44:0/3057893417 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff524005520 con 0x563b65d1f170 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff520c00640 1 -- 172.21.15.44:0/3057893417 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7ff524025470 con 0x563b65d1f170 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff53cc12740 10 client.26485 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff53cc12740 10 client.26485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff53cc12740 20 client.26485 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff53cc12740 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d386b0 0x563b65d3aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff53cc12740 4 client.26485 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b65c91a70 con 0x563b65d386b0 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.737+0000 7ff52be00640 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d386b0 0x563b65d3aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.738+0000 7ff520c00640 10 client.26485 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.738+0000 7ff52be00640 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d386b0 0x563b65d3aaa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdd9d683f9c0f1cc server_cookie=911d1622d4beaf83 in_seq=0 out_seq=0 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff520c00640 1 -- 172.21.15.44:0/3057893417 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b65c91a70 con 0x563b65d386b0 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff520c00640 10 client.26485 handle_command_reply: tid=0 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 client.26485 shutdown 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1f170 msgr2=0x563b65d256a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1f170 0x563b65d256a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff524011dd0 tx=0x7ff524005160 comp rx=0 tx=0).stop 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 shutdown_connections 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d386b0 0x563b65d3aaa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d28670 0x563b65d26120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1fa90 0x563b65d25be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 --2- 172.21.15.44:0/3057893417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d1f170 0x563b65d256a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff520c00640 0 client.26485 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 >> 172.21.15.44:0/3057893417 conn(0x563b65d198a0 msgr2=0x563b65d1b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 shutdown_connections 2024-04-20T11:26:40.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 1 -- 172.21.15.44:0/3057893417 wait complete. 2024-04-20T11:26:40.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.739+0000 7ff53cc12740 20 client.26485 trim_cache size 0 max 16384 2024-04-20T11:26:40.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.754+0000 7ff53cc12740 1 Processor -- start 2024-04-20T11:26:40.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.754+0000 7ff53cc12740 1 -- start start 2024-04-20T11:26:40.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d27dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d34320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff53cc12740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341afbb0 con 0x563b65d1f170 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff53cc12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341b18d0 con 0x563b65d1fa90 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff53cc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341b0280 con 0x563b65d28670 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff532a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d27dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff532a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d27dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35984/0 (socket says 172.21.15.44:35984) 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff532a00640 1 -- 172.21.15.44:0/185617192 learned_addr learned my addr 172.21.15.44:0/185617192 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff52be00640 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d34320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff533400640 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff52be00640 1 -- 172.21.15.44:0/185617192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 msgr2=0x563b65d402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff52be00640 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff52be00640 1 -- 172.21.15.44:0/185617192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 msgr2=0x563b65d27dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff52be00640 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d27dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff52be00640 1 -- 172.21.15.44:0/185617192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b659963f0 con 0x563b65d1fa90 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.755+0000 7ff532a00640 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d27dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.756+0000 7ff52be00640 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d34320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563b65d2b730 tx=0x7ff51c015070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8328f200ddae87b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.756+0000 7ff523400640 1 -- 172.21.15.44:0/185617192 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c005020 con 0x563b65d1fa90 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.756+0000 7ff523400640 1 -- 172.21.15.44:0/185617192 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff51c0047a0 con 0x563b65d1fa90 2024-04-20T11:26:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.756+0000 7ff523400640 1 -- 172.21.15.44:0/185617192 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c02e3b0 con 0x563b65d1fa90 2024-04-20T11:26:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 -- 172.21.15.44:0/185617192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 msgr2=0x563b65d34320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d34320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563b65d2b730 tx=0x7ff51c015070 comp rx=0 tx=0).stop 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 -- 172.21.15.44:0/185617192 shutdown_connections 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d34320 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 --2- 172.21.15.44:0/185617192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d27dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 -- 172.21.15.44:0/185617192 >> 172.21.15.44:0/185617192 conn(0x563b65d198a0 msgr2=0x563b65d20990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 -- 172.21.15.44:0/185617192 shutdown_connections 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 -- 172.21.15.44:0/185617192 wait complete. 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.757+0000 7ff53cc12740 1 -- start start 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 Processor -- start 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d2ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d2bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d9cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341b1140 con 0x563b65d1f170 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341b09e0 con 0x563b65d1fa90 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5341adaf0 con 0x563b65d28670 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff53cc12740 10 client.0 _ll_register_callbacks cb 0x7ff524009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff532a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d2ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff532a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d2ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36000/0 (socket says 172.21.15.44:36000) 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff532a00640 1 -- 172.21.15.44:0/1310059786 learned_addr learned my addr 172.21.15.44:0/1310059786 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff52be00640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d2bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.758+0000 7ff533400640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d9cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff532a00640 1 -- 172.21.15.44:0/1310059786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 msgr2=0x563b65d9cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff532a00640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d9cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff532a00640 1 -- 172.21.15.44:0/1310059786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 msgr2=0x563b65d2bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff532a00640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d2bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff532a00640 1 -- 172.21.15.44:0/1310059786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b65a607d0 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff532a00640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d2ba20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff524011dd0 tx=0x7ff524005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb315576fb5304 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff520c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff520c00640 1 -- 172.21.15.44:0/1310059786 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524029050 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff520c00640 1 -- 172.21.15.44:0/1310059786 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff524025cb0 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.759+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51c0047a0 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff520c00640 1 -- 172.21.15.44:0/1310059786 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff524046050 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 10 client.16365 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563b65d9e210 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff520c00640 1 -- 172.21.15.44:0/1310059786 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff51c0047a0 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff520c00640 1 -- 172.21.15.44:0/1310059786 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7ff5240035f0 con 0x563b65d1f170 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 10 client.16365 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:26:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 20 client.16365 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d9ea10 0x563b65da0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 4 client.16365 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.760+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b65c8b8e0 con 0x563b65d9ea10 2024-04-20T11:26:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.761+0000 7ff52be00640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d9ea10 0x563b65da0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.761+0000 7ff52be00640 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d9ea10 0x563b65da0e30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee4762ab68238b6 server_cookie=ffb530c32257fb71 in_seq=0 out_seq=0 2024-04-20T11:26:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.761+0000 7ff520c00640 10 client.16365 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff520c00640 1 -- 172.21.15.44:0/1310059786 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b65c8b8e0 con 0x563b65d9ea10 2024-04-20T11:26:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff520c00640 10 client.16365 handle_command_reply: tid=0 2024-04-20T11:26:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 client.16365 shutdown 2024-04-20T11:26:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 msgr2=0x563b65d2ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d2ba20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff524011dd0 tx=0x7ff524005b90 comp rx=0 tx=0).stop 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 shutdown_connections 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x563b65d9ea10 0x563b65da0e30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b65d28670 0x563b65d9cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b65d1fa90 0x563b65d2bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 --2- 172.21.15.44:0/1310059786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b65d1f170 0x563b65d2ba20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff520c00640 0 client.16365 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 >> 172.21.15.44:0/1310059786 conn(0x563b65d198a0 msgr2=0x563b65d180d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 shutdown_connections 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 1 -- 172.21.15.44:0/1310059786 wait complete. 2024-04-20T11:26:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.762+0000 7ff53cc12740 20 client.16365 trim_cache size 0 max 16384 2024-04-20T11:26:40.767 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff5180a3d20 msgr2=0x7ff5180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff5180a3d20 0x7ff5180a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x563b65d1aa30 tx=0x7ff51c005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff53419a7c0 msgr2=0x7ff53419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff53419a7c0 0x7ff53419ac70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009bf0 tx=0x7ff52c032f70 comp rx=0 tx=0).stop 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 shutdown_connections 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff53419a7c0 0x7ff53419ac70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff5180a3d20 0x7ff5180a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff534107140 0x7ff5341a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 --2- 172.21.15.44:0/4215282546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff534102070 0x7ff5341a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 >> 172.21.15.44:0/4215282546 conn(0x7ff5340fd930 msgr2=0x7ff53410d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 shutdown_connections 2024-04-20T11:26:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.763+0000 7ff533e00640 1 -- 172.21.15.44:0/4215282546 wait complete. 2024-04-20T11:26:40.778 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3186554730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:40.781 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:26:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3186554730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:40.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3186554730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:40.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 -- 172.21.15.44:0/3343182093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 msgr2=0x7f9b94102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3343182093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b94102460 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009900 tx=0x7f9b8402bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 -- 172.21.15.44:0/3343182093 shutdown_connections 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3343182093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3343182093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94102a30 0x7f9b94106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3343182093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b94102460 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 -- 172.21.15.44:0/3343182093 >> 172.21.15.44:0/3343182093 conn(0x7f9b940fd900 msgr2=0x7f9b940ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 -- 172.21.15.44:0/3343182093 shutdown_connections 2024-04-20T11:26:40.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.994+0000 7f9b99c00640 1 -- 172.21.15.44:0/3343182093 wait complete. 2024-04-20T11:26:40.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.995+0000 7f9b99c00640 1 Processor -- start 2024-04-20T11:26:40.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 -- start start 2024-04-20T11:26:40.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b9419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94198670 0x7f9b94198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941122d0 con 0x7f9b94102080 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94112150 con 0x7f9b94107150 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b99c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94112450 con 0x7f9b94198670 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.996+0000 7f9b93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b9419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b99200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94198670 0x7f9b94198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b9419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36014/0 (socket says 172.21.15.44:36014) 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b99200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94198670 0x7f9b94198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53974/0 (socket says 172.21.15.44:53974) 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60784/0 (socket says 172.21.15.44:60784) 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b93e00640 1 -- 172.21.15.44:0/3527835512 learned_addr learned my addr 172.21.15.44:0/3527835512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 -- 172.21.15.44:0/3527835512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94198670 msgr2=0x7f9b94198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94198670 0x7f9b94198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 -- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 msgr2=0x7f9b9419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:40.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b9419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:40.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 -- 172.21.15.44:0/3527835512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b94199190 con 0x7f9b94107150 2024-04-20T11:26:40.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.997+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9419e9b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00e9c0 tx=0x7f9b7c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:40.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.998+0000 7f9b91600640 1 -- 172.21.15.44:0/3527835512 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00ccc0 con 0x7f9b94107150 2024-04-20T11:26:40.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.998+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b94199480 con 0x7f9b94107150 2024-04-20T11:26:40.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.998+0000 7f9b91600640 1 -- 172.21.15.44:0/3527835512 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9b7c0035f0 con 0x7f9b94107150 2024-04-20T11:26:40.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.998+0000 7f9b91600640 1 -- 172.21.15.44:0/3527835512 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00ce60 con 0x7f9b94107150 2024-04-20T11:26:40.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:40.998+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941a5680 con 0x7f9b94107150 2024-04-20T11:26:41.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.001+0000 7f9b9de68740 1 Processor -- start 2024-04-20T11:26:41.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.001+0000 7f9b9de68740 1 -- start start 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.001+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 0x55bd0d017960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.001+0000 7f9b91600640 1 -- 172.21.15.44:0/3527835512 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9b7c0037a0 con 0x7f9b94107150 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d020540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d022e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b9de68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bd0cf78050 con 0x55bd0d017580 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b9de68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bd0cf1e490 con 0x55bd0d020a80 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b9de68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd0cf74010 con 0x55bd0d017ea0 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 0x55bd0d017960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 0x55bd0d017960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36022/0 (socket says 172.21.15.44:36022) 2024-04-20T11:26:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 -- 172.21.15.44:0/981929645 learned_addr learned my addr 172.21.15.44:0/981929645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b93e00640 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d020540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b8be00640 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d022e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 -- 172.21.15.44:0/981929645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 msgr2=0x55bd0d020540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d020540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 -- 172.21.15.44:0/981929645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 msgr2=0x55bd0d022e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d022e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.002+0000 7f9b99200640 1 -- 172.21.15.44:0/981929645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0cf26070 con 0x55bd0d017580 2024-04-20T11:26:41.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.003+0000 7f9b99200640 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 0x55bd0d017960 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009f60 tx=0x7f9b8c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef72c847f3ae8490 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.003+0000 7f9b89600640 1 -- 172.21.15.44:0/981929645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c01e070 con 0x55bd0d017580 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.003+0000 7f9b89600640 1 -- 172.21.15.44:0/981929645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9b8c003c30 con 0x55bd0d017580 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.003+0000 7f9b89600640 1 -- 172.21.15.44:0/981929645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00fa10 con 0x55bd0d017580 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b91600640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9b780a3d20 0x7f9b780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 -- 172.21.15.44:0/981929645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 msgr2=0x55bd0d017960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 0x55bd0d017960 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009f60 tx=0x7f9b8c009f90 comp rx=0 tx=0).stop 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b91600640 1 -- 172.21.15.44:0/3527835512 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9b7c0d0ab0 con 0x7f9b94107150 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 -- 172.21.15.44:0/981929645 shutdown_connections 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d022e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d020540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 --2- 172.21.15.44:0/981929645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017580 0x55bd0d017960 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 -- 172.21.15.44:0/981929645 >> 172.21.15.44:0/981929645 conn(0x55bd0d011cb0 msgr2=0x55bd0d012090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9b780a3d20 0x7f9b780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 -- 172.21.15.44:0/981929645 shutdown_connections 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.004+0000 7f9b9de68740 1 -- 172.21.15.44:0/981929645 wait complete. 2024-04-20T11:26:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.005+0000 7f9b9de68740 1 -- start start 2024-04-20T11:26:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.005+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9b780a3d20 0x7f9b780a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000c00 tx=0x7f9b84005a20 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.005+0000 7f9b9de68740 1 Processor -- start 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017580 0x55bd0d01db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d01e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d01e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bd0d023870 con 0x55bd0d017ea0 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bd0cf31860 con 0x55bd0d020a80 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd0d0239f0 con 0x55bd0d017580 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b9de68740 10 client.0 _ll_register_callbacks cb 0x55bd0cefffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d01e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d01e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36024/0 (socket says 172.21.15.44:36024) 2024-04-20T11:26:41.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 -- 172.21.15.44:0/3111703302 learned_addr learned my addr 172.21.15.44:0/3111703302 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017580 0x55bd0d01db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b99200640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d01e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 -- 172.21.15.44:0/3111703302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017580 msgr2=0x55bd0d01db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017580 0x55bd0d01db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 -- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 msgr2=0x55bd0d01e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d01e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b8be00640 1 -- 172.21.15.44:0/3111703302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0cf35950 con 0x55bd0d017ea0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.006+0000 7f9b99200640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d01e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d01e060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c01f000 tx=0x7f9b7c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade8d7d26bc1d9ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b82000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b82000640 1 -- 172.21.15.44:0/3111703302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c09c3c0 con 0x55bd0d017ea0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b82000640 1 -- 172.21.15.44:0/3111703302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9b7c09beb0 con 0x55bd0d017ea0 2024-04-20T11:26:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c003c30 con 0x55bd0d017ea0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b82000640 1 -- 172.21.15.44:0/3111703302 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c0bd0d0 con 0x55bd0d017ea0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b9de68740 10 client.16377 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bd0cf03820 con 0x55bd0d017ea0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.007+0000 7f9b82000640 1 -- 172.21.15.44:0/3111703302 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9b8c003c30 con 0x55bd0d017ea0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b82000640 1 -- 172.21.15.44:0/3111703302 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f9b7c004000 con 0x55bd0d017ea0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b9de68740 10 client.16377 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b9de68740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b9de68740 20 client.16377 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d030ac0 0x55bd0d032eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b9de68740 4 client.16377 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd0cf85a70 con 0x55bd0d030ac0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.008+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d030ac0 0x55bd0d032eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.009+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d030ac0 0x55bd0d032eb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0fbb23434f585e2 server_cookie=40024264a8a1e6ce in_seq=0 out_seq=0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.009+0000 7f9b82000640 10 client.16377 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b82000640 1 -- 172.21.15.44:0/3111703302 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd0cf85a70 con 0x55bd0d030ac0 2024-04-20T11:26:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b82000640 10 client.16377 handle_command_reply: tid=0 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 client.16377 shutdown 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 msgr2=0x55bd0d01e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d01e060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c01f000 tx=0x7f9b7c009580 comp rx=0 tx=0).stop 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 shutdown_connections 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d030ac0 0x55bd0d032eb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d020a80 0x55bd0d01e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d01e060 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3111703302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017580 0x55bd0d01db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b82000640 0 client.16377 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 >> 172.21.15.44:0/3111703302 conn(0x55bd0d011cb0 msgr2=0x55bd0d013a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 shutdown_connections 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 1 -- 172.21.15.44:0/3111703302 wait complete. 2024-04-20T11:26:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.010+0000 7f9b9de68740 20 client.16377 trim_cache size 0 max 16384 2024-04-20T11:26:41.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.025+0000 7f9b9de68740 1 Processor -- start 2024-04-20T11:26:41.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.025+0000 7f9b9de68740 1 -- start start 2024-04-20T11:26:41.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d020250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d02c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 0x55bd0d0137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b9de68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ab990 con 0x55bd0d020a80 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b9de68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ad6b0 con 0x55bd0d017580 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b9de68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ac060 con 0x55bd0d017ea0 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 0x55bd0d0137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 0x55bd0d0137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36036/0 (socket says 172.21.15.44:36036) 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 -- 172.21.15.44:0/1752868098 learned_addr learned my addr 172.21.15.44:0/1752868098 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b93e00640 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d020250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b8be00640 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d02c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 -- 172.21.15.44:0/1752868098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 msgr2=0x55bd0d02c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d02c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 -- 172.21.15.44:0/1752868098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 msgr2=0x55bd0d020250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d020250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b99200640 1 -- 172.21.15.44:0/1752868098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0d0231e0 con 0x55bd0d020a80 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.026+0000 7f9b93e00640 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d020250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.027+0000 7f9b99200640 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 0x55bd0d0137f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000f80 tx=0x7f9b8c009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707d6a013180e2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.027+0000 7f9b88c00640 1 -- 172.21.15.44:0/1752868098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c01e070 con 0x55bd0d020a80 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.027+0000 7f9b88c00640 1 -- 172.21.15.44:0/1752868098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9b8c01f040 con 0x55bd0d020a80 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.027+0000 7f9b88c00640 1 -- 172.21.15.44:0/1752868098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00f880 con 0x55bd0d020a80 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 -- 172.21.15.44:0/1752868098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 msgr2=0x55bd0d0137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 0x55bd0d0137f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000f80 tx=0x7f9b8c009af0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 -- 172.21.15.44:0/1752868098 shutdown_connections 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d020a80 0x55bd0d0137f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d017ea0 0x55bd0d02c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 --2- 172.21.15.44:0/1752868098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d020250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 -- 172.21.15.44:0/1752868098 >> 172.21.15.44:0/1752868098 conn(0x55bd0d011cb0 msgr2=0x55bd0d0130e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 -- 172.21.15.44:0/1752868098 shutdown_connections 2024-04-20T11:26:41.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 -- 172.21.15.44:0/1752868098 wait complete. 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.028+0000 7f9b9de68740 1 -- start start 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 Processor -- start 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d010f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d028c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d020a80 0x55bd0d0291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941acf20 con 0x55bd0d017ea0 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ac7c0 con 0x55bd0d017580 2024-04-20T11:26:41.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941a9930 con 0x55bd0d020a80 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b9de68740 10 client.0 _ll_register_callbacks cb 0x7f9b7c003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d028c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d028c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36040/0 (socket says 172.21.15.44:36040) 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b8be00640 1 -- 172.21.15.44:0/3099837749 learned_addr learned my addr 172.21.15.44:0/3099837749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.029+0000 7f9b99200640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d020a80 0x55bd0d0291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d010f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b8be00640 1 -- 172.21.15.44:0/3099837749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d020a80 msgr2=0x55bd0d0291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d020a80 0x55bd0d0291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b8be00640 1 -- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 msgr2=0x55bd0d010f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d010f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b8be00640 1 -- 172.21.15.44:0/3099837749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0cc8a3f0 con 0x55bd0d017ea0 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b99200640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d020a80 0x55bd0d0291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d010f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b8be00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d028c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c6a0 tx=0x7f9b7c0992e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694f37e57a824d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:41.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b82000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b82000640 1 -- 172.21.15.44:0/3099837749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c09c830 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b82000640 1 -- 172.21.15.44:0/3099837749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9b7c0987b0 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.030+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c01f040 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b82000640 1 -- 172.21.15.44:0/3099837749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c0bd0d0 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 10 client.16383 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bd0cd547d0 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b82000640 1 -- 172.21.15.44:0/3099837749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9b8c01f040 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b82000640 1 -- 172.21.15.44:0/3099837749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f9b7c0989b0 con 0x55bd0d017ea0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 10 client.16383 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 20 client.16383 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d09c0f0 0x55bd0d09e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 4 client.16383 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.031+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd0cf7f8e0 con 0x55bd0d09c0f0 2024-04-20T11:26:41.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d09c0f0 0x55bd0d09e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:41.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b93e00640 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d09c0f0 0x55bd0d09e510 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428adde9cbb3ecd9 server_cookie=b0558d6fc01df471 in_seq=0 out_seq=0 2024-04-20T11:26:41.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b82000640 10 client.16383 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b82000640 1 -- 172.21.15.44:0/3099837749 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd0cf7f8e0 con 0x55bd0d09c0f0 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b82000640 10 client.16383 handle_command_reply: tid=0 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 client.16383 shutdown 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 msgr2=0x55bd0d028c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d028c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c6a0 tx=0x7f9b7c0992e0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 shutdown_connections 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55bd0d09c0f0 0x55bd0d09e510 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd0d020a80 0x55bd0d0291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bd0d017ea0 0x55bd0d028c40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b9de68740 1 --2- 172.21.15.44:0/3099837749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bd0d017580 0x55bd0d010f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.032+0000 7f9b82000640 0 client.16383 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 >> 172.21.15.44:0/3099837749 conn(0x55bd0d011cb0 msgr2=0x55bd0d013970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 shutdown_connections 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b9de68740 1 -- 172.21.15.44:0/3099837749 wait complete. 2024-04-20T11:26:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b9de68740 20 client.16383 trim_cache size 0 max 16384 2024-04-20T11:26:41.036 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9b780a3d20 msgr2=0x7f9b780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9b780a3d20 0x7f9b780a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000c00 tx=0x7f9b84005a20 comp rx=0 tx=0).stop 2024-04-20T11:26:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 msgr2=0x7f9b9419e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:41.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.033+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9419e9b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00e9c0 tx=0x7f9b7c00edc0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 shutdown_connections 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b94198670 0x7f9b94198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b94107150 0x7f9b9419e9b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9b780a3d20 0x7f9b780a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 --2- 172.21.15.44:0/3527835512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b94102080 0x7f9b9419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 >> 172.21.15.44:0/3527835512 conn(0x7f9b940fd900 msgr2=0x7f9b940fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 shutdown_connections 2024-04-20T11:26:41.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:41.034+0000 7f9b99c00640 1 -- 172.21.15.44:0/3527835512 wait complete. 2024-04-20T11:26:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:41 smithi078 ceph-mon[21464]: pgmap v125: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:41 smithi044 ceph-mon[16855]: pgmap v125: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:41 smithi165 ceph-mon[21975]: pgmap v125: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:42.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:26:42.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:26:42.051 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:26:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 -- 172.21.15.44:0/2545609517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c108590 msgr2=0x7f467c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 --2- 172.21.15.44:0/2545609517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c108590 0x7f467c108a10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4664009900 tx=0x7f466402bc40 comp rx=0 tx=0).stop 2024-04-20T11:26:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 -- 172.21.15.44:0/2545609517 shutdown_connections 2024-04-20T11:26:42.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 --2- 172.21.15.44:0/2545609517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c108f50 0x7f467c1153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 --2- 172.21.15.44:0/2545609517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c108590 0x7f467c108a10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 --2- 172.21.15.44:0/2545609517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c10e600 0x7f467c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 -- 172.21.15.44:0/2545609517 >> 172.21.15.44:0/2545609517 conn(0x7f467c0764a0 msgr2=0x7f467c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 -- 172.21.15.44:0/2545609517 shutdown_connections 2024-04-20T11:26:42.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.282+0000 7f4680a00640 1 -- 172.21.15.44:0/2545609517 wait complete. 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.283+0000 7f4680a00640 1 Processor -- start 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 -- start start 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 0x7f467c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c108f50 0x7f467c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c10e600 0x7f467c1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1189d0 con 0x7f467c10e600 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f467c118850 con 0x7f467c108590 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4680a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f467c118b50 con 0x7f467c108f50 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4679e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 0x7f467c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4679e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 0x7f467c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60848/0 (socket says 172.21.15.44:60848) 2024-04-20T11:26:42.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f4679e00640 1 -- 172.21.15.44:0/2237198662 learned_addr learned my addr 172.21.15.44:0/2237198662 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.284+0000 7f467a800640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c10e600 0x7f467c1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679400640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c108f50 0x7f467c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679e00640 1 -- 172.21.15.44:0/2237198662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c108f50 msgr2=0x7f467c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679e00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c108f50 0x7f467c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679e00640 1 -- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c10e600 msgr2=0x7f467c1a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679e00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c10e600 0x7f467c1a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679e00640 1 -- 172.21.15.44:0/2237198662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c1a3ee0 con 0x7f467c108590 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4679e00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 0x7f467c19eef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f466c00e9f0 tx=0x7f466c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.285+0000 7f4672a00640 1 -- 172.21.15.44:0/2237198662 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c00cd80 con 0x7f467c108590 2024-04-20T11:26:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.286+0000 7f4672a00640 1 -- 172.21.15.44:0/2237198662 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f466c010930 con 0x7f467c108590 2024-04-20T11:26:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.286+0000 7f4672a00640 1 -- 172.21.15.44:0/2237198662 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c018b00 con 0x7f467c108590 2024-04-20T11:26:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.286+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c1a41d0 con 0x7f467c108590 2024-04-20T11:26:42.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.286+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c1aba60 con 0x7f467c108590 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.289+0000 7f4683779740 1 Processor -- start 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.289+0000 7f4683779740 1 -- start start 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.289+0000 7f4672a00640 1 -- 172.21.15.44:0/2237198662 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f466c018ca0 con 0x7f467c108590 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4683779740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea312fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4683779740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 0x55ea31304ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4683779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea31305410 0x55ea31307800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4683779740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ea31260d20 con 0x55ea312fc830 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4683779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea312073e0 con 0x55ea31305410 2024-04-20T11:26:42.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4683779740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ea3125cce0 con 0x55ea312fbf10 2024-04-20T11:26:42.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 0x55ea31304ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 0x55ea31304ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36058/0 (socket says 172.21.15.44:36058) 2024-04-20T11:26:42.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 -- 172.21.15.44:0/2696150885 learned_addr learned my addr 172.21.15.44:0/2696150885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f467a800640 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea312fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679400640 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea31305410 0x55ea31307800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 -- 172.21.15.44:0/2696150885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 msgr2=0x55ea312fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea312fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 -- 172.21.15.44:0/2696150885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea31305410 msgr2=0x55ea31307800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.290+0000 7f4679e00640 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea31305410 0x55ea31307800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.291+0000 7f4679e00640 1 -- 172.21.15.44:0/2696150885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea31226980 con 0x55ea312fc830 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.291+0000 7f4679e00640 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 0x55ea31304ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f466c0ce150 tx=0x7f466c0d9550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ecfe0c6882dfbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.291+0000 7f466b000640 1 -- 172.21.15.44:0/2696150885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c0d9e40 con 0x55ea312fc830 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.291+0000 7f466b000640 1 -- 172.21.15.44:0/2696150885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f466c0da040 con 0x55ea312fc830 2024-04-20T11:26:42.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.291+0000 7f466b000640 1 -- 172.21.15.44:0/2696150885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c0dd8a0 con 0x55ea312fc830 2024-04-20T11:26:42.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.291+0000 7f4672a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f46600a3de0 0x7f46600a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4672a00640 1 -- 172.21.15.44:0/2237198662 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f466c0d0a20 con 0x7f467c108590 2024-04-20T11:26:42.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f467a800640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f46600a3de0 0x7f46600a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 -- 172.21.15.44:0/2696150885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 msgr2=0x55ea31304ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 0x55ea31304ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f466c0ce150 tx=0x7f466c0d9550 comp rx=0 tx=0).stop 2024-04-20T11:26:42.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 -- 172.21.15.44:0/2696150885 shutdown_connections 2024-04-20T11:26:42.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea31305410 0x55ea31307800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fc830 0x55ea31304ed0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 --2- 172.21.15.44:0/2696150885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea312fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 -- 172.21.15.44:0/2696150885 >> 172.21.15.44:0/2696150885 conn(0x55ea312f6640 msgr2=0x55ea312f6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 -- 172.21.15.44:0/2696150885 shutdown_connections 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.292+0000 7f4683779740 1 -- 172.21.15.44:0/2696150885 wait complete. 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.293+0000 7f467a800640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f46600a3de0 0x7f46600a62a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4674005fd0 tx=0x7f4674009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.293+0000 7f4683779740 1 -- start start 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.293+0000 7f4683779740 1 Processor -- start 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea31302440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea31302980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31302ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ea31308200 con 0x55ea31305410 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea3121a7a0 con 0x55ea312fc830 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ea31308380 con 0x55ea312fbf10 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4683779740 10 client.0 _ll_register_callbacks cb 0x55ea3121cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea31302440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f467a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31302ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea31302440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54060/0 (socket says 172.21.15.44:54060) 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 -- 172.21.15.44:0/1268724235 learned_addr learned my addr 172.21.15.44:0/1268724235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679400640 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea31302980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 -- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 msgr2=0x55ea31302980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea31302980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 -- 172.21.15.44:0/1268724235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 msgr2=0x55ea31302ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31302ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.294+0000 7f4679e00640 1 -- 172.21.15.44:0/1268724235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3103b7e0 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f4679e00640 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea31302440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f466c098f40 tx=0x7f466c0deda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb1e1b371ecd6b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f465f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f465f400640 1 -- 172.21.15.44:0/1268724235 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c098d80 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f465f400640 1 -- 172.21.15.44:0/1268724235 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f466c09ade0 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466c0da040 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f465f400640 1 -- 172.21.15.44:0/1268724235 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f466c09c440 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f4683779740 10 client.26521 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea3109bf70 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.295+0000 7f465f400640 1 -- 172.21.15.44:0/1268724235 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f466c0da040 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f465f400640 1 -- 172.21.15.44:0/1268724235 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f466c0ce460 con 0x55ea312fbf10 2024-04-20T11:26:42.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4683779740 10 client.26521 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4683779740 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4683779740 20 client.26521 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4683779740 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea31315620 0x55ea31317a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4683779740 4 client.26521 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea3126e740 con 0x55ea31315620 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.296+0000 7f4679400640 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea31315620 0x55ea31317a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.297+0000 7f4679400640 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea31315620 0x55ea31317a10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3201ccd760583363 server_cookie=34b5b19a36681f87 in_seq=0 out_seq=0 2024-04-20T11:26:42.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.297+0000 7f465f400640 10 client.26521 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f465f400640 1 -- 172.21.15.44:0/1268724235 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea3126e740 con 0x55ea31315620 2024-04-20T11:26:42.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f465f400640 10 client.26521 handle_command_reply: tid=0 2024-04-20T11:26:42.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 client.26521 shutdown 2024-04-20T11:26:42.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 msgr2=0x55ea31302440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea31302440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f466c098f40 tx=0x7f466c0deda0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 shutdown_connections 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea31315620 0x55ea31317a10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31302ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea31302980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f465f400640 0 client.26521 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 --2- 172.21.15.44:0/1268724235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fbf10 0x55ea31302440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 >> 172.21.15.44:0/1268724235 conn(0x55ea312f6640 msgr2=0x55ea312f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 shutdown_connections 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 1 -- 172.21.15.44:0/1268724235 wait complete. 2024-04-20T11:26:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.298+0000 7f4683779740 20 client.26521 trim_cache size 0 max 16384 2024-04-20T11:26:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 Processor -- start 2024-04-20T11:26:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 -- start start 2024-04-20T11:26:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fbf10 0x55ea31304b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea3131aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea31305410 0x55ea3131af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1b4380 con 0x55ea312fbf10 2024-04-20T11:26:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1b60a0 con 0x55ea312fc830 2024-04-20T11:26:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4683779740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1b4a50 con 0x55ea31305410 2024-04-20T11:26:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.318+0000 7f4679400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea3131aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f467a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea31305410 0x55ea3131af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea3131aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60872/0 (socket says 172.21.15.44:60872) 2024-04-20T11:26:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f467a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea31305410 0x55ea3131af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54074/0 (socket says 172.21.15.44:54074) 2024-04-20T11:26:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fbf10 0x55ea31304b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679400640 1 -- 172.21.15.44:0/1734670360 learned_addr learned my addr 172.21.15.44:0/1734670360 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 -- 172.21.15.44:0/1734670360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea31305410 msgr2=0x55ea3131af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea31305410 0x55ea3131af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 -- 172.21.15.44:0/1734670360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 msgr2=0x55ea3131aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea3131aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 -- 172.21.15.44:0/1734670360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea312fc660 con 0x55ea312fbf10 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.319+0000 7f4679e00640 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fbf10 0x55ea31304b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f466c099460 tx=0x7f466c0d9550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74bc338a0d01738 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f466a600640 1 -- 172.21.15.44:0/1734670360 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c09c620 con 0x55ea312fbf10 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f466a600640 1 -- 172.21.15.44:0/1734670360 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f466c0d4830 con 0x55ea312fbf10 2024-04-20T11:26:42.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f466a600640 1 -- 172.21.15.44:0/1734670360 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c09c8b0 con 0x55ea312fbf10 2024-04-20T11:26:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 -- 172.21.15.44:0/1734670360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fbf10 msgr2=0x55ea31304b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fbf10 0x55ea31304b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f466c099460 tx=0x7f466c0d9550 comp rx=0 tx=0).stop 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 -- 172.21.15.44:0/1734670360 shutdown_connections 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea31305410 0x55ea3131af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fc830 0x55ea3131aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 --2- 172.21.15.44:0/1734670360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea312fbf10 0x55ea31304b70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 -- 172.21.15.44:0/1734670360 >> 172.21.15.44:0/1734670360 conn(0x55ea312f6640 msgr2=0x55ea312f5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.320+0000 7f4683779740 1 -- 172.21.15.44:0/1734670360 shutdown_connections 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.321+0000 7f4683779740 1 -- 172.21.15.44:0/1734670360 wait complete. 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.321+0000 7f4683779740 1 -- start start 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.321+0000 7f4683779740 1 Processor -- start 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fbf10 0x55ea3131a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fc830 0x55ea3131ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31318c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1b5910 con 0x55ea31305410 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1b51b0 con 0x55ea312fbf10 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1b22c0 con 0x55ea312fc830 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4683779740 10 client.0 _ll_register_callbacks cb 0x7f46740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31318c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31318c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36086/0 (socket says 172.21.15.44:36086) 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 -- 172.21.15.44:0/4253600370 learned_addr learned my addr 172.21.15.44:0/4253600370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4679e00640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fbf10 0x55ea3131a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4679400640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fc830 0x55ea3131ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 -- 172.21.15.44:0/4253600370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fc830 msgr2=0x55ea3131ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fc830 0x55ea3131ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 -- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fbf10 msgr2=0x55ea3131a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fbf10 0x55ea3131a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f467a800640 1 -- 172.21.15.44:0/4253600370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea30f33170 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.322+0000 7f4679e00640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fbf10 0x55ea3131a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f467a800640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31318c30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4674011470 tx=0x7f4674004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa048e88fa440a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f465f400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f465f400640 1 -- 172.21.15.44:0/4253600370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674004b20 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f465f400640 1 -- 172.21.15.44:0/4253600370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4674004cc0 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466c0d4830 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f465f400640 1 -- 172.21.15.44:0/4253600370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f467400f870 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f4683779740 10 client.16401 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.323+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea3137c170 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f465f400640 1 -- 172.21.15.44:0/4253600370 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f466c0d4830 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f465f400640 1 -- 172.21.15.44:0/4253600370 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f4674031030 con 0x55ea31305410 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4683779740 10 client.16401 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4683779740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4683779740 20 client.16401 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:42.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4683779740 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea3137ca00 0x55ea3137ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4683779740 4 client.16401 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:26:42.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea312685b0 con 0x55ea3137ca00 2024-04-20T11:26:42.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.324+0000 7f4679e00640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea3137ca00 0x55ea3137ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.325+0000 7f4679e00640 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea3137ca00 0x55ea3137ee20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58c4a03647048fd server_cookie=2f546ef4632fe406 in_seq=0 out_seq=0 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.325+0000 7f465f400640 10 client.16401 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f465f400640 1 -- 172.21.15.44:0/4253600370 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea312685b0 con 0x55ea3137ca00 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f465f400640 10 client.16401 handle_command_reply: tid=0 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 client.16401 shutdown 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 msgr2=0x55ea31318c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31318c30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4674011470 tx=0x7f4674004a40 comp rx=0 tx=0).stop 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 shutdown_connections 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55ea3137ca00 0x55ea3137ee20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea31305410 0x55ea31318c30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea312fc830 0x55ea3131ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f465f400640 0 client.16401 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 --2- 172.21.15.44:0/4253600370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea312fbf10 0x55ea3131a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 >> 172.21.15.44:0/4253600370 conn(0x55ea312f6640 msgr2=0x55ea312f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 shutdown_connections 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.326+0000 7f4683779740 1 -- 172.21.15.44:0/4253600370 wait complete. 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4683779740 20 client.16401 trim_cache size 0 max 16384 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "0a756998-e7e5-433e-8394-c25c6bf06964", 2024-04-20T11:26:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:26:42.330 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f46600a3de0 msgr2=0x7f46600a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f46600a3de0 0x7f46600a62a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4674005fd0 tx=0x7f4674009500 comp rx=0 tx=0).stop 2024-04-20T11:26:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 msgr2=0x7f467c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 0x7f467c19eef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f466c00e9f0 tx=0x7f466c00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 shutdown_connections 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f46600a3de0 0x7f46600a62a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f467c10e600 0x7f467c1a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c108f50 0x7f467c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.327+0000 7f4680a00640 1 --2- 172.21.15.44:0/2237198662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108590 0x7f467c19eef0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.328+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 >> 172.21.15.44:0/2237198662 conn(0x7f467c0764a0 msgr2=0x7f467c077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.328+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 shutdown_connections 2024-04-20T11:26:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.328+0000 7f4680a00640 1 -- 172.21.15.44:0/2237198662 wait complete. 2024-04-20T11:26:42.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:26:42.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:26:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 -- 172.21.15.44:0/2411548902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 msgr2=0x7f325c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 --2- 172.21.15.44:0/2411548902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c102450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f324c009900 tx=0x7f324c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 -- 172.21.15.44:0/2411548902 shutdown_connections 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 --2- 172.21.15.44:0/2411548902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 0x7f325c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 --2- 172.21.15.44:0/2411548902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f325c102a20 0x7f325c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 --2- 172.21.15.44:0/2411548902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c102450 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 -- 172.21.15.44:0/2411548902 >> 172.21.15.44:0/2411548902 conn(0x7f325c0fd930 msgr2=0x7f325c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 -- 172.21.15.44:0/2411548902 shutdown_connections 2024-04-20T11:26:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.564+0000 7f3260e00640 1 -- 172.21.15.44:0/2411548902 wait complete. 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 Processor -- start 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 -- start start 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 0x7f325c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f325c19a7e0 0x7f325c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1122e0 con 0x7f325c102070 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f325c112160 con 0x7f325c19a7e0 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.565+0000 7f3260e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f325c112460 con 0x7f325c107140 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f325b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 0x7f325c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f325b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36104/0 (socket says 172.21.15.44:36104) 2024-04-20T11:26:42.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f325b400640 1 -- 172.21.15.44:0/1708433536 learned_addr learned my addr 172.21.15.44:0/1708433536 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f325be00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f325c19a7e0 0x7f325c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 -- 172.21.15.44:0/1708433536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f325c19a7e0 msgr2=0x7f325c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f325c19a7e0 0x7f325c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 -- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 msgr2=0x7f325c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 -- 172.21.15.44:0/1708433536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325c19b370 con 0x7f325c107140 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f325b400640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.566+0000 7f3252a00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 0x7f325c1a0bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f324400e9c0 tx=0x7f324400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.567+0000 7f3258c00640 1 -- 172.21.15.44:0/1708433536 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324400ccc0 con 0x7f325c107140 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.567+0000 7f3258c00640 1 -- 172.21.15.44:0/1708433536 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3244010930 con 0x7f325c107140 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.567+0000 7f3258c00640 1 -- 172.21.15.44:0/1708433536 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440189c0 con 0x7f325c107140 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.567+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325c19b660 con 0x7f325c107140 2024-04-20T11:26:42.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.567+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325c1a77f0 con 0x7f325c107140 2024-04-20T11:26:42.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.570+0000 7f32651ff740 1 Processor -- start 2024-04-20T11:26:42.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.570+0000 7f3258c00640 1 -- 172.21.15.44:0/1708433536 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3244010ae0 con 0x7f325c107140 2024-04-20T11:26:42.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.570+0000 7f32651ff740 1 -- start start 2024-04-20T11:26:42.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f32651ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410d8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f32651ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f32651ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410e3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f32651ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b24103e150 con 0x55b2410d8380 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f32651ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b240fe4370 con 0x55b2410e1880 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f32651ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b24103a110 con 0x55b2410d8ca0 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f325be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f325be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410d8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36108/0 (socket says 172.21.15.44:36108) 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f325be00640 1 -- 172.21.15.44:0/3632286456 learned_addr learned my addr 172.21.15.44:0/3632286456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f3252a00640 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410e3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f325b400640 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f325be00640 1 -- 172.21.15.44:0/3632286456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 msgr2=0x55b2410e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.571+0000 7f325be00640 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f325be00640 1 -- 172.21.15.44:0/3632286456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 msgr2=0x55b2410e3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f325be00640 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410e3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f325be00640 1 -- 172.21.15.44:0/3632286456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b240e1a7d0 con 0x55b2410d8380 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f3258c00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32400a3d20 0x7f32400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f325be00640 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410d8760 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3254009f40 tx=0x7f3254010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b946b7cec1aebe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f3258c00640 1 -- 172.21.15.44:0/1708433536 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f32440d0b30 con 0x7f325c107140 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f325b400640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32400a3d20 0x7f32400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f3250c00640 1 -- 172.21.15.44:0/3632286456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254003b70 con 0x55b2410d8380 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f3250c00640 1 -- 172.21.15.44:0/3632286456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3254003d10 con 0x55b2410d8380 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.572+0000 7f3250c00640 1 -- 172.21.15.44:0/3632286456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254011b60 con 0x55b2410d8380 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 -- 172.21.15.44:0/3632286456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 msgr2=0x55b2410d8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410d8760 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3254009f40 tx=0x7f3254010860 comp rx=0 tx=0).stop 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f325b400640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32400a3d20 0x7f32400a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c005a20 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 -- 172.21.15.44:0/3632286456 shutdown_connections 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410e3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 --2- 172.21.15.44:0/3632286456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410d8760 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 -- 172.21.15.44:0/3632286456 >> 172.21.15.44:0/3632286456 conn(0x55b2410d2ab0 msgr2=0x55b2410d2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 -- 172.21.15.44:0/3632286456 shutdown_connections 2024-04-20T11:26:42.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.573+0000 7f32651ff740 1 -- 172.21.15.44:0/3632286456 wait complete. 2024-04-20T11:26:42.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 -- start start 2024-04-20T11:26:42.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 Processor -- start 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410deef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b2410e4670 con 0x55b2410d8380 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b240ff7740 con 0x55b2410e1880 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.574+0000 7f32651ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b2410e47f0 con 0x55b2410d8ca0 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f32651ff740 10 client.0 _ll_register_callbacks cb 0x55b240ff2b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410de9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36112/0 (socket says 172.21.15.44:36112) 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 -- 172.21.15.44:0/4011744480 learned_addr learned my addr 172.21.15.44:0/4011744480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f3252a00640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410deef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325be00640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 -- 172.21.15.44:0/4011744480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 msgr2=0x55b2410deef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410deef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 -- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 msgr2=0x55b2410df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 -- 172.21.15.44:0/4011744480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b240d503f0 con 0x55b2410d8380 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f3252a00640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410deef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f325b400640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410de9b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f324c036040 tx=0x7f324c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1e75416acf1c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.575+0000 7f3249600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f3249600640 1 -- 172.21.15.44:0/4011744480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c04f020 con 0x55b2410d8380 2024-04-20T11:26:42.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f3249600640 1 -- 172.21.15.44:0/4011744480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f324c004990 con 0x55b2410d8380 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3254003d10 con 0x55b2410d8380 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f3249600640 1 -- 172.21.15.44:0/4011744480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f324c05a050 con 0x55b2410d8380 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f32651ff740 10 client.16413 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b240d100f0 con 0x55b2410d8380 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.576+0000 7f3249600640 1 -- 172.21.15.44:0/4011744480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3254003d10 con 0x55b2410d8380 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f3249600640 1 -- 172.21.15.44:0/4011744480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f324c0344e0 con 0x55b2410d8380 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f32651ff740 10 client.16413 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f32651ff740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f32651ff740 20 client.16413 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f32651ff740 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2410f1dd0 0x55b2410f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f32651ff740 4 client.16413 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b24104bb70 con 0x55b2410f1dd0 2024-04-20T11:26:42.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.577+0000 7f3252a00640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2410f1dd0 0x55b2410f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.578+0000 7f3252a00640 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2410f1dd0 0x55b2410f4220 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2e13e48d81b8df server_cookie=5e60bdef4929dd23 in_seq=0 out_seq=0 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.578+0000 7f3249600640 10 client.16413 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f3249600640 1 -- 172.21.15.44:0/4011744480 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b24104bb70 con 0x55b2410f1dd0 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f3249600640 10 client.16413 handle_command_reply: tid=0 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 client.16413 shutdown 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 msgr2=0x55b2410de9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410de9b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f324c036040 tx=0x7f324c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 shutdown_connections 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2410f1dd0 0x55b2410f4220 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410deef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 --2- 172.21.15.44:0/4011744480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410de9b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f3249600640 0 client.16413 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 >> 172.21.15.44:0/4011744480 conn(0x55b2410d2ab0 msgr2=0x55b2410d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 shutdown_connections 2024-04-20T11:26:42.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 1 -- 172.21.15.44:0/4011744480 wait complete. 2024-04-20T11:26:42.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.579+0000 7f32651ff740 20 client.16413 trim_cache size 0 max 16384 2024-04-20T11:26:42.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 Processor -- start 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 -- start start 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410d8380 0x55b2410e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 0x55b2410ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410e1880 0x55b2410ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1afce0 con 0x55b2410d8ca0 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1b1a00 con 0x55b2410d8380 2024-04-20T11:26:42.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.599+0000 7f32651ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1b03b0 con 0x55b2410e1880 2024-04-20T11:26:42.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 0x55b2410ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f325b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410d8380 0x55b2410e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 0x55b2410ed530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36122/0 (socket says 172.21.15.44:36122) 2024-04-20T11:26:42.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 -- 172.21.15.44:0/86900507 learned_addr learned my addr 172.21.15.44:0/86900507 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f325be00640 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410e1880 0x55b2410ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 -- 172.21.15.44:0/86900507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410e1880 msgr2=0x55b2410ddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410e1880 0x55b2410ddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 -- 172.21.15.44:0/86900507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410d8380 msgr2=0x55b2410e10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410d8380 0x55b2410e10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 -- 172.21.15.44:0/86900507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2410e3fe0 con 0x55b2410d8ca0 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f325be00640 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410e1880 0x55b2410ddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.600+0000 7f3252a00640 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 0x55b2410ed530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3244099510 tx=0x7f32440d30b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76008607e0be9799 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f324be00640 1 -- 172.21.15.44:0/86900507 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324409c4a0 con 0x55b2410d8ca0 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f324be00640 1 -- 172.21.15.44:0/86900507 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f324409d240 con 0x55b2410d8ca0 2024-04-20T11:26:42.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f324be00640 1 -- 172.21.15.44:0/86900507 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324400ce60 con 0x55b2410d8ca0 2024-04-20T11:26:42.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 -- 172.21.15.44:0/86900507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 msgr2=0x55b2410ed530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 0x55b2410ed530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3244099510 tx=0x7f32440d30b0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 -- 172.21.15.44:0/86900507 shutdown_connections 2024-04-20T11:26:42.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410e1880 0x55b2410ddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8ca0 0x55b2410ed530 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 --2- 172.21.15.44:0/86900507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410d8380 0x55b2410e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.601+0000 7f32651ff740 1 -- 172.21.15.44:0/86900507 >> 172.21.15.44:0/86900507 conn(0x55b2410d2ab0 msgr2=0x55b2410d3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.602+0000 7f32651ff740 1 -- 172.21.15.44:0/86900507 shutdown_connections 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.602+0000 7f32651ff740 1 -- 172.21.15.44:0/86900507 wait complete. 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.602+0000 7f32651ff740 1 -- start start 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.602+0000 7f32651ff740 1 Processor -- start 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1b1270 con 0x55b2410d8380 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1b0b10 con 0x55b2410e1880 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f325c1adc20 con 0x55b2410d8ca0 2024-04-20T11:26:42.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f32651ff740 10 client.0 _ll_register_callbacks cb 0x7f324c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410e9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36128/0 (socket says 172.21.15.44:36128) 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 -- 172.21.15.44:0/1745088448 learned_addr learned my addr 172.21.15.44:0/1745088448 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f3252a00640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325be00640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 -- 172.21.15.44:0/1745088448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 msgr2=0x55b2410e9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 -- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 msgr2=0x55b2410db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410db090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325b400640 1 -- 172.21.15.44:0/1745088448 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2410db8c0 con 0x55b2410d8380 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.603+0000 7f325be00640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f325b400640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410e9a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f324c006010 tx=0x7f324c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf37b82e0998315 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f3249600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:42.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f3249600640 1 -- 172.21.15.44:0/1745088448 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c04f070 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f3249600640 1 -- 172.21.15.44:0/1745088448 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f324c004990 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324409d240 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f3249600640 1 -- 172.21.15.44:0/1745088448 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f324c0346e0 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f32651ff740 10 client.16419 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.604+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b241157f50 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f3249600640 1 -- 172.21.15.44:0/1745088448 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f324409d240 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f3249600640 1 -- 172.21.15.44:0/1745088448 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f324c040450 con 0x55b2410d8380 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f32651ff740 10 client.16419 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f32651ff740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f32651ff740 20 client.16419 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f32651ff740 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2411587e0 0x55b24115ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f32651ff740 4 client.16419 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:26:42.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2410ce170 con 0x55b2411587e0 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.605+0000 7f3252a00640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2411587e0 0x55b24115ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.606+0000 7f3252a00640 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2411587e0 0x55b24115ac00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec62c666e030300 server_cookie=de91bfe5eb579f46 in_seq=0 out_seq=0 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.606+0000 7f3249600640 10 client.16419 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.606+0000 7f3249600640 1 -- 172.21.15.44:0/1745088448 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b2410ce170 con 0x55b2411587e0 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.606+0000 7f3249600640 10 client.16419 handle_command_reply: tid=0 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.606+0000 7f32651ff740 1 client.16419 shutdown 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 msgr2=0x55b2410e9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410e9a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f324c006010 tx=0x7f324c004460 comp rx=0 tx=0).stop 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 shutdown_connections 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55b2411587e0 0x55b24115ac00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2410e1880 0x55b2410db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2410d8ca0 0x55b2410e9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f3249600640 0 client.16419 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 --2- 172.21.15.44:0/1745088448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2410d8380 0x55b2410e9a40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 >> 172.21.15.44:0/1745088448 conn(0x55b2410d2ab0 msgr2=0x55b2410de950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 shutdown_connections 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 1 -- 172.21.15.44:0/1745088448 wait complete. 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.607+0000 7f32651ff740 20 client.16419 trim_cache size 0 max 16384 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:26:42.610 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:26:42.611 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:42.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32400a3d20 msgr2=0x7f32400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32400a3d20 0x7f32400a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c005a20 comp rx=0 tx=0).stop 2024-04-20T11:26:42.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 msgr2=0x7f325c1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 0x7f325c1a0bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f324400e9c0 tx=0x7f324400ee80 comp rx=0 tx=0).stop 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 shutdown_connections 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f325c19a7e0 0x7f325c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f325c107140 0x7f325c1a0bb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32400a3d20 0x7f32400a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 --2- 172.21.15.44:0/1708433536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f325c102070 0x7f325c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 >> 172.21.15.44:0/1708433536 conn(0x7f325c0fd930 msgr2=0x7f325c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 shutdown_connections 2024-04-20T11:26:42.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.608+0000 7f3260e00640 1 -- 172.21.15.44:0/1708433536 wait complete. 2024-04-20T11:26:42.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:26:42.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:26:42.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:42 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:26:42.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:42 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:42.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:42 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:26:42.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:42 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:42.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:42 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:26:42.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 -- 172.21.15.44:0/415259597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd78108400 msgr2=0x7fcd78108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 --2- 172.21.15.44:0/415259597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd78108400 0x7fcd78108860 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 -- 172.21.15.44:0/415259597 shutdown_connections 2024-04-20T11:26:42.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 --2- 172.21.15.44:0/415259597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd78115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 --2- 172.21.15.44:0/415259597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd78108400 0x7fcd78108860 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 --2- 172.21.15.44:0/415259597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd7810e470 0x7fcd7810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.845+0000 7fcd77200640 1 -- 172.21.15.44:0/415259597 >> 172.21.15.44:0/415259597 conn(0x7fcd78077ae0 msgr2=0x7fcd78079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:42.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.846+0000 7fcd77200640 1 -- 172.21.15.44:0/415259597 shutdown_connections 2024-04-20T11:26:42.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.846+0000 7fcd77200640 1 -- 172.21.15.44:0/415259597 wait complete. 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 Processor -- start 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 -- start start 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd78108400 0x7fcd781a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd781a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd7810e470 0x7fcd7819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781188c0 con 0x7fcd78108da0 2024-04-20T11:26:42.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78118740 con 0x7fcd7810e470 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd77200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78118a40 con 0x7fcd78108400 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd6d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd781a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd6d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd781a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36154/0 (socket says 172.21.15.44:36154) 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.847+0000 7fcd6d400640 1 -- 172.21.15.44:0/3096974435 learned_addr learned my addr 172.21.15.44:0/3096974435 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd7810e470 0x7fcd7819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd75e00640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd78108400 0x7fcd781a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 -- 172.21.15.44:0/3096974435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd78108400 msgr2=0x7fcd781a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd78108400 0x7fcd781a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 -- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 msgr2=0x7fcd781a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd781a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 -- 172.21.15.44:0/3096974435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7819f240 con 0x7fcd7810e470 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd6d400640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd781a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd76800640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd7810e470 0x7fcd7819eac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e970 tx=0x7fcd7000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000ccc0 con 0x7fcd7810e470 2024-04-20T11:26:42.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7819f530 con 0x7fcd7810e470 2024-04-20T11:26:42.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd70005020 con 0x7fcd7810e470 2024-04-20T11:26:42.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.848+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70004a20 con 0x7fcd7810e470 2024-04-20T11:26:42.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.849+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781ab7d0 con 0x7fcd7810e470 2024-04-20T11:26:42.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.850+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd780690f0 con 0x7fcd7810e470 2024-04-20T11:26:42.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.852+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcd70004c20 con 0x7fcd7810e470 2024-04-20T11:26:42.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.853+0000 7fcd6f400640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd5c0a3da0 0x7fcd5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:42.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.853+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fcd700d0780 con 0x7fcd7810e470 2024-04-20T11:26:42.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.853+0000 7fcd75e00640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd5c0a3da0 0x7fcd5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:42.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.854+0000 7fcd75e00640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd5c0a3da0 0x7fcd5c0a6260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd7819fc00 tx=0x7fcd68009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:42.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:42.857+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcd7009ce30 con 0x7fcd7810e470 2024-04-20T11:26:42.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:42 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:26:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:42 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:42 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:26:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:42 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:42 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:26:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:42 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:26:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:42 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:42 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:26:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:42 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:42 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:26:43.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.109+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd7819fff0 con 0x7fcd7810e470 2024-04-20T11:26:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.111+0000 7fcd6f400640 1 -- 172.21.15.44:0/3096974435 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11970 (secure 0 0 0) 0x7fcd7009c730 con 0x7fcd7810e470 2024-04-20T11:26:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:26:43.111 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:26:00.778182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:26:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 39 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd5c0a3da0 msgr2=0x7fcd5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd5c0a3da0 0x7fcd5c0a6260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd7819fc00 tx=0x7fcd68009500 comp rx=0 tx=0).stop 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd7810e470 msgr2=0x7fcd7819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd7810e470 0x7fcd7819eac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e970 tx=0x7fcd7000ee30 comp rx=0 tx=0).stop 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 shutdown_connections 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd7810e470 0x7fcd7819eac0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd5c0a3da0 0x7fcd5c0a6260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd78108da0 0x7fcd781a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 --2- 172.21.15.44:0/3096974435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd78108400 0x7fcd781a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 >> 172.21.15.44:0/3096974435 conn(0x7fcd78077ae0 msgr2=0x7fcd780790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 shutdown_connections 2024-04-20T11:26:43.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.114+0000 7fcd77200640 1 -- 172.21.15.44:0/3096974435 wait complete. 2024-04-20T11:26:43.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:26:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 -- 172.21.15.44:0/3833012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c102b50 msgr2=0x7fbd5c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 --2- 172.21.15.44:0/3833012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c102b50 0x7fbd5c106b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009ca0 tx=0x7fbd4402c100 comp rx=0 tx=0).stop 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 -- 172.21.15.44:0/3833012365 shutdown_connections 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 --2- 172.21.15.44:0/3833012365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 --2- 172.21.15.44:0/3833012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c102b50 0x7fbd5c106b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 --2- 172.21.15.44:0/3833012365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd5c1021a0 0x7fbd5c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.354+0000 7fbd61c00640 1 -- 172.21.15.44:0/3833012365 >> 172.21.15.44:0/3833012365 conn(0x7fbd5c0793b0 msgr2=0x7fbd5c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.355+0000 7fbd61c00640 1 -- 172.21.15.44:0/3833012365 shutdown_connections 2024-04-20T11:26:43.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.355+0000 7fbd61c00640 1 -- 172.21.15.44:0/3833012365 wait complete. 2024-04-20T11:26:43.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 Processor -- start 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 -- start start 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c1021a0 0x7fbd5c074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd5c074ce0 0x7fbd5c1a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c112440 con 0x7fbd5c074ce0 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1122c0 con 0x7fbd5c107270 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.356+0000 7fbd61c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1125c0 con 0x7fbd5c1021a0 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd52000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd52000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c0747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60978/0 (socket says 172.21.15.44:60978) 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd52000640 1 -- 172.21.15.44:0/2026550574 learned_addr learned my addr 172.21.15.44:0/2026550574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c1021a0 0x7fbd5c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5b400640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd5c074ce0 0x7fbd5c1a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 -- 172.21.15.44:0/2026550574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 msgr2=0x7fbd5c0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 -- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd5c074ce0 msgr2=0x7fbd5c1a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd5c074ce0 0x7fbd5c1a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 -- 172.21.15.44:0/2026550574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c1a71f0 con 0x7fbd5c1021a0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd52000640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.357+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c1021a0 0x7fbd5c074260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d6d0 tx=0x7fbd4c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.358+0000 7fbd53e00640 1 -- 172.21.15.44:0/2026550574 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c017070 con 0x7fbd5c1021a0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.358+0000 7fbd53e00640 1 -- 172.21.15.44:0/2026550574 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd4c009820 con 0x7fbd5c1021a0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.358+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c1a74e0 con 0x7fbd5c1021a0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.358+0000 7fbd53e00640 1 -- 172.21.15.44:0/2026550574 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004e00 con 0x7fbd5c1021a0 2024-04-20T11:26:43.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.358+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c1a7a30 con 0x7fbd5c1021a0 2024-04-20T11:26:43.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.361+0000 7fbd649b9740 1 Processor -- start 2024-04-20T11:26:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.361+0000 7fbd53e00640 1 -- 172.21.15.44:0/2026550574 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbd4c0099d0 con 0x7fbd5c1021a0 2024-04-20T11:26:43.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.361+0000 7fbd649b9740 1 -- start start 2024-04-20T11:26:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d88a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d892f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8958a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd649b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ef8d7ef050 con 0x55ef8d889fb0 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd649b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef8d795490 con 0x55ef8d8934b0 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd649b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ef8d7eb010 con 0x55ef8d88a8d0 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd5b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d88a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd52000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd5b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d88a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36188/0 (socket says 172.21.15.44:36188) 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd5b400640 1 -- 172.21.15.44:0/358932657 learned_addr learned my addr 172.21.15.44:0/358932657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d892f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd52000640 1 -- 172.21.15.44:0/358932657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 msgr2=0x55ef8d892f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd52000640 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d892f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd52000640 1 -- 172.21.15.44:0/358932657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 msgr2=0x55ef8d88a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.362+0000 7fbd52000640 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d88a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd52000640 1 -- 172.21.15.44:0/358932657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef8d79d070 con 0x55ef8d8934b0 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d892f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd5b400640 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d88a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd52000640 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8958a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd4402c710 tx=0x7fbd44035eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a5ba54b2416eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd4be00640 1 -- 172.21.15.44:0/358932657 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44043070 con 0x55ef8d8934b0 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd53e00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd400a3d50 0x7fbd400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd4be00640 1 -- 172.21.15.44:0/358932657 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd44032d40 con 0x55ef8d8934b0 2024-04-20T11:26:43.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd4be00640 1 -- 172.21.15.44:0/358932657 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44039040 con 0x55ef8d8934b0 2024-04-20T11:26:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.363+0000 7fbd53e00640 1 -- 172.21.15.44:0/2026550574 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbd4c0d01f0 con 0x7fbd5c1021a0 2024-04-20T11:26:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd5b400640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd400a3d50 0x7fbd400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 -- 172.21.15.44:0/358932657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 msgr2=0x55ef8d8958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8958a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd4402c710 tx=0x7fbd44035eb0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 -- 172.21.15.44:0/358932657 shutdown_connections 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8958a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d892f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 --2- 172.21.15.44:0/358932657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d88a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 -- 172.21.15.44:0/358932657 >> 172.21.15.44:0/358932657 conn(0x55ef8d8846e0 msgr2=0x55ef8d884ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 -- 172.21.15.44:0/358932657 shutdown_connections 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd649b9740 1 -- 172.21.15.44:0/358932657 wait complete. 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.364+0000 7fbd5b400640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd400a3d50 0x7fbd400a6210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd540124a0 tx=0x7fbd54005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 -- start start 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 Processor -- start 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d889fb0 0x55ef8d8904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d890a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d8934b0 0x55ef8d890f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ef8d896240 con 0x55ef8d88a8d0 2024-04-20T11:26:43.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef8d7a8860 con 0x55ef8d889fb0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ef8d8963c0 con 0x55ef8d8934b0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.365+0000 7fbd649b9740 10 client.0 _ll_register_callbacks cb 0x55ef8d776fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d890a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d890a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36196/0 (socket says 172.21.15.44:36196) 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 -- 172.21.15.44:0/248968873 learned_addr learned my addr 172.21.15.44:0/248968873 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd5b400640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d8934b0 0x55ef8d890f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d889fb0 0x55ef8d8904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 -- 172.21.15.44:0/248968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d8934b0 msgr2=0x55ef8d890f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d8934b0 0x55ef8d890f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 -- 172.21.15.44:0/248968873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d889fb0 msgr2=0x55ef8d8904f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d889fb0 0x55ef8d8904f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.366+0000 7fbd52000640 1 -- 172.21.15.44:0/248968873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8d7ac950 con 0x55ef8d88a8d0 2024-04-20T11:26:43.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd52000640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d890a30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbd44032ae0 tx=0x7fbd44032b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cad947a1d27f9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd48c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd48c00640 1 -- 172.21.15.44:0/248968873 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4404c030 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd48c00640 1 -- 172.21.15.44:0/248968873 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbd44032340 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd44032d40 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd48c00640 1 -- 172.21.15.44:0/248968873 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbd44059050 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd649b9740 10 client.16437 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.367+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ef8d77a820 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd48c00640 1 -- 172.21.15.44:0/248968873 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd44032d40 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd48c00640 1 -- 172.21.15.44:0/248968873 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fbd44035200 con 0x55ef8d88a8d0 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd649b9740 10 client.16437 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd649b9740 10 client.16437 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd649b9740 20 client.16437 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd649b9740 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d8a32f0 0x55ef8d8a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd649b9740 4 client.16437 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef8d7fca70 con 0x55ef8d8a32f0 2024-04-20T11:26:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.368+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d8a32f0 0x55ef8d8a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.369+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d8a32f0 0x55ef8d8a56e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86f15996029dfd52 server_cookie=9981bf6dc47908cd in_seq=0 out_seq=0 2024-04-20T11:26:43.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.369+0000 7fbd48c00640 10 client.16437 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd48c00640 1 -- 172.21.15.44:0/248968873 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef8d7fca70 con 0x55ef8d8a32f0 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd48c00640 10 client.16437 handle_command_reply: tid=0 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 client.16437 shutdown 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 msgr2=0x55ef8d890a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d890a30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbd44032ae0 tx=0x7fbd44032b10 comp rx=0 tx=0).stop 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 shutdown_connections 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d8934b0 0x55ef8d890f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d890a30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d8a32f0 0x55ef8d8a56e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 --2- 172.21.15.44:0/248968873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d889fb0 0x55ef8d8904f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 >> 172.21.15.44:0/248968873 conn(0x55ef8d8846e0 msgr2=0x55ef8d886430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd48c00640 0 client.16437 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 shutdown_connections 2024-04-20T11:26:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 1 -- 172.21.15.44:0/248968873 wait complete. 2024-04-20T11:26:43.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.370+0000 7fbd649b9740 20 client.16437 trim_cache size 0 max 16384 2024-04-20T11:26:43.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.385+0000 7fbd649b9740 1 Processor -- start 2024-04-20T11:26:43.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 -- start start 2024-04-20T11:26:43.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d892c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d89f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1aff20 con 0x55ef8d889fb0 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1b1c40 con 0x55ef8d8934b0 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd649b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1b05f0 con 0x55ef8d88a8d0 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd5aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d892c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd5aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d892c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36204/0 (socket says 172.21.15.44:36204) 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd5aa00640 1 -- 172.21.15.44:0/3744904661 learned_addr learned my addr 172.21.15.44:0/3744904661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd52000640 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d89f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.386+0000 7fbd5b400640 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd5aa00640 1 -- 172.21.15.44:0/3744904661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 msgr2=0x55ef8d89f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d89f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd5aa00640 1 -- 172.21.15.44:0/3744904661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 msgr2=0x55ef8d8aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd5aa00640 1 -- 172.21.15.44:0/3744904661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef8d5013f0 con 0x55ef8d889fb0 2024-04-20T11:26:43.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d892c20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d6a0 tx=0x7fbd4c0d2640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ffbc7a655fe38c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd4b400640 1 -- 172.21.15.44:0/3744904661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c020040 con 0x55ef8d889fb0 2024-04-20T11:26:43.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd4b400640 1 -- 172.21.15.44:0/3744904661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd4c0a4e10 con 0x55ef8d889fb0 2024-04-20T11:26:43.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.387+0000 7fbd4b400640 1 -- 172.21.15.44:0/3744904661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c098a50 con 0x55ef8d889fb0 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 -- 172.21.15.44:0/3744904661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 msgr2=0x55ef8d892c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d892c20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d6a0 tx=0x7fbd4c0d2640 comp rx=0 tx=0).stop 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 -- 172.21.15.44:0/3744904661 shutdown_connections 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d8aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d88a8d0 0x55ef8d89f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3744904661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d889fb0 0x55ef8d892c20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 -- 172.21.15.44:0/3744904661 >> 172.21.15.44:0/3744904661 conn(0x55ef8d8846e0 msgr2=0x55ef8d88b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 -- 172.21.15.44:0/3744904661 shutdown_connections 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 -- 172.21.15.44:0/3744904661 wait complete. 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.388+0000 7fbd649b9740 1 -- start start 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 Processor -- start 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 0x55ef8d89b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d896bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d897110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1b14b0 con 0x55ef8d88a8d0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1b0d50 con 0x55ef8d8934b0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1ade80 con 0x55ef8d889fb0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd649b9740 10 client.0 _ll_register_callbacks cb 0x7fbd44032970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd52000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d896bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd5aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 0x55ef8d89b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd5b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d897110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd5b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d897110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32784/0 (socket says 172.21.15.44:32784) 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 -- 172.21.15.44:0/3763436071 learned_addr learned my addr 172.21.15.44:0/3763436071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.389+0000 7fbd5aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 0x55ef8d89b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54200/0 (socket says 172.21.15.44:54200) 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 -- 172.21.15.44:0/3763436071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 msgr2=0x55ef8d89b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 0x55ef8d89b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 -- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 msgr2=0x55ef8d896bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d896bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 -- 172.21.15.44:0/3763436071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8d5cb7d0 con 0x55ef8d8934b0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 0x55ef8d89b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd52000640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d896bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd5b400640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d897110 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbd540044e0 tx=0x7fbd540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6ab876bd3e4d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.390+0000 7fbd48c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.391+0000 7fbd48c00640 1 -- 172.21.15.44:0/3763436071 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5402e060 con 0x55ef8d8934b0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.391+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd4c0a4e10 con 0x55ef8d8934b0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.391+0000 7fbd48c00640 1 -- 172.21.15.44:0/3763436071 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fbd5401f700 con 0x55ef8d8934b0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.391+0000 7fbd48c00640 1 -- 172.21.15.44:0/3763436071 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbd5401f900 con 0x55ef8d8934b0 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.391+0000 7fbd649b9740 10 client.26573 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:43.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.391+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ef8d9093a0 con 0x55ef8d8934b0 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd48c00640 1 -- 172.21.15.44:0/3763436071 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbd4c0a4e10 con 0x55ef8d8934b0 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd48c00640 1 -- 172.21.15.44:0/3763436071 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7fbd54031020 con 0x55ef8d8934b0 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd649b9740 10 client.26573 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd649b9740 10 client.26573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd649b9740 20 client.26573 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d909bc0 0x55ef8d90bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd649b9740 4 client.26573 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef8d7f68e0 con 0x55ef8d909bc0 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.392+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d909bc0 0x55ef8d90bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd5aa00640 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d909bc0 0x55ef8d90bfe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5023b83ff19d5b4d server_cookie=a9bb9f9092e5cc90 in_seq=0 out_seq=0 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd48c00640 10 client.26573 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd48c00640 1 -- 172.21.15.44:0/3763436071 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef8d7f68e0 con 0x55ef8d909bc0 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd48c00640 10 client.26573 handle_command_reply: tid=0 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 client.26573 shutdown 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 msgr2=0x55ef8d897110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d897110 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbd540044e0 tx=0x7fbd540046c0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 shutdown_connections 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55ef8d909bc0 0x55ef8d90bfe0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef8d8934b0 0x55ef8d897110 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ef8d88a8d0 0x55ef8d896bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd48c00640 0 client.26573 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 --2- 172.21.15.44:0/3763436071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef8d889fb0 0x55ef8d89b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 >> 172.21.15.44:0/3763436071 conn(0x55ef8d8846e0 msgr2=0x55ef8d890410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 shutdown_connections 2024-04-20T11:26:43.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.393+0000 7fbd649b9740 1 -- 172.21.15.44:0/3763436071 wait complete. 2024-04-20T11:26:43.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd649b9740 20 client.26573 trim_cache size 0 max 16384 2024-04-20T11:26:43.397 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:43.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd400a3d50 msgr2=0x7fbd400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd400a3d50 0x7fbd400a6210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd540124a0 tx=0x7fbd54005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c1021a0 msgr2=0x7fbd5c074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c1021a0 0x7fbd5c074260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d6d0 tx=0x7fbd4c00db90 comp rx=0 tx=0).stop 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 shutdown_connections 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbd400a3d50 0x7fbd400a6210 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd5c074ce0 0x7fbd5c1a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd5c107270 0x7fbd5c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 --2- 172.21.15.44:0/2026550574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd5c1021a0 0x7fbd5c074260 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 >> 172.21.15.44:0/2026550574 conn(0x7fbd5c0793b0 msgr2=0x7fbd5c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.394+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 shutdown_connections 2024-04-20T11:26:43.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.395+0000 7fbd61c00640 1 -- 172.21.15.44:0/2026550574 wait complete. 2024-04-20T11:26:43.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:26:43.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.625+0000 7f59df200640 1 -- 172.21.15.44:0/2298691507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 msgr2=0x7f59e0103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.625+0000 7f59df200640 1 --2- 172.21.15.44:0/2298691507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e0103400 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009900 tx=0x7f59d002bcc0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 -- 172.21.15.44:0/2298691507 shutdown_connections 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 --2- 172.21.15.44:0/2298691507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e010c440 0x7f59e010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 --2- 172.21.15.44:0/2298691507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e0103940 0x7f59e010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 --2- 172.21.15.44:0/2298691507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e0103400 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 -- 172.21.15.44:0/2298691507 >> 172.21.15.44:0/2298691507 conn(0x7f59e00fc880 msgr2=0x7f59e00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 -- 172.21.15.44:0/2298691507 shutdown_connections 2024-04-20T11:26:43.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.626+0000 7f59df200640 1 -- 172.21.15.44:0/2298691507 wait complete. 2024-04-20T11:26:43.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 Processor -- start 2024-04-20T11:26:43.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 -- start start 2024-04-20T11:26:43.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e0103940 0x7f59e019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e010c440 0x7f59e0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0112360 con 0x7f59e0103020 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01121e0 con 0x7f59e0103940 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59df200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01124e0 con 0x7f59e010c440 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.627+0000 7f59dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36238/0 (socket says 172.21.15.44:36238) 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 -- 172.21.15.44:0/3187905593 learned_addr learned my addr 172.21.15.44:0/3187905593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59d5400640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e0103940 0x7f59e019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59de800640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e010c440 0x7f59e0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 -- 172.21.15.44:0/3187905593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e010c440 msgr2=0x7f59e0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e010c440 0x7f59e0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 -- 172.21.15.44:0/3187905593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e0103940 msgr2=0x7f59e019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e0103940 0x7f59e019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 -- 172.21.15.44:0/3187905593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e0198fc0 con 0x7f59e0103020 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59d5400640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e0103940 0x7f59e019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:43.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.628+0000 7f59dde00640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e019e5f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f59d0005ab0 tx=0x7f59d0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.629+0000 7f59d7400640 1 -- 172.21.15.44:0/3187905593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0043020 con 0x7f59e0103020 2024-04-20T11:26:43.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.629+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e0199250 con 0x7f59e0103020 2024-04-20T11:26:43.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.629+0000 7f59d7400640 1 -- 172.21.15.44:0/3187905593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59d0004250 con 0x7f59e0103020 2024-04-20T11:26:43.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.629+0000 7f59d7400640 1 -- 172.21.15.44:0/3187905593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0003a80 con 0x7f59e0103020 2024-04-20T11:26:43.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.629+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e01a5570 con 0x7f59e0103020 2024-04-20T11:26:43.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.633+0000 7f59e73fd740 1 Processor -- start 2024-04-20T11:26:43.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.633+0000 7f59e73fd740 1 -- start start 2024-04-20T11:26:43.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59d7400640 1 -- 172.21.15.44:0/3187905593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f59d0046020 con 0x7f59e0103020 2024-04-20T11:26:43.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5862960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e586de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59e73fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d1e57c3050 con 0x55d1e5862ea0 2024-04-20T11:26:43.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59e73fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1e5769490 con 0x55d1e5862580 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59e73fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d1e57bf010 con 0x55d1e586ba80 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59de800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5862960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59de800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5862960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32816/0 (socket says 172.21.15.44:32816) 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59de800640 1 -- 172.21.15.44:0/1331026549 learned_addr learned my addr 172.21.15.44:0/1331026549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.634+0000 7f59d5400640 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e586de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59de800640 1 -- 172.21.15.44:0/1331026549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 msgr2=0x55d1e586de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59de800640 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e586de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59de800640 1 -- 172.21.15.44:0/1331026549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 msgr2=0x55d1e586b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59de800640 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59de800640 1 -- 172.21.15.44:0/1331026549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e5771070 con 0x55d1e5862580 2024-04-20T11:26:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59de800640 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5862960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009f20 tx=0x7f59d8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8893840af7725b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59cf000640 1 -- 172.21.15.44:0/1331026549 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8011700 con 0x55d1e5862580 2024-04-20T11:26:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59d7400640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f59c40a3d20 0x7f59c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59cf000640 1 -- 172.21.15.44:0/1331026549 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59d8011da0 con 0x55d1e5862580 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.635+0000 7f59cf000640 1 -- 172.21.15.44:0/1331026549 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d801a380 con 0x55d1e5862580 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59d7400640 1 -- 172.21.15.44:0/3187905593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f59d00f2710 con 0x7f59e0103020 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59dde00640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f59c40a3d20 0x7f59c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 -- 172.21.15.44:0/1331026549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 msgr2=0x55d1e5862960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5862960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009f20 tx=0x7f59d8010860 comp rx=0 tx=0).stop 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59dde00640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f59c40a3d20 0x7f59c40a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59d00f0090 tx=0x7f59d00f8550 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 -- 172.21.15.44:0/1331026549 shutdown_connections 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e586de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1331026549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5862960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.636+0000 7f59e73fd740 1 -- 172.21.15.44:0/1331026549 >> 172.21.15.44:0/1331026549 conn(0x55d1e585ccb0 msgr2=0x55d1e585d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.637+0000 7f59e73fd740 1 -- 172.21.15.44:0/1331026549 shutdown_connections 2024-04-20T11:26:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.637+0000 7f59e73fd740 1 -- 172.21.15.44:0/1331026549 wait complete. 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.637+0000 7f59e73fd740 1 -- start start 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.637+0000 7f59e73fd740 1 Processor -- start 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5868ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e5868ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5869530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d1e586e870 con 0x55d1e5862ea0 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1e577c860 con 0x55d1e5862580 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d1e586e9f0 con 0x55d1e586ba80 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59e73fd740 10 client.0 _ll_register_callbacks cb 0x55d1e574afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5868ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5868ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32826/0 (socket says 172.21.15.44:32826) 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 -- 172.21.15.44:0/301661794 learned_addr learned my addr 172.21.15.44:0/301661794 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59d5400640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e5868ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5869530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 -- 172.21.15.44:0/301661794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 msgr2=0x55d1e5869530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5869530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 -- 172.21.15.44:0/301661794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 msgr2=0x55d1e5868ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e5868ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.638+0000 7f59de800640 1 -- 172.21.15.44:0/301661794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e5780950 con 0x55d1e5862580 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59de800640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5868ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f59d801c830 tx=0x7f59d8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8afc16352dd6ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59c3400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59c3400640 1 -- 172.21.15.44:0/301661794 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d80117f0 con 0x55d1e5862580 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59c3400640 1 -- 172.21.15.44:0/301661794 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f59d8011990 con 0x55d1e5862580 2024-04-20T11:26:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d8011da0 con 0x55d1e5862580 2024-04-20T11:26:43.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59c3400640 1 -- 172.21.15.44:0/301661794 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59d8032050 con 0x55d1e5862580 2024-04-20T11:26:43.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59e73fd740 10 client.26585 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:43.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.639+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d1e574e820 con 0x55d1e5862580 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59c3400640 1 -- 172.21.15.44:0/301661794 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59d8011da0 con 0x55d1e5862580 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59c3400640 1 -- 172.21.15.44:0/301661794 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f59d801e0d0 con 0x55d1e5862580 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59e73fd740 10 client.26585 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59e73fd740 10 client.26585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59e73fd740 20 client.26585 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59e73fd740 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e587b900 0x55d1e587dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59e73fd740 4 client.26585 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e57d0a70 con 0x55d1e587b900 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.640+0000 7f59d5400640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e587b900 0x55d1e587dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.641+0000 7f59d5400640 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e587b900 0x55d1e587dcf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1bfc061d0dfaf9bd server_cookie=1ec0a7237937dbf5 in_seq=0 out_seq=0 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.641+0000 7f59c3400640 10 client.26585 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59c3400640 1 -- 172.21.15.44:0/301661794 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1e57d0a70 con 0x55d1e587b900 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59c3400640 10 client.26585 handle_command_reply: tid=0 2024-04-20T11:26:43.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 client.26585 shutdown 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 msgr2=0x55d1e5868ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5868ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f59d801c830 tx=0x7f59d8013040 comp rx=0 tx=0).stop 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 shutdown_connections 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5869530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e587b900 0x55d1e587dcf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e5868ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 --2- 172.21.15.44:0/301661794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862580 0x55d1e5868ab0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 >> 172.21.15.44:0/301661794 conn(0x55d1e585ccb0 msgr2=0x55d1e585ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.642+0000 7f59c3400640 0 client.26585 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.643+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 shutdown_connections 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.643+0000 7f59e73fd740 1 -- 172.21.15.44:0/301661794 wait complete. 2024-04-20T11:26:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.643+0000 7f59e73fd740 20 client.26585 trim_cache size 0 max 16384 2024-04-20T11:26:43.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 Processor -- start 2024-04-20T11:26:43.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 -- start start 2024-04-20T11:26:43.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862580 0x55d1e586b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 0x55d1e5877730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5867c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ab8c0 con 0x55d1e5862580 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ad5e0 con 0x55d1e5862ea0 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.658+0000 7f59e73fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01abf90 con 0x55d1e586ba80 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 0x55d1e5877730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 0x55d1e5877730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32830/0 (socket says 172.21.15.44:32830) 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 -- 172.21.15.44:0/3206726791 learned_addr learned my addr 172.21.15.44:0/3206726791 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59de800640 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862580 0x55d1e586b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 -- 172.21.15.44:0/3206726791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 msgr2=0x55d1e5867c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5867c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 -- 172.21.15.44:0/3206726791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862580 msgr2=0x55d1e586b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862580 0x55d1e586b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 -- 172.21.15.44:0/3206726791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e54d53f0 con 0x55d1e5862ea0 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59d5400640 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 0x55d1e5877730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55d1e585de40 tx=0x7f59c8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deeb8700f62dfe70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59ce600640 1 -- 172.21.15.44:0/3206726791 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8021220 con 0x55d1e5862ea0 2024-04-20T11:26:43.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59ce600640 1 -- 172.21.15.44:0/3206726791 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59c8025020 con 0x55d1e5862ea0 2024-04-20T11:26:43.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.659+0000 7f59ce600640 1 -- 172.21.15.44:0/3206726791 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8011da0 con 0x55d1e5862ea0 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 -- 172.21.15.44:0/3206726791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 msgr2=0x55d1e5877730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 0x55d1e5877730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55d1e585de40 tx=0x7f59c8005f90 comp rx=0 tx=0).stop 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 -- 172.21.15.44:0/3206726791 shutdown_connections 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e586ba80 0x55d1e5867c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e5862ea0 0x55d1e5877730 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 --2- 172.21.15.44:0/3206726791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862580 0x55d1e586b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 -- 172.21.15.44:0/3206726791 >> 172.21.15.44:0/3206726791 conn(0x55d1e585ccb0 msgr2=0x55d1e58570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 -- 172.21.15.44:0/3206726791 shutdown_connections 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.660+0000 7f59e73fd740 1 -- 172.21.15.44:0/3206726791 wait complete. 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 -- start start 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 Processor -- start 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 0x55d1e586f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e586ba80 0x55d1e5870010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ace50 con 0x55d1e5862ea0 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ac6f0 con 0x55d1e586ba80 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01a9800 con 0x55d1e5862580 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.661+0000 7f59e73fd740 10 client.0 _ll_register_callbacks cb 0x7f59d8009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59de800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 0x55d1e586f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59de800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 0x55d1e586f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54270/0 (socket says 172.21.15.44:54270) 2024-04-20T11:26:43.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59de800640 1 -- 172.21.15.44:0/1655538412 learned_addr learned my addr 172.21.15.44:0/1655538412 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59de800640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e586ba80 0x55d1e5870010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 -- 172.21.15.44:0/1655538412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 msgr2=0x55d1e586f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 0x55d1e586f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 -- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e586ba80 msgr2=0x55d1e5870010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e586ba80 0x55d1e5870010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 -- 172.21.15.44:0/1655538412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e559f7d0 con 0x55d1e5862ea0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59de800640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 0x55d1e586f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59d5400640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586fad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55d1e587e250 tx=0x7f59c8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59224f70d2932bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.662+0000 7f59c3400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59c3400640 1 -- 172.21.15.44:0/1655538412 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8011b20 con 0x55d1e5862ea0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59c3400640 1 -- 172.21.15.44:0/1655538412 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f59c8011cc0 con 0x55d1e5862ea0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c8025020 con 0x55d1e5862ea0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59c3400640 1 -- 172.21.15.44:0/1655538412 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59c8005050 con 0x55d1e5862ea0 2024-04-20T11:26:43.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59e73fd740 10 client.16458 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:43.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d1e58e1cf0 con 0x55d1e5862ea0 2024-04-20T11:26:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.663+0000 7f59c3400640 1 -- 172.21.15.44:0/1655538412 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59c8025020 con 0x55d1e5862ea0 2024-04-20T11:26:43.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59c3400640 1 -- 172.21.15.44:0/1655538412 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f59c803b050 con 0x55d1e5862ea0 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59e73fd740 10 client.16458 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59e73fd740 10 client.16458 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59e73fd740 20 client.16458 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e58e23e0 0x55d1e58e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59e73fd740 4 client.16458 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1e57ca8e0 con 0x55d1e58e23e0 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59de800640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e58e23e0 0x55d1e58e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59de800640 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e58e23e0 0x55d1e58e4800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8f7acd4b50bbf7e server_cookie=53927bd29db5deab in_seq=0 out_seq=0 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.664+0000 7f59c3400640 10 client.16458 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59c3400640 1 -- 172.21.15.44:0/1655538412 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1e57ca8e0 con 0x55d1e58e23e0 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59c3400640 10 client.16458 handle_command_reply: tid=0 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 client.16458 shutdown 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 msgr2=0x55d1e586fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586fad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55d1e587e250 tx=0x7f59c8005750 comp rx=0 tx=0).stop 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 shutdown_connections 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1e586ba80 0x55d1e5870010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d1e58e23e0 0x55d1e58e4800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1e5862ea0 0x55d1e586fad0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 --2- 172.21.15.44:0/1655538412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1e5862580 0x55d1e586f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59c3400640 0 client.16458 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 >> 172.21.15.44:0/1655538412 conn(0x55d1e585ccb0 msgr2=0x55d1e5857960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 shutdown_connections 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 1 -- 172.21.15.44:0/1655538412 wait complete. 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.665+0000 7f59e73fd740 20 client.16458 trim_cache size 0 max 16384 2024-04-20T11:26:43.669 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:43.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f59c40a3d20 msgr2=0x7f59c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f59c40a3d20 0x7f59c40a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59d00f0090 tx=0x7f59d00f8550 comp rx=0 tx=0).stop 2024-04-20T11:26:43.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 msgr2=0x7f59e019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e019e5f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f59d0005ab0 tx=0x7f59d0004970 comp rx=0 tx=0).stop 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 shutdown_connections 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e010c440 0x7f59e0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59e0103940 0x7f59e019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f59c40a3d20 0x7f59c40a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 --2- 172.21.15.44:0/3187905593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e0103020 0x7f59e019e5f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 >> 172.21.15.44:0/3187905593 conn(0x7f59e00fc880 msgr2=0x7f59e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 shutdown_connections 2024-04-20T11:26:43.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.666+0000 7f59df200640 1 -- 172.21.15.44:0/3187905593 wait complete. 2024-04-20T11:26:43.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:43 smithi044 ceph-mon[16855]: pgmap v126: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:26:43.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3096974435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:43.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:26:43.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 -- 172.21.15.44:0/383323805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160102080 msgr2=0x7f4160102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 --2- 172.21.15.44:0/383323805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160102080 0x7f4160102460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4154009900 tx=0x7f415402bc40 comp rx=0 tx=0).stop 2024-04-20T11:26:43.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 -- 172.21.15.44:0/383323805 shutdown_connections 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 --2- 172.21.15.44:0/383323805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160107150 0x7f416010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 --2- 172.21.15.44:0/383323805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4160102a30 0x7f4160106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 --2- 172.21.15.44:0/383323805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160102080 0x7f4160102460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.900+0000 7f4166e00640 1 -- 172.21.15.44:0/383323805 >> 172.21.15.44:0/383323805 conn(0x7f41600fd900 msgr2=0x7f41600ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.901+0000 7f4166e00640 1 -- 172.21.15.44:0/383323805 shutdown_connections 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.901+0000 7f4166e00640 1 -- 172.21.15.44:0/383323805 wait complete. 2024-04-20T11:26:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.902+0000 7f4166e00640 1 Processor -- start 2024-04-20T11:26:43.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.902+0000 7f4166e00640 1 -- start start 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 0x7f4160108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4160107150 0x7f4160109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601097c0 0x7f41601aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4160112320 con 0x7f4160102080 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41601121a0 con 0x7f41601097c0 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41601124a0 con 0x7f4160107150 2024-04-20T11:26:43.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4165a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 0x7f4160108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4165a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 0x7f4160108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36290/0 (socket says 172.21.15.44:36290) 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4165a00640 1 -- 172.21.15.44:0/2784209734 learned_addr learned my addr 172.21.15.44:0/2784209734 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4165000640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4160107150 0x7f4160109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.903+0000 7f4166400640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601097c0 0x7f41601aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4165a00640 1 -- 172.21.15.44:0/2784209734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4160107150 msgr2=0x7f4160109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4165a00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4160107150 0x7f4160109280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4165a00640 1 -- 172.21.15.44:0/2784209734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601097c0 msgr2=0x7f41601aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4165a00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601097c0 0x7f41601aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4165a00640 1 -- 172.21.15.44:0/2784209734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41601ab530 con 0x7f4160102080 2024-04-20T11:26:43.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4165a00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 0x7f4160108d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f415402dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f415e000640 1 -- 172.21.15.44:0/2784209734 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154043020 con 0x7f4160102080 2024-04-20T11:26:43.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f415e000640 1 -- 172.21.15.44:0/2784209734 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4154004310 con 0x7f4160102080 2024-04-20T11:26:43.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41601ab7c0 con 0x7f4160102080 2024-04-20T11:26:43.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.904+0000 7f415e000640 1 -- 172.21.15.44:0/2784209734 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154003be0 con 0x7f4160102080 2024-04-20T11:26:43.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.905+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601abce0 con 0x7f4160102080 2024-04-20T11:26:43.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.908+0000 7f416b02c740 1 Processor -- start 2024-04-20T11:26:43.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.908+0000 7f416b02c740 1 -- start start 2024-04-20T11:26:43.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.908+0000 7f415e000640 1 -- 172.21.15.44:0/2784209734 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4154046020 con 0x7f4160102080 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.908+0000 7f416b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f416b02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9afa90 0x55f17a9b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f416b02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 0x55f17a9baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f416b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f17a914050 con 0x55f17a9af170 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f416b02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f17a8ba490 con 0x55f17a9b8670 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f416b02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f17a910010 con 0x55f17a9afa90 2024-04-20T11:26:43.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4166400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 0x55f17a9baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9afa90 0x55f17a9b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 0x55f17a9baa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32860/0 (socket says 172.21.15.44:32860) 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 -- 172.21.15.44:0/4114412012 learned_addr learned my addr 172.21.15.44:0/4114412012 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 -- 172.21.15.44:0/4114412012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9afa90 msgr2=0x55f17a9b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9afa90 0x55f17a9b8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 -- 172.21.15.44:0/4114412012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 msgr2=0x55f17a9af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.909+0000 7f4165000640 1 -- 172.21.15.44:0/4114412012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17a8c2070 con 0x55f17a9b8670 2024-04-20T11:26:43.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.910+0000 7f4165000640 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 0x55f17a9baa60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f414c00bc30 tx=0x7f414c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a61e0c0b670b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.910+0000 7f4152a00640 1 -- 172.21.15.44:0/4114412012 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c01c480 con 0x55f17a9b8670 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.910+0000 7f415e000640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f41480a3d20 0x7f41480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.910+0000 7f4152a00640 1 -- 172.21.15.44:0/4114412012 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f414c01cb20 con 0x55f17a9b8670 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.910+0000 7f4152a00640 1 -- 172.21.15.44:0/4114412012 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c011cd0 con 0x55f17a9b8670 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.910+0000 7f415e000640 1 -- 172.21.15.44:0/2784209734 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f41540f26e0 con 0x7f4160102080 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f4166400640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f41480a3d20 0x7f41480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 -- 172.21.15.44:0/4114412012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 msgr2=0x55f17a9baa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 0x55f17a9baa60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f414c00bc30 tx=0x7f414c0107a0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 -- 172.21.15.44:0/4114412012 shutdown_connections 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9b8670 0x55f17a9baa60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9afa90 0x55f17a9b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 --2- 172.21.15.44:0/4114412012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 -- 172.21.15.44:0/4114412012 >> 172.21.15.44:0/4114412012 conn(0x55f17a9a98a0 msgr2=0x55f17a9a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f4166400640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f41480a3d20 0x7f41480a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4158005fd0 tx=0x7f4158009600 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 -- 172.21.15.44:0/4114412012 shutdown_connections 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.911+0000 7f416b02c740 1 -- 172.21.15.44:0/4114412012 wait complete. 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.912+0000 7f416b02c740 1 -- start start 2024-04-20T11:26:43.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.912+0000 7f416b02c740 1 Processor -- start 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9b8670 0x55f17a9b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f17a9bb460 con 0x55f17a9af170 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f17a8cd860 con 0x55f17a9afa90 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f17a9bb5e0 con 0x55f17a9b8670 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f416b02c740 10 client.0 _ll_register_callbacks cb 0x55f17a89bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9b5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36304/0 (socket says 172.21.15.44:36304) 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 -- 172.21.15.44:0/4183198577 learned_addr learned my addr 172.21.15.44:0/4183198577 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4166400640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9b8670 0x55f17a9b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 -- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9b8670 msgr2=0x55f17a9b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9b8670 0x55f17a9b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 -- 172.21.15.44:0/4183198577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 msgr2=0x55f17a9b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165a00640 1 -- 172.21.15.44:0/4183198577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17a8d1950 con 0x55f17a9af170 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4166400640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9b8670 0x55f17a9b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.913+0000 7f4165000640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f4165a00640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9b5650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f415402dfd0 tx=0x7f4154005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c243707faa131b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f4147400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f4147400640 1 -- 172.21.15.44:0/4183198577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41540be2e0 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f4147400640 1 -- 172.21.15.44:0/4183198577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f41540f00c0 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414c01cb20 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f4147400640 1 -- 172.21.15.44:0/4183198577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f41540be100 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f416b02c740 10 client.16470 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f17a89f820 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.914+0000 7f4147400640 1 -- 172.21.15.44:0/4183198577 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f414c01cb20 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f4147400640 1 -- 172.21.15.44:0/4183198577 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f41540c6f80 con 0x55f17a9af170 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f416b02c740 10 client.16470 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f416b02c740 10 client.16470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f416b02c740 20 client.16470 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f416b02c740 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17a9c8590 0x55f17a9caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f416b02c740 4 client.16470 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f17a921a70 con 0x55f17a9c8590 2024-04-20T11:26:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.915+0000 7f4165000640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17a9c8590 0x55f17a9caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.916+0000 7f4165000640 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17a9c8590 0x55f17a9caa30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7cdfad968eb8f48 server_cookie=6f57e9ecac1f5e51 in_seq=0 out_seq=0 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.916+0000 7f4147400640 10 client.16470 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f4147400640 1 -- 172.21.15.44:0/4183198577 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f17a921a70 con 0x55f17a9c8590 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f4147400640 10 client.16470 handle_command_reply: tid=0 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 client.16470 shutdown 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 msgr2=0x55f17a9b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9b5650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f415402dfd0 tx=0x7f4154005c00 comp rx=0 tx=0).stop 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 shutdown_connections 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17a9c8590 0x55f17a9caa30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9b8670 0x55f17a9b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 --2- 172.21.15.44:0/4183198577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9af170 0x55f17a9b5650 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f4147400640 0 client.16470 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.917+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 >> 172.21.15.44:0/4183198577 conn(0x55f17a9a98a0 msgr2=0x55f17a9ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.918+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 shutdown_connections 2024-04-20T11:26:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.918+0000 7f416b02c740 1 -- 172.21.15.44:0/4183198577 wait complete. 2024-04-20T11:26:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.918+0000 7f416b02c740 20 client.16470 trim_cache size 0 max 16384 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 Processor -- start 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 -- start start 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b3ff0 con 0x55f17a9b8670 2024-04-20T11:26:43.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b5d10 con 0x55f17a9afa90 2024-04-20T11:26:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.933+0000 7f416b02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b46c0 con 0x55f17a9af170 2024-04-20T11:26:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36316/0 (socket says 172.21.15.44:36316) 2024-04-20T11:26:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 -- 172.21.15.44:0/3200189498 learned_addr learned my addr 172.21.15.44:0/3200189498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4165000640 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4165a00640 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 -- 172.21.15.44:0/3200189498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 msgr2=0x55f17a9b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 -- 172.21.15.44:0/3200189498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 msgr2=0x55f17a9c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4166400640 1 -- 172.21.15.44:0/3200189498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17a6263f0 con 0x55f17a9b8670 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.934+0000 7f4165000640 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.935+0000 7f4166400640 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9a87e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f41580092f0 tx=0x7f4158016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39546dd0c69a6f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.935+0000 7f4152000640 1 -- 172.21.15.44:0/3200189498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580219b0 con 0x55f17a9b8670 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.935+0000 7f4152000640 1 -- 172.21.15.44:0/3200189498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4158021b50 con 0x55f17a9b8670 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.935+0000 7f4152000640 1 -- 172.21.15.44:0/3200189498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158029640 con 0x55f17a9b8670 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 -- 172.21.15.44:0/3200189498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 msgr2=0x55f17a9a87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9a87e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f41580092f0 tx=0x7f4158016000 comp rx=0 tx=0).stop 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 -- 172.21.15.44:0/3200189498 shutdown_connections 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9a87e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 --2- 172.21.15.44:0/3200189498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 -- 172.21.15.44:0/3200189498 >> 172.21.15.44:0/3200189498 conn(0x55f17a9a98a0 msgr2=0x55f17a9a80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 -- 172.21.15.44:0/3200189498 shutdown_connections 2024-04-20T11:26:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 -- 172.21.15.44:0/3200189498 wait complete. 2024-04-20T11:26:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 -- start start 2024-04-20T11:26:43.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.936+0000 7f416b02c740 1 Processor -- start 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b5580 con 0x55f17a9b8670 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b4e20 con 0x55f17a9afa90 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b1f30 con 0x55f17a9af170 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f416b02c740 10 client.0 _ll_register_callbacks cb 0x7f4154005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f4165000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f4166400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f4165000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32882/0 (socket says 172.21.15.44:32882) 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f4165000640 1 -- 172.21.15.44:0/2502674055 learned_addr learned my addr 172.21.15.44:0/2502674055 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.937+0000 7f4165a00640 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4165000640 1 -- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 msgr2=0x55f17a9c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4165000640 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4165000640 1 -- 172.21.15.44:0/2502674055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 msgr2=0x55f17a9b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4165000640 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4165000640 1 -- 172.21.15.44:0/2502674055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17a6f07d0 con 0x55f17a9afa90 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4165000640 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b20f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f414c010ca0 tx=0x7f414c011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51b64863e5f5de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4147400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4147400640 1 -- 172.21.15.44:0/2502674055 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c0118e0 con 0x55f17a9afa90 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4158021b50 con 0x55f17a9afa90 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.938+0000 7f4147400640 1 -- 172.21.15.44:0/2502674055 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f414c011a80 con 0x55f17a9afa90 2024-04-20T11:26:43.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f4147400640 1 -- 172.21.15.44:0/2502674055 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f414c011ca0 con 0x55f17a9afa90 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 10 client.26609 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f17aa2eaf0 con 0x55f17a9afa90 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f4147400640 1 -- 172.21.15.44:0/2502674055 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4158021b50 con 0x55f17a9afa90 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f4147400640 1 -- 172.21.15.44:0/2502674055 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f414c03b050 con 0x55f17a9afa90 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 10 client.26609 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 10 client.26609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 20 client.26609 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17aa2f160 0x55f17aa31580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 4 client.26609 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f17a91b8e0 con 0x55f17aa2f160 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.939+0000 7f4165a00640 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17aa2f160 0x55f17aa31580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f4165a00640 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17aa2f160 0x55f17aa31580 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b53c96e7e4bf1e28 server_cookie=20ac9515422d7be2 in_seq=0 out_seq=0 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f4147400640 10 client.26609 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:43.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f4147400640 1 -- 172.21.15.44:0/2502674055 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f17a91b8e0 con 0x55f17aa2f160 2024-04-20T11:26:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f4147400640 10 client.26609 handle_command_reply: tid=0 2024-04-20T11:26:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 client.26609 shutdown 2024-04-20T11:26:43.943 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:43 smithi078 ceph-mon[21464]: pgmap v126: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:26:43.943 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3096974435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 msgr2=0x55f17a9b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b20f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f414c010ca0 tx=0x7f414c011870 comp rx=0 tx=0).stop 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 shutdown_connections 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f17aa2f160 0x55f17aa31580 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f17a9b8670 0x55f17a9b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f4147400640 0 client.26609 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:26:43.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f17a9afa90 0x55f17a9b20f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.940+0000 7f416b02c740 1 --2- 172.21.15.44:0/2502674055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f17a9af170 0x55f17a9c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 >> 172.21.15.44:0/2502674055 conn(0x55f17a9a98a0 msgr2=0x55f17a9b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 shutdown_connections 2024-04-20T11:26:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f416b02c740 1 -- 172.21.15.44:0/2502674055 wait complete. 2024-04-20T11:26:43.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f416b02c740 20 client.26609 trim_cache size 0 max 16384 2024-04-20T11:26:43.945 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f41480a3d20 msgr2=0x7f41480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f4166e00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f41480a3d20 0x7f41480a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4158005fd0 tx=0x7f4158009600 comp rx=0 tx=0).stop 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 msgr2=0x7f4160108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.941+0000 7f4166e00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 0x7f4160108d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f415402dde0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 shutdown_connections 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601097c0 0x7f41601aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4160107150 0x7f4160109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f41480a3d20 0x7f41480a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 --2- 172.21.15.44:0/2784209734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4160102080 0x7f4160108d40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 >> 172.21.15.44:0/2784209734 conn(0x7f41600fd900 msgr2=0x7f41600fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 shutdown_connections 2024-04-20T11:26:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:43.942+0000 7f4166e00640 1 -- 172.21.15.44:0/2784209734 wait complete. 2024-04-20T11:26:43.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:26:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:43 smithi165 ceph-mon[21975]: pgmap v126: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:26:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3096974435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:26:44.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 -- 172.21.15.44:0/3405507001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 msgr2=0x7f8a2c104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 --2- 172.21.15.44:0/3405507001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c104660 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009900 tx=0x7f8a1c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:26:44.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 -- 172.21.15.44:0/3405507001 shutdown_connections 2024-04-20T11:26:44.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 --2- 172.21.15.44:0/3405507001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a2c109350 0x7f8a2c110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 --2- 172.21.15.44:0/3405507001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c104c30 0x7f8a2c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 --2- 172.21.15.44:0/3405507001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c104660 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 -- 172.21.15.44:0/3405507001 >> 172.21.15.44:0/3405507001 conn(0x7f8a2c0ffaa0 msgr2=0x7f8a2c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 -- 172.21.15.44:0/3405507001 shutdown_connections 2024-04-20T11:26:44.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.176+0000 7f8a2b200640 1 -- 172.21.15.44:0/3405507001 wait complete. 2024-04-20T11:26:44.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.177+0000 7f8a2b200640 1 Processor -- start 2024-04-20T11:26:44.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 -- start start 2024-04-20T11:26:44.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 0x7f8a2c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a2c19a830 0x7f8a2c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1144f0 con 0x7f8a2c109350 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c114370 con 0x7f8a2c104280 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c114670 con 0x7f8a2c19a830 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a29400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 0x7f8a2c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a29400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 0x7f8a2c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36348/0 (socket says 172.21.15.44:36348) 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a29400640 1 -- 172.21.15.44:0/942335346 learned_addr learned my addr 172.21.15.44:0/942335346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a29e00640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.178+0000 7f8a2a800640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a2c19a830 0x7f8a2c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29e00640 1 -- 172.21.15.44:0/942335346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a2c19a830 msgr2=0x7f8a2c19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29e00640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a2c19a830 0x7f8a2c19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29e00640 1 -- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 msgr2=0x7f8a2c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29e00640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 0x7f8a2c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29e00640 1 -- 172.21.15.44:0/942335346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c19b3c0 con 0x7f8a2c104280 2024-04-20T11:26:44.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29400640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 0x7f8a2c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a29e00640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c1a06c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009230 tx=0x7f8a1c02dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a22a00640 1 -- 172.21.15.44:0/942335346 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c043020 con 0x7f8a2c104280 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a22a00640 1 -- 172.21.15.44:0/942335346 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a1c02de50 con 0x7f8a2c104280 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a22a00640 1 -- 172.21.15.44:0/942335346 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c003a10 con 0x7f8a2c104280 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.179+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c19b650 con 0x7f8a2c104280 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.180+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c1a7830 con 0x7f8a2c104280 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 Processor -- start 2024-04-20T11:26:44.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 -- start start 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a22a00640 1 -- 172.21.15.44:0/942335346 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8a1c046020 con 0x7f8a2c104280 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f84b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f854130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f856a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56532f7b0050 con 0x56532f84ba90 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56532f756490 con 0x56532f84b170 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.183+0000 7f8a33e77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56532f7ac010 con 0x56532f854670 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f854130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f854130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36356/0 (socket says 172.21.15.44:36356) 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29e00640 1 -- 172.21.15.44:0/2465754908 learned_addr learned my addr 172.21.15.44:0/2465754908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a2a800640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f84b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29400640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f856a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29400640 1 -- 172.21.15.44:0/2465754908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 msgr2=0x56532f84b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29400640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f84b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29400640 1 -- 172.21.15.44:0/2465754908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 msgr2=0x56532f854130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29400640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f854130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29400640 1 -- 172.21.15.44:0/2465754908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532f75e070 con 0x56532f854670 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a2a800640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f84b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.184+0000 7f8a29e00640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f854130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a29400640 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f856a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c19bb80 tx=0x7f8a18011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f8c2bf2f91058b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a22a00640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8a100a3d20 0x7f8a100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a17400640 1 -- 172.21.15.44:0/2465754908 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18004020 con 0x56532f854670 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a17400640 1 -- 172.21.15.44:0/2465754908 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a18005520 con 0x56532f854670 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a17400640 1 -- 172.21.15.44:0/2465754908 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a180035f0 con 0x56532f854670 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a22a00640 1 -- 172.21.15.44:0/942335346 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8a1c0f2510 con 0x7f8a2c104280 2024-04-20T11:26:44.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.185+0000 7f8a2a800640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8a100a3d20 0x7f8a100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 -- 172.21.15.44:0/2465754908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 msgr2=0x56532f856a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f856a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c19bb80 tx=0x7f8a18011950 comp rx=0 tx=0).stop 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 -- 172.21.15.44:0/2465754908 shutdown_connections 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f856a60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f854130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2465754908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f84b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 -- 172.21.15.44:0/2465754908 >> 172.21.15.44:0/2465754908 conn(0x56532f8458a0 msgr2=0x56532f845c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 -- 172.21.15.44:0/2465754908 shutdown_connections 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a2a800640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8a100a3d20 0x7f8a100a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009fd0 tx=0x7f8a24009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.186+0000 7f8a33e77740 1 -- 172.21.15.44:0/2465754908 wait complete. 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 -- start start 2024-04-20T11:26:44.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 Processor -- start 2024-04-20T11:26:44.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 0x56532f851630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84ba90 0x56532f851b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f854670 0x56532f8520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56532f857460 con 0x56532f84b170 2024-04-20T11:26:44.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56532f769860 con 0x56532f854670 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56532f8575e0 con 0x56532f84ba90 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.187+0000 7f8a33e77740 10 client.0 _ll_register_callbacks cb 0x56532f737fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 0x56532f851630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84ba90 0x56532f851b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 0x56532f851630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36368/0 (socket says 172.21.15.44:36368) 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29e00640 1 -- 172.21.15.44:0/3017478671 learned_addr learned my addr 172.21.15.44:0/3017478671 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a2a800640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f854670 0x56532f8520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 -- 172.21.15.44:0/3017478671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f854670 msgr2=0x56532f8520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f854670 0x56532f8520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 -- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 msgr2=0x56532f851630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 0x56532f851630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 -- 172.21.15.44:0/3017478671 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532f76d950 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29e00640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 0x56532f851630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a29400640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84ba90 0x56532f851b70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a1801b040 tx=0x7f8a18005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02dffa93722c7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.188+0000 7f8a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.189+0000 7f8a0be00640 1 -- 172.21.15.44:0/3017478671 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18028020 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.189+0000 7f8a0be00640 1 -- 172.21.15.44:0/3017478671 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8a1800f750 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.189+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a18005520 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.189+0000 7f8a0be00640 1 -- 172.21.15.44:0/3017478671 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a18033050 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.189+0000 7f8a33e77740 10 client.26614 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.189+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56532f73b820 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a0be00640 1 -- 172.21.15.44:0/3017478671 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a18005520 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a0be00640 1 -- 172.21.15.44:0/3017478671 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f8a180247e0 con 0x56532f84ba90 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a33e77740 10 client.26614 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a33e77740 10 client.26614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:26:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a33e77740 20 client.26614 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a33e77740 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f864610 0x56532f866a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a33e77740 4 client.26614 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532f7bda70 con 0x56532f864610 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.190+0000 7f8a29e00640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f864610 0x56532f866a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.191+0000 7f8a29e00640 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f864610 0x56532f866a00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eab6e572f04bef23 server_cookie=3e43395065c385ef in_seq=0 out_seq=0 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.191+0000 7f8a0be00640 10 client.26614 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a0be00640 1 -- 172.21.15.44:0/3017478671 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56532f7bda70 con 0x56532f864610 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a0be00640 10 client.26614 handle_command_reply: tid=0 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 client.26614 shutdown 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84ba90 msgr2=0x56532f851b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84ba90 0x56532f851b70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a1801b040 tx=0x7f8a18005300 comp rx=0 tx=0).stop 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 shutdown_connections 2024-04-20T11:26:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f864610 0x56532f866a00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f854670 0x56532f8520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84ba90 0x56532f851b70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 --2- 172.21.15.44:0/3017478671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84b170 0x56532f851630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a0be00640 0 client.26614 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 >> 172.21.15.44:0/3017478671 conn(0x56532f8458a0 msgr2=0x56532f8475f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 shutdown_connections 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 1 -- 172.21.15.44:0/3017478671 wait complete. 2024-04-20T11:26:44.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.192+0000 7f8a33e77740 20 client.26614 trim_cache size 0 max 16384 2024-04-20T11:26:44.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.207+0000 7f8a33e77740 1 Processor -- start 2024-04-20T11:26:44.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.207+0000 7f8a33e77740 1 -- start start 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f860350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f86c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f86c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a33e77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1afd00 con 0x56532f84ba90 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a33e77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b1a20 con 0x56532f84b170 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a33e77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b03d0 con 0x56532f854670 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f86c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f86c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36374/0 (socket says 172.21.15.44:36374) 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 -- 172.21.15.44:0/2423131243 learned_addr learned my addr 172.21.15.44:0/2423131243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29e00640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f860350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a2a800640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f86c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 -- 172.21.15.44:0/2423131243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 msgr2=0x56532f86c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f86c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 -- 172.21.15.44:0/2423131243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 msgr2=0x56532f860350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f860350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29400640 1 -- 172.21.15.44:0/2423131243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532f4c23f0 con 0x56532f84ba90 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a29e00640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f860350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.208+0000 7f8a2a800640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f86c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:44.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.209+0000 7f8a29400640 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f86c2a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a18024be0 tx=0x7f8a18012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35321d3805a5a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.209+0000 7f8a16a00640 1 -- 172.21.15.44:0/2423131243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18028070 con 0x56532f84ba90 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.209+0000 7f8a16a00640 1 -- 172.21.15.44:0/2423131243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a18016030 con 0x56532f84ba90 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.209+0000 7f8a16a00640 1 -- 172.21.15.44:0/2423131243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1801d070 con 0x56532f84ba90 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 -- 172.21.15.44:0/2423131243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 msgr2=0x56532f86c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f86c2a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a18024be0 tx=0x7f8a18012f10 comp rx=0 tx=0).stop 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 -- 172.21.15.44:0/2423131243 shutdown_connections 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f854670 0x56532f86c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f84ba90 0x56532f86c2a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 --2- 172.21.15.44:0/2423131243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84b170 0x56532f860350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 -- 172.21.15.44:0/2423131243 >> 172.21.15.44:0/2423131243 conn(0x56532f8458a0 msgr2=0x56532f84c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 -- 172.21.15.44:0/2423131243 shutdown_connections 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 -- 172.21.15.44:0/2423131243 wait complete. 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 -- start start 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.210+0000 7f8a33e77740 1 Processor -- start 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84b170 0x56532f85cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84ba90 0x56532f84e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 0x56532f84e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b1290 con 0x56532f854670 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1b0b30 con 0x56532f84ba90 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c1adc60 con 0x56532f84b170 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a33e77740 10 client.0 _ll_register_callbacks cb 0x7f8a18009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a2a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 0x56532f84e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a2a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 0x56532f84e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36382/0 (socket says 172.21.15.44:36382) 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a2a800640 1 -- 172.21.15.44:0/4064227345 learned_addr learned my addr 172.21.15.44:0/4064227345 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a29e00640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84b170 0x56532f85cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.211+0000 7f8a29400640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84ba90 0x56532f84e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a2a800640 1 -- 172.21.15.44:0/4064227345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84b170 msgr2=0x56532f85cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a2a800640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84b170 0x56532f85cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a2a800640 1 -- 172.21.15.44:0/4064227345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84ba90 msgr2=0x56532f84e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a2a800640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84ba90 0x56532f84e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a2a800640 1 -- 172.21.15.44:0/4064227345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532f58c7d0 con 0x56532f854670 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a2a800640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 0x56532f84e570 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a24003bb0 tx=0x7f8a24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb39a3b93d745f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a0be00640 1 -- 172.21.15.44:0/4064227345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24003e20 con 0x56532f854670 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a0be00640 1 -- 172.21.15.44:0/4064227345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8a240047c0 con 0x56532f854670 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.212+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a18016030 con 0x56532f854670 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a0be00640 1 -- 172.21.15.44:0/4064227345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a2400f610 con 0x56532f854670 2024-04-20T11:26:44.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 10 client.16494 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56532f8ca9c0 con 0x56532f854670 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a0be00640 1 -- 172.21.15.44:0/4064227345 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a18016030 con 0x56532f854670 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a0be00640 1 -- 172.21.15.44:0/4064227345 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f8a24032030 con 0x56532f854670 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 10 client.16494 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 10 client.16494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 20 client.16494 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f8cb220 0x56532f8cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 4 client.16494 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.213+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56532f7b78e0 con 0x56532f8cb220 2024-04-20T11:26:44.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.214+0000 7f8a29e00640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f8cb220 0x56532f8cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.214+0000 7f8a29e00640 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f8cb220 0x56532f8cd640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edd17e11f2156d48 server_cookie=a9cc9b3dc89813a3 in_seq=0 out_seq=0 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.214+0000 7f8a0be00640 10 client.16494 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.214+0000 7f8a0be00640 1 -- 172.21.15.44:0/4064227345 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56532f7b78e0 con 0x56532f8cb220 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.214+0000 7f8a0be00640 10 client.16494 handle_command_reply: tid=0 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.214+0000 7f8a33e77740 1 client.16494 shutdown 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 msgr2=0x56532f84e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 0x56532f84e570 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a24003bb0 tx=0x7f8a24004a40 comp rx=0 tx=0).stop 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 shutdown_connections 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56532f8cb220 0x56532f8cd640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532f854670 0x56532f84e570 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56532f84ba90 0x56532f84e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 --2- 172.21.15.44:0/4064227345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532f84b170 0x56532f85cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 >> 172.21.15.44:0/4064227345 conn(0x56532f8458a0 msgr2=0x56532f852c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a0be00640 0 client.16494 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:26:44.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 shutdown_connections 2024-04-20T11:26:44.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 1 -- 172.21.15.44:0/4064227345 wait complete. 2024-04-20T11:26:44.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.215+0000 7f8a33e77740 20 client.16494 trim_cache size 0 max 16384 2024-04-20T11:26:44.219 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8a100a3d20 msgr2=0x7f8a100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8a100a3d20 0x7f8a100a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009fd0 tx=0x7f8a24009500 comp rx=0 tx=0).stop 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 msgr2=0x7f8a2c1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c1a06c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009230 tx=0x7f8a1c02dc60 comp rx=0 tx=0).stop 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 shutdown_connections 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a2c19a830 0x7f8a2c19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8a100a3d20 0x7f8a100a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a2c109350 0x7f8a2c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 --2- 172.21.15.44:0/942335346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a2c104280 0x7f8a2c1a06c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 >> 172.21.15.44:0/942335346 conn(0x7f8a2c0ffaa0 msgr2=0x7f8a2c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 shutdown_connections 2024-04-20T11:26:44.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.216+0000 7f8a2b200640 1 -- 172.21.15.44:0/942335346 wait complete. 2024-04-20T11:26:44.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:26:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.450+0000 7f0649200640 1 -- 172.21.15.44:0/3585088675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f064410d3d0 msgr2=0x7f064410d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.450+0000 7f0649200640 1 --2- 172.21.15.44:0/3585088675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f064410d3d0 0x7f064410d7b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0634009900 tx=0x7f063402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 -- 172.21.15.44:0/3585088675 shutdown_connections 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 --2- 172.21.15.44:0/3585088675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108ea0 0x7f06441152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 --2- 172.21.15.44:0/3585088675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0644108500 0x7f0644108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 --2- 172.21.15.44:0/3585088675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f064410d3d0 0x7f064410d7b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 -- 172.21.15.44:0/3585088675 >> 172.21.15.44:0/3585088675 conn(0x7f0644077d80 msgr2=0x7f0644106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 -- 172.21.15.44:0/3585088675 shutdown_connections 2024-04-20T11:26:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.451+0000 7f0649200640 1 -- 172.21.15.44:0/3585088675 wait complete. 2024-04-20T11:26:44.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 Processor -- start 2024-04-20T11:26:44.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 -- start start 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108500 0x7f064419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 0x7f064419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064410d3d0 0x7f06441a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0644118950 con 0x7f0644108500 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06441187d0 con 0x7f064410d3d0 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.452+0000 7f0649200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0644118ad0 con 0x7f0644108ea0 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 0x7f064419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 0x7f064419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54384/0 (socket says 172.21.15.44:54384) 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 -- 172.21.15.44:0/2848201530 learned_addr learned my addr 172.21.15.44:0/2848201530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f0643400640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108500 0x7f064419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f0643e00640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064410d3d0 0x7f06441a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 -- 172.21.15.44:0/2848201530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064410d3d0 msgr2=0x7f06441a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064410d3d0 0x7f06441a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 -- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108500 msgr2=0x7f064419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108500 0x7f064419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 -- 172.21.15.44:0/2848201530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06441a3e30 con 0x7f0644108ea0 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f0643e00640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064410d3d0 0x7f06441a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f0643400640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108500 0x7f064419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.453+0000 7f063be00640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 0x7f064419f3d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e9f0 tx=0x7f062c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.454+0000 7f0640c00640 1 -- 172.21.15.44:0/2848201530 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c00cd80 con 0x7f0644108ea0 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.454+0000 7f0640c00640 1 -- 172.21.15.44:0/2848201530 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f062c005520 con 0x7f0644108ea0 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.454+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06441a4120 con 0x7f0644108ea0 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.454+0000 7f0640c00640 1 -- 172.21.15.44:0/2848201530 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c010690 con 0x7f0644108ea0 2024-04-20T11:26:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.454+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06441ab9b0 con 0x7f0644108ea0 2024-04-20T11:26:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.457+0000 7f064d608740 1 Processor -- start 2024-04-20T11:26:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f0640c00640 1 -- 172.21.15.44:0/2848201530 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f062c0046c0 con 0x7f0644108ea0 2024-04-20T11:26:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 -- start start 2024-04-20T11:26:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 0x559203f04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f0d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f0d670 0x559203f0fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559203e69050 con 0x559203f04170 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559203e0f490 con 0x559203f0d670 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f064d608740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559203e65010 con 0x559203f04a90 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 0x559203f04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 0x559203f04550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36418/0 (socket says 172.21.15.44:36418) 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.458+0000 7f0643e00640 1 -- 172.21.15.44:0/4242148499 learned_addr learned my addr 172.21.15.44:0/4242148499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643400640 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f063be00640 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f0d670 0x559203f0fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643e00640 1 -- 172.21.15.44:0/4242148499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 msgr2=0x559203f0d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643e00640 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f0d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643e00640 1 -- 172.21.15.44:0/4242148499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f0d670 msgr2=0x559203f0fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643e00640 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f0d670 0x559203f0fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643e00640 1 -- 172.21.15.44:0/4242148499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559203e17070 con 0x559203f04170 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0643e00640 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 0x559203f04550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f063c009f80 tx=0x7f063c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6664c9592c2c7000 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.459+0000 7f0640c00640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06280a3d20 0x7f06280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f0640c00640 1 -- 172.21.15.44:0/2848201530 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f062c0d0c50 con 0x7f0644108ea0 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f0638c00640 1 -- 172.21.15.44:0/4242148499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c012710 con 0x559203f04170 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f0638c00640 1 -- 172.21.15.44:0/4242148499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f063c012db0 con 0x559203f04170 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f0643400640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06280a3d20 0x7f06280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f0638c00640 1 -- 172.21.15.44:0/4242148499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c01a3f0 con 0x559203f04170 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 -- 172.21.15.44:0/4242148499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 msgr2=0x559203f04550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 0x559203f04550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f063c009f80 tx=0x7f063c011870 comp rx=0 tx=0).stop 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 -- 172.21.15.44:0/4242148499 shutdown_connections 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f0d670 0x559203f0fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f0d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 --2- 172.21.15.44:0/4242148499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f04170 0x559203f04550 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f0643400640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06280a3d20 0x7f06280a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0634006010 tx=0x7f0634005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.460+0000 7f064d608740 1 -- 172.21.15.44:0/4242148499 >> 172.21.15.44:0/4242148499 conn(0x559203efe8a0 msgr2=0x559203efec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.461+0000 7f064d608740 1 -- 172.21.15.44:0/4242148499 shutdown_connections 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.461+0000 7f064d608740 1 -- 172.21.15.44:0/4242148499 wait complete. 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.461+0000 7f064d608740 1 -- start start 2024-04-20T11:26:44.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.461+0000 7f064d608740 1 Processor -- start 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04170 0x559203f0a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04a90 0x559203f0ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f0b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559203f10460 con 0x559203f0d670 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559203e22860 con 0x559203f04a90 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559203f105e0 con 0x559203f04170 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f064d608740 10 client.0 _ll_register_callbacks cb 0x559203df0fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f0b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f063be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04a90 0x559203f0ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f0b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36432/0 (socket says 172.21.15.44:36432) 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f0643e00640 1 -- 172.21.15.44:0/159800800 learned_addr learned my addr 172.21.15.44:0/159800800 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.462+0000 7f0643400640 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04170 0x559203f0a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f063be00640 1 -- 172.21.15.44:0/159800800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04170 msgr2=0x559203f0a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f063be00640 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04170 0x559203f0a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f063be00640 1 -- 172.21.15.44:0/159800800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 msgr2=0x559203f0b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f063be00640 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f0b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f063be00640 1 -- 172.21.15.44:0/159800800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559203e26950 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f063be00640 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04a90 0x559203f0ac20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f062c0ce3e0 tx=0x7f062c098fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f82aff4fbd19dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f0631600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f0631600640 1 -- 172.21.15.44:0/159800800 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c09d0c0 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f0631600640 1 -- 172.21.15.44:0/159800800 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f062c0a5440 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f063c012db0 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f0631600640 1 -- 172.21.15.44:0/159800800 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f062c09d360 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f064d608740 10 client.26639 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.463+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559203df4820 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f0631600640 1 -- 172.21.15.44:0/159800800 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f063c012db0 con 0x559203f04a90 2024-04-20T11:26:44.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f0631600640 1 -- 172.21.15.44:0/159800800 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f062c0ceb60 con 0x559203f04a90 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f064d608740 10 client.26639 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f064d608740 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f064d608740 20 client.26639 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f064d608740 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f1d7b0 0x559203f1fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f064d608740 4 client.26639 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559203e76a70 con 0x559203f1d7b0 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.464+0000 7f0643400640 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f1d7b0 0x559203f1fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.465+0000 7f0643400640 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f1d7b0 0x559203f1fbd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4036a9c4891c70 server_cookie=fa743119daba944 in_seq=0 out_seq=0 2024-04-20T11:26:44.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.465+0000 7f0631600640 10 client.26639 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:44.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f0631600640 1 -- 172.21.15.44:0/159800800 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559203e76a70 con 0x559203f1d7b0 2024-04-20T11:26:44.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f0631600640 10 client.26639 handle_command_reply: tid=0 2024-04-20T11:26:44.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 client.26639 shutdown 2024-04-20T11:26:44.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04a90 msgr2=0x559203f0ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04a90 0x559203f0ac20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f062c0ce3e0 tx=0x7f062c098fc0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 shutdown_connections 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f1d7b0 0x559203f1fbd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f0b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04a90 0x559203f0ac20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 --2- 172.21.15.44:0/159800800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04170 0x559203f0a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 >> 172.21.15.44:0/159800800 conn(0x559203efe8a0 msgr2=0x559203f005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f0631600640 0 client.26639 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 shutdown_connections 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 1 -- 172.21.15.44:0/159800800 wait complete. 2024-04-20T11:26:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.466+0000 7f064d608740 20 client.26639 trim_cache size 0 max 16384 2024-04-20T11:26:44.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 Processor -- start 2024-04-20T11:26:44.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 -- start start 2024-04-20T11:26:44.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f0ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f04e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06441b40a0 con 0x559203f0d670 2024-04-20T11:26:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06441b5dc0 con 0x559203f04170 2024-04-20T11:26:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f064d608740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06441b4770 con 0x559203f04a90 2024-04-20T11:26:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f04e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f04e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36434/0 (socket says 172.21.15.44:36434) 2024-04-20T11:26:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f0643e00640 1 -- 172.21.15.44:0/2610091890 learned_addr learned my addr 172.21.15.44:0/2610091890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.482+0000 7f0643400640 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f0ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f063be00640 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643e00640 1 -- 172.21.15.44:0/2610091890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 msgr2=0x559203f19320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643e00640 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f19320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643e00640 1 -- 172.21.15.44:0/2610091890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 msgr2=0x559203f0ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643e00640 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f0ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643e00640 1 -- 172.21.15.44:0/2610091890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559203f09690 con 0x559203f0d670 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643400640 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f0ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0643e00640 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f04e70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f063c00fbf0 tx=0x7f063c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c13fdfc58ccd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.483+0000 7f0633e00640 1 -- 172.21.15.44:0/2610091890 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0099c0 con 0x559203f0d670 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f0633e00640 1 -- 172.21.15.44:0/2610091890 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f063c009b60 con 0x559203f0d670 2024-04-20T11:26:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f0633e00640 1 -- 172.21.15.44:0/2610091890 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c012970 con 0x559203f0d670 2024-04-20T11:26:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 -- 172.21.15.44:0/2610091890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 msgr2=0x559203f04e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f04e70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f063c00fbf0 tx=0x7f063c009f30 comp rx=0 tx=0).stop 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 -- 172.21.15.44:0/2610091890 shutdown_connections 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f04e70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f19320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 --2- 172.21.15.44:0/2610091890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f0ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 -- 172.21.15.44:0/2610091890 >> 172.21.15.44:0/2610091890 conn(0x559203efe8a0 msgr2=0x559203efd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 -- 172.21.15.44:0/2610091890 shutdown_connections 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.484+0000 7f064d608740 1 -- 172.21.15.44:0/2610091890 wait complete. 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.485+0000 7f064d608740 1 -- start start 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.485+0000 7f064d608740 1 Processor -- start 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f11980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f82d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06441b5630 con 0x559203f0d670 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06441b4ed0 con 0x559203f04170 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06441b2020 con 0x559203f04a90 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f064d608740 10 client.0 _ll_register_callbacks cb 0x7f062c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f82d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f82d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36448/0 (socket says 172.21.15.44:36448) 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 -- 172.21.15.44:0/1983456086 learned_addr learned my addr 172.21.15.44:0/1983456086 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643400640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f11980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f063be00640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 -- 172.21.15.44:0/1983456086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 msgr2=0x559203f11ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f11ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 -- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 msgr2=0x559203f11980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f11980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643e00640 1 -- 172.21.15.44:0/1983456086 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559203b7b3f0 con 0x559203f0d670 2024-04-20T11:26:44.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.486+0000 7f0643400640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f11980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:44.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f0643e00640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f82d50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f063c000f80 tx=0x7f063c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158a71da92481068 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:44.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f0631600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:44.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f0631600640 1 -- 172.21.15.44:0/1983456086 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c034070 con 0x559203f0d670 2024-04-20T11:26:44.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f063c009b60 con 0x559203f0d670 2024-04-20T11:26:44.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f0631600640 1 -- 172.21.15.44:0/1983456086 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f063c00f850 con 0x559203f0d670 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f0631600640 1 -- 172.21.15.44:0/1983456086 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f063c040050 con 0x559203f0d670 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f064d608740 10 client.16512 fetch_fsmap learned FSMap version 39 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.487+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559203c457d0 con 0x559203f0d670 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f0631600640 1 -- 172.21.15.44:0/1983456086 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f063c009b60 con 0x559203f0d670 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f0631600640 1 -- 172.21.15.44:0/1983456086 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8423+0+0 (secure 0 0 0) 0x7f063c0053d0 con 0x559203f0d670 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f064d608740 10 client.16512 fetch_fsmap finished waiting for FSMap version 39 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f064d608740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f064d608740 20 client.16512 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f064d608740 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f84660 0x559203f86a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f064d608740 4 client.16512 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559203e708e0 con 0x559203f84660 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.488+0000 7f0643400640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f84660 0x559203f86a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f0643400640 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f84660 0x559203f86a50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0b3a5de41fc846 server_cookie=6389dad1d0d42367 in_seq=0 out_seq=0 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f0631600640 10 client.16512 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f0631600640 1 -- 172.21.15.44:0/1983456086 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559203e708e0 con 0x559203f84660 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f0631600640 10 client.16512 handle_command_reply: tid=0 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 client.16512 shutdown 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 msgr2=0x559203f82d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f82d50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f063c000f80 tx=0x7f063c004d30 comp rx=0 tx=0).stop 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 shutdown_connections 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559203f84660 0x559203f86a50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559203f0d670 0x559203f82d50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559203f04a90 0x559203f11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 --2- 172.21.15.44:0/1983456086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559203f04170 0x559203f11980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f0631600640 0 client.16512 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.489+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 >> 172.21.15.44:0/1983456086 conn(0x559203efe8a0 msgr2=0x559203efd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 shutdown_connections 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f064d608740 1 -- 172.21.15.44:0/1983456086 wait complete. 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f064d608740 20 client.16512 trim_cache size 0 max 16384 2024-04-20T11:26:44.493 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06280a3d20 msgr2=0x7f06280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f0649200640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06280a3d20 0x7f06280a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0634006010 tx=0x7f0634005e40 comp rx=0 tx=0).stop 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 msgr2=0x7f064419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.490+0000 7f0649200640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 0x7f064419f3d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e9f0 tx=0x7f062c00edf0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 shutdown_connections 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f064410d3d0 0x7f06441a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0644108ea0 0x7f064419f3d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f06280a3d20 0x7f06280a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 --2- 172.21.15.44:0/2848201530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0644108500 0x7f064419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 >> 172.21.15.44:0/2848201530 conn(0x7f0644077d80 msgr2=0x7f06440783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 shutdown_connections 2024-04-20T11:26:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:44.491+0000 7f0649200640 1 -- 172.21.15.44:0/2848201530 wait complete. 2024-04-20T11:26:45.509 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:26:45.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:26:45.510 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:26:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 -- 172.21.15.44:0/3479408428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 msgr2=0x7f324c074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 --2- 172.21.15.44:0/3479408428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c074710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f323c009ca0 tx=0x7f323c02c190 comp rx=0 tx=0).stop 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 -- 172.21.15.44:0/3479408428 shutdown_connections 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 --2- 172.21.15.44:0/3479408428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f324c10e6a0 0x7f324c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 --2- 172.21.15.44:0/3479408428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c074c50 0x7f324c10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 --2- 172.21.15.44:0/3479408428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c074710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 -- 172.21.15.44:0/3479408428 >> 172.21.15.44:0/3479408428 conn(0x7f324c0fd930 msgr2=0x7f324c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.737+0000 7f3252200640 1 -- 172.21.15.44:0/3479408428 shutdown_connections 2024-04-20T11:26:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.738+0000 7f3252200640 1 -- 172.21.15.44:0/3479408428 wait complete. 2024-04-20T11:26:45.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.738+0000 7f3252200640 1 Processor -- start 2024-04-20T11:26:45.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 -- start start 2024-04-20T11:26:45.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c074c50 0x7f324c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f324c10e6a0 0x7f324c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f324c114560 con 0x7f324c074c50 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1143e0 con 0x7f324c074330 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3252200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1146e0 con 0x7f324c10e6a0 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3250e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f324be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c074c50 0x7f324c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3250e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:33034/0 (socket says 172.21.15.44:33034) 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.739+0000 7f3250e00640 1 -- 172.21.15.44:0/2037197528 learned_addr learned my addr 172.21.15.44:0/2037197528 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3251800640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f324c10e6a0 0x7f324c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3250e00640 1 -- 172.21.15.44:0/2037197528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f324c10e6a0 msgr2=0x7f324c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3250e00640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f324c10e6a0 0x7f324c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3250e00640 1 -- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c074c50 msgr2=0x7f324c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3250e00640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c074c50 0x7f324c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3250e00640 1 -- 172.21.15.44:0/2037197528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c19b030 con 0x7f324c074330 2024-04-20T11:26:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3250e00640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c1a06a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f323c005e30 tx=0x7f323c0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3249600640 1 -- 172.21.15.44:0/2037197528 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c039040 con 0x7f324c074330 2024-04-20T11:26:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3249600640 1 -- 172.21.15.44:0/2037197528 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f323c044070 con 0x7f324c074330 2024-04-20T11:26:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3249600640 1 -- 172.21.15.44:0/2037197528 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c004330 con 0x7f324c074330 2024-04-20T11:26:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.740+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c19b2c0 con 0x7f324c074330 2024-04-20T11:26:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.741+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c1a74f0 con 0x7f324c074330 2024-04-20T11:26:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.744+0000 7f3249600640 1 -- 172.21.15.44:0/2037197528 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f323c0044d0 con 0x7f324c074330 2024-04-20T11:26:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.744+0000 7f3256413740 1 Processor -- start 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.744+0000 7f3256413740 1 -- start start 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.744+0000 7f3256413740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3256413740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c1238a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3256413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c1238c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3256413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564c122e5d20 con 0x564c1238a5f0 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3256413740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c1228c3e0 con 0x564c123810f0 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3256413740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564c122e1ce0 con 0x564c12381a10 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3250e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c1238a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3251800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3250e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c1238a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54440/0 (socket says 172.21.15.44:54440) 2024-04-20T11:26:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3250e00640 1 -- 172.21.15.44:0/4127513954 learned_addr learned my addr 172.21.15.44:0/4127513954 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f324be00640 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c1238c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3251800640 1 -- 172.21.15.44:0/4127513954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 msgr2=0x564c1238a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3251800640 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c1238a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3251800640 1 -- 172.21.15.44:0/4127513954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 msgr2=0x564c1238c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3251800640 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c1238c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3251800640 1 -- 172.21.15.44:0/4127513954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c122ab980 con 0x564c123810f0 2024-04-20T11:26:45.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.745+0000 7f3249600640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32300a3da0 0x7f32300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.746+0000 7f3249600640 1 -- 172.21.15.44:0/2037197528 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f323c0f29c0 con 0x7f324c074330 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.746+0000 7f3251800640 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123814d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f324c075280 tx=0x7f3244009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3927d42be71696a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.746+0000 7f3250e00640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32300a3da0 0x7f32300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.746+0000 7f3242000640 1 -- 172.21.15.44:0/4127513954 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244011ce0 con 0x564c123810f0 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.746+0000 7f3242000640 1 -- 172.21.15.44:0/4127513954 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3244011e80 con 0x564c123810f0 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.746+0000 7f3242000640 1 -- 172.21.15.44:0/4127513954 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324400fa70 con 0x564c123810f0 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 -- 172.21.15.44:0/4127513954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 msgr2=0x564c123814d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123814d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f324c075280 tx=0x7f3244009ed0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3250e00640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32300a3da0 0x7f32300a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f323c0baf40 tx=0x7f323c0f6500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 -- 172.21.15.44:0/4127513954 shutdown_connections 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c1238c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c1238a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 --2- 172.21.15.44:0/4127513954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123814d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 -- 172.21.15.44:0/4127513954 >> 172.21.15.44:0/4127513954 conn(0x564c1237b820 msgr2=0x564c1237bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 -- 172.21.15.44:0/4127513954 shutdown_connections 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 -- 172.21.15.44:0/4127513954 wait complete. 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.747+0000 7f3256413740 1 -- start start 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 Processor -- start 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c12387bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c12388130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564c1238d3e0 con 0x564c1238a5f0 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c1229f7a0 con 0x564c123810f0 2024-04-20T11:26:45.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564c1238d560 con 0x564c12381a10 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3256413740 10 client.0 _ll_register_callbacks cb 0x564c122a1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3251800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.748+0000 7f3251800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c12388130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c12388130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36492/0 (socket says 172.21.15.44:36492) 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 -- 172.21.15.44:0/3886622408 learned_addr learned my addr 172.21.15.44:0/3886622408 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f324be00640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c12387bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 -- 172.21.15.44:0/3886622408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 msgr2=0x564c12387bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c12387bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 -- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 msgr2=0x564c123876b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123876b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 -- 172.21.15.44:0/3886622408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c120c07e0 con 0x564c1238a5f0 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f324be00640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c12387bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f3251800640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c12388130 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f324400f710 tx=0x7f3244012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0eb2405776b0359 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f323aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.749+0000 7f323aa00640 1 -- 172.21.15.44:0/3886622408 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324402e060 con 0x564c1238a5f0 2024-04-20T11:26:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f323aa00640 1 -- 172.21.15.44:0/3886622408 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f32440056f0 con 0x564c1238a5f0 2024-04-20T11:26:45.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c122f3740 con 0x564c1238a5f0 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f323aa00640 1 -- 172.21.15.44:0/3886622408 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f324400fe80 con 0x564c1238a5f0 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f3256413740 10 client.16524 fetch_fsmap learned FSMap version 40 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564c12120f70 con 0x564c1238a5f0 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f323aa00640 1 -- 172.21.15.44:0/3886622408 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x564c122f3740 con 0x564c1238a5f0 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f323aa00640 1 -- 172.21.15.44:0/3886622408 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f3244031020 con 0x564c1238a5f0 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f3256413740 10 client.16524 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.750+0000 7f3256413740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f3256413740 20 client.16524 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f3256413740 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c1239a9a0 0x564c1239cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f3256413740 4 client.16524 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c122ed5b0 con 0x564c1239a9a0 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f324be00640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c1239a9a0 0x564c1239cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f324be00640 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c1239a9a0 0x564c1239cd90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145f157b0b7a7d1a server_cookie=f7a3274d12499368 in_seq=0 out_seq=0 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.751+0000 7f323aa00640 10 client.16524 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.752+0000 7f323aa00640 1 -- 172.21.15.44:0/3886622408 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c122ed5b0 con 0x564c1239a9a0 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.752+0000 7f323aa00640 10 client.16524 handle_command_reply: tid=0 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.752+0000 7f3256413740 1 client.16524 shutdown 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.752+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 msgr2=0x564c12388130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.752+0000 7f3256413740 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c12388130 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f324400f710 tx=0x7f3244012f50 comp rx=0 tx=0).stop 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 shutdown_connections 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c1239a9a0 0x564c1239cd90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c1238a5f0 0x564c12388130 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c12387bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 --2- 172.21.15.44:0/3886622408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c123810f0 0x564c123876b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 >> 172.21.15.44:0/3886622408 conn(0x564c1237b820 msgr2=0x564c1237d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f323aa00640 0 client.16524 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:45.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 shutdown_connections 2024-04-20T11:26:45.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 1 -- 172.21.15.44:0/3886622408 wait complete. 2024-04-20T11:26:45.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.753+0000 7f3256413740 20 client.16524 trim_cache size 0 max 16384 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 Processor -- start 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 -- start start 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 0x564c12389de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c123962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1237a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1afc90 con 0x564c123810f0 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1b19b0 con 0x564c1238a5f0 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3256413740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1b0360 con 0x564c12381a10 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3251800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 0x564c12389de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3251800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 0x564c12389de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36502/0 (socket says 172.21.15.44:36502) 2024-04-20T11:26:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f3251800640 1 -- 172.21.15.44:0/577246574 learned_addr learned my addr 172.21.15.44:0/577246574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.773+0000 7f324be00640 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c123962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1237a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 -- 172.21.15.44:0/577246574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 msgr2=0x564c123962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c123962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 -- 172.21.15.44:0/577246574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 msgr2=0x564c1237a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1237a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 -- 172.21.15.44:0/577246574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c12381840 con 0x564c123810f0 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3251800640 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 0x564c12389de0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f324400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854127705c060a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.774+0000 7f3241600640 1 -- 172.21.15.44:0/577246574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324402e070 con 0x564c123810f0 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3241600640 1 -- 172.21.15.44:0/577246574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32440117b0 con 0x564c123810f0 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3241600640 1 -- 172.21.15.44:0/577246574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244012a50 con 0x564c123810f0 2024-04-20T11:26:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 -- 172.21.15.44:0/577246574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 msgr2=0x564c12389de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 0x564c12389de0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f324400f680 comp rx=0 tx=0).stop 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 -- 172.21.15.44:0/577246574 shutdown_connections 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1237a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c12381a10 0x564c123962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 --2- 172.21.15.44:0/577246574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c123810f0 0x564c12389de0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 -- 172.21.15.44:0/577246574 >> 172.21.15.44:0/577246574 conn(0x564c1237b820 msgr2=0x564c1237a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 -- 172.21.15.44:0/577246574 shutdown_connections 2024-04-20T11:26:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.775+0000 7f3256413740 1 -- 172.21.15.44:0/577246574 wait complete. 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.776+0000 7f3256413740 1 -- start start 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.776+0000 7f3256413740 1 Processor -- start 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c123810f0 0x564c1238e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 0x564c1238e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1238ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1b1220 con 0x564c12381a10 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1b0ac0 con 0x564c1238a5f0 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f324c1adc00 con 0x564c123810f0 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3256413740 10 client.0 _ll_register_callbacks cb 0x7f32340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 0x564c1238e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 0x564c1238e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36512/0 (socket says 172.21.15.44:36512) 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 -- 172.21.15.44:0/4197846458 learned_addr learned my addr 172.21.15.44:0/4197846458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3251800640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c123810f0 0x564c1238e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3251800640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1238ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 -- 172.21.15.44:0/4197846458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c123810f0 msgr2=0x564c1238e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c123810f0 0x564c1238e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 -- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 msgr2=0x564c1238ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1238ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f324be00640 1 -- 172.21.15.44:0/4197846458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c11fb8170 con 0x564c12381a10 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.777+0000 7f3251800640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c123810f0 0x564c1238e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f324be00640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 0x564c1238e790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3234000c00 tx=0x7f3234012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e56246539a4022 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f323aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f323aa00640 1 -- 172.21.15.44:0/4197846458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234022730 con 0x564c12381a10 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f323aa00640 1 -- 172.21.15.44:0/4197846458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f32340228d0 con 0x564c12381a10 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32440117b0 con 0x564c12381a10 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f323aa00640 1 -- 172.21.15.44:0/4197846458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3234005020 con 0x564c12381a10 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f3256413740 10 client.16530 fetch_fsmap learned FSMap version 40 2024-04-20T11:26:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.778+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564c124096a0 con 0x564c12381a10 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f323aa00640 1 -- 172.21.15.44:0/4197846458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32440117b0 con 0x564c12381a10 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f323aa00640 1 -- 172.21.15.44:0/4197846458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f3234029070 con 0x564c12381a10 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3256413740 10 client.16530 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3256413740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3256413740 20 client.16530 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3256413740 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c12409f00 0x564c1240c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3256413740 4 client.16530 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c12269b10 con 0x564c12409f00 2024-04-20T11:26:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.779+0000 7f3251800640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c12409f00 0x564c1240c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.780+0000 7f3251800640 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c12409f00 0x564c1240c320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc41fe59d03ef606 server_cookie=70bbd72b3568ef4f in_seq=0 out_seq=0 2024-04-20T11:26:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.780+0000 7f323aa00640 10 client.16530 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.780+0000 7f323aa00640 1 -- 172.21.15.44:0/4197846458 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c12269b10 con 0x564c12409f00 2024-04-20T11:26:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.780+0000 7f323aa00640 10 client.16530 handle_command_reply: tid=0 2024-04-20T11:26:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.780+0000 7f3256413740 1 client.16530 shutdown 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 msgr2=0x564c1238e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 0x564c1238e790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3234000c00 tx=0x7f3234012e00 comp rx=0 tx=0).stop 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 shutdown_connections 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564c12409f00 0x564c1240c320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c1238a5f0 0x564c1238ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12381a10 0x564c1238e790 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 --2- 172.21.15.44:0/4197846458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564c123810f0 0x564c1238e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f323aa00640 0 client.16530 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 >> 172.21.15.44:0/4197846458 conn(0x564c1237b820 msgr2=0x564c1237aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 shutdown_connections 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 1 -- 172.21.15.44:0/4197846458 wait complete. 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.781+0000 7f3256413740 20 client.16530 trim_cache size 0 max 16384 2024-04-20T11:26:45.784 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:45.785 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:26:45.785 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "403df977-d5e1-40bb-90ca-3da05e2576bb", 2024-04-20T11:26:45.785 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:26:45.785 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32300a3da0 msgr2=0x7f32300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32300a3da0 0x7f32300a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f323c0baf40 tx=0x7f323c0f6500 comp rx=0 tx=0).stop 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 msgr2=0x7f324c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c1a06a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f323c005e30 tx=0x7f323c0346f0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 shutdown_connections 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f324c10e6a0 0x7f324c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f32300a3da0 0x7f32300a6260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f324c074c50 0x7f324c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 --2- 172.21.15.44:0/2037197528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f324c074330 0x7f324c1a06a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 >> 172.21.15.44:0/2037197528 conn(0x7f324c0fd930 msgr2=0x7f324c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 shutdown_connections 2024-04-20T11:26:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:45.782+0000 7f3252200640 1 -- 172.21.15.44:0/2037197528 wait complete. 2024-04-20T11:26:45.798 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:45 smithi044 ceph-mon[16855]: pgmap v127: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:45.798 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:45 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:26:45.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:26:45.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:26:45.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:45 smithi078 ceph-mon[21464]: pgmap v127: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:45.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:45 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:26:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:45 smithi165 ceph-mon[21975]: pgmap v127: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:45 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:26:46.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.011+0000 7f3958a00640 1 -- 172.21.15.44:0/3866133453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103020 msgr2=0x7f3954103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.011+0000 7f3958a00640 1 --2- 172.21.15.44:0/3866133453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103020 0x7f3954103400 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3944009900 tx=0x7f394402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 -- 172.21.15.44:0/3866133453 shutdown_connections 2024-04-20T11:26:46.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 --2- 172.21.15.44:0/3866133453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395410e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 --2- 172.21.15.44:0/3866133453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3954103940 0x7f395410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 --2- 172.21.15.44:0/3866133453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103020 0x7f3954103400 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 -- 172.21.15.44:0/3866133453 >> 172.21.15.44:0/3866133453 conn(0x7f39540fc880 msgr2=0x7f39540feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:46.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 -- 172.21.15.44:0/3866133453 shutdown_connections 2024-04-20T11:26:46.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.012+0000 7f3958a00640 1 -- 172.21.15.44:0/3866133453 wait complete. 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 Processor -- start 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 -- start start 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3954103020 0x7f39541a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103940 0x7f39541a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3954112360 con 0x7f3954103020 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39541121e0 con 0x7f3954103940 2024-04-20T11:26:46.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.013+0000 7f3958a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39541124e0 con 0x7f395410c440 2024-04-20T11:26:46.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3953200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3953200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54486/0 (socket says 172.21.15.44:54486) 2024-04-20T11:26:46.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3953200640 1 -- 172.21.15.44:0/196286045 learned_addr learned my addr 172.21.15.44:0/196286045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:46.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3949e00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103940 0x7f39541a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3952800640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3954103020 0x7f39541a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3949e00640 1 -- 172.21.15.44:0/196286045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 msgr2=0x7f395419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3949e00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3949e00640 1 -- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3954103020 msgr2=0x7f39541a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3949e00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3954103020 0x7f39541a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3949e00640 1 -- 172.21.15.44:0/196286045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395419b010 con 0x7f3954103940 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.014+0000 7f3953200640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.015+0000 7f3949e00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103940 0x7f39541a0c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e7a0 tx=0x7f393c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.015+0000 7f394be00640 1 -- 172.21.15.44:0/196286045 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c00ccc0 con 0x7f3954103940 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.015+0000 7f394be00640 1 -- 172.21.15.44:0/196286045 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f393c009530 con 0x7f3954103940 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.015+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395419b300 con 0x7f3954103940 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.015+0000 7f394be00640 1 -- 172.21.15.44:0/196286045 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0188e0 con 0x7f3954103940 2024-04-20T11:26:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.015+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a7520 con 0x7f3954103940 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.018+0000 7f395c190740 1 Processor -- start 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 -- start start 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f394be00640 1 -- 172.21.15.44:0/196286045 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f393c018a80 con 0x7f3954103940 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e2061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5600e1fc7150 con 0x5600e2061ca0 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600e1f6d370 con 0x5600e206a880 2024-04-20T11:26:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f395c190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5600e1fc3110 con 0x5600e2061380 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f3952800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f3949e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f3952800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36544/0 (socket says 172.21.15.44:36544) 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f3952800640 1 -- 172.21.15.44:0/2854344645 learned_addr learned my addr 172.21.15.44:0/2854344645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.019+0000 7f3953200640 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e2061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3952800640 1 -- 172.21.15.44:0/2854344645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 msgr2=0x5600e2061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3952800640 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e2061760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3952800640 1 -- 172.21.15.44:0/2854344645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 msgr2=0x5600e206cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3952800640 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3952800640 1 -- 172.21.15.44:0/2854344645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e1da37d0 con 0x5600e2061ca0 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3952800640 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206a340 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3944009900 tx=0x7f394402c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e632094a25521e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3943400640 1 -- 172.21.15.44:0/2854344645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394402c890 con 0x5600e2061ca0 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3943400640 1 -- 172.21.15.44:0/2854344645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3944004040 con 0x5600e2061ca0 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f3943400640 1 -- 172.21.15.44:0/2854344645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944004920 con 0x5600e2061ca0 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.020+0000 7f394be00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39380a3da0 0x7f39380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f394be00640 1 -- 172.21.15.44:0/196286045 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f393c0d0860 con 0x7f3954103940 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f3953200640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39380a3da0 0x7f39380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 -- 172.21.15.44:0/2854344645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 msgr2=0x5600e206a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206a340 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3944009900 tx=0x7f394402c6a0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 -- 172.21.15.44:0/2854344645 shutdown_connections 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206a340 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 --2- 172.21.15.44:0/2854344645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e2061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.021+0000 7f395c190740 1 -- 172.21.15.44:0/2854344645 >> 172.21.15.44:0/2854344645 conn(0x5600e205bab0 msgr2=0x5600e205be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.022+0000 7f395c190740 1 -- 172.21.15.44:0/2854344645 shutdown_connections 2024-04-20T11:26:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.022+0000 7f3953200640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39380a3da0 0x7f39380a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f394c009270 tx=0x7f394c009680 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.022+0000 7f395c190740 1 -- 172.21.15.44:0/2854344645 wait complete. 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.022+0000 7f395c190740 1 -- start start 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.022+0000 7f395c190740 1 Processor -- start 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e20679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 0x5600e2067f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e206a880 0x5600e2068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5600e206d610 con 0x5600e206a880 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600e1f80740 con 0x5600e2061ca0 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5600e206d790 con 0x5600e2061380 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f395c190740 10 client.0 _ll_register_callbacks cb 0x5600e1f7bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 0x5600e2067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 0x5600e2067f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:33130/0 (socket says 172.21.15.44:33130) 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 -- 172.21.15.44:0/3355267603 learned_addr learned my addr 172.21.15.44:0/3355267603 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3952800640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e20679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3953200640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e206a880 0x5600e2068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 -- 172.21.15.44:0/3355267603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 msgr2=0x5600e20679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e20679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 -- 172.21.15.44:0/3355267603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e206a880 msgr2=0x5600e2068460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e206a880 0x5600e2068460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3949e00640 1 -- 172.21.15.44:0/3355267603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e1cd93f0 con 0x5600e2061ca0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.023+0000 7f3952800640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e20679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3949e00640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 0x5600e2067f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e770 tx=0x7f393c098c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c99e00dc222af82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3937e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3937e00640 1 -- 172.21.15.44:0/3355267603 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c085e60 con 0x5600e2061ca0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3937e00640 1 -- 172.21.15.44:0/3355267603 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f393c0870e0 con 0x5600e2061ca0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3944004040 con 0x5600e2061ca0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3937e00640 1 -- 172.21.15.44:0/3355267603 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f393c00ce60 con 0x5600e2061ca0 2024-04-20T11:26:46.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f395c190740 10 client.26675 fetch_fsmap learned FSMap version 40 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5600e1c990f0 con 0x5600e2061ca0 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3937e00640 1 -- 172.21.15.44:0/3355267603 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3944004040 con 0x5600e2061ca0 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.024+0000 7f3937e00640 1 -- 172.21.15.44:0/3355267603 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f393c09bd70 con 0x5600e2061ca0 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f395c190740 10 client.26675 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f395c190740 10 client.26675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f395c190740 20 client.26675 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f395c190740 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e207ad70 0x5600e207d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f395c190740 4 client.26675 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600e1fd4b70 con 0x5600e207ad70 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f3952800640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e207ad70 0x5600e207d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f3952800640 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e207ad70 0x5600e207d160 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab45d334bd45a78c server_cookie=c7f9df6cab92787a in_seq=0 out_seq=0 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.025+0000 7f3937e00640 10 client.26675 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f3937e00640 1 -- 172.21.15.44:0/3355267603 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600e1fd4b70 con 0x5600e207ad70 2024-04-20T11:26:46.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f3937e00640 10 client.26675 handle_command_reply: tid=0 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 client.26675 shutdown 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 msgr2=0x5600e2067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 0x5600e2067f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e770 tx=0x7f393c098c80 comp rx=0 tx=0).stop 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 shutdown_connections 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e207ad70 0x5600e207d160 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e206a880 0x5600e2068460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061ca0 0x5600e2067f20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 --2- 172.21.15.44:0/3355267603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e20679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f3937e00640 0 client.26675 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.026+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 >> 172.21.15.44:0/3355267603 conn(0x5600e205bab0 msgr2=0x5600e205d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.027+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 shutdown_connections 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.027+0000 7f395c190740 1 -- 172.21.15.44:0/3355267603 wait complete. 2024-04-20T11:26:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.027+0000 7f395c190740 20 client.26675 trim_cache size 0 max 16384 2024-04-20T11:26:46.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.046+0000 7f395c190740 1 Processor -- start 2024-04-20T11:26:46.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.046+0000 7f395c190740 1 -- start start 2024-04-20T11:26:46.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f395c190740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061380 0x5600e2066920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f395c190740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e20764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f395c190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e206a880 0x5600e2082a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f395c190740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39541afcc0 con 0x5600e2061ca0 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f395c190740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b19e0 con 0x5600e2061380 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f395c190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b0390 con 0x5600e206a880 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3949e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e20764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3949e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e20764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36558/0 (socket says 172.21.15.44:36558) 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3949e00640 1 -- 172.21.15.44:0/4088193176 learned_addr learned my addr 172.21.15.44:0/4088193176 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3952800640 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061380 0x5600e2066920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3953200640 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e206a880 0x5600e2082a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3952800640 1 -- 172.21.15.44:0/4088193176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e206a880 msgr2=0x5600e2082a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3952800640 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e206a880 0x5600e2082a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3952800640 1 -- 172.21.15.44:0/4088193176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 msgr2=0x5600e20764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3952800640 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e20764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3952800640 1 -- 172.21.15.44:0/4088193176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e2061ad0 con 0x5600e2061380 2024-04-20T11:26:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.047+0000 7f3949e00640 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e20764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:26:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.048+0000 7f3952800640 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061380 0x5600e2066920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f394402c200 tx=0x7f3944005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f15cb93837aaadb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.048+0000 7f3942a00640 1 -- 172.21.15.44:0/4088193176 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394402ca60 con 0x5600e2061380 2024-04-20T11:26:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.048+0000 7f3942a00640 1 -- 172.21.15.44:0/4088193176 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3944051910 con 0x5600e2061380 2024-04-20T11:26:46.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.048+0000 7f3942a00640 1 -- 172.21.15.44:0/4088193176 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944034900 con 0x5600e2061380 2024-04-20T11:26:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 -- 172.21.15.44:0/4088193176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061380 msgr2=0x5600e2066920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061380 0x5600e2066920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f394402c200 tx=0x7f3944005450 comp rx=0 tx=0).stop 2024-04-20T11:26:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 -- 172.21.15.44:0/4088193176 shutdown_connections 2024-04-20T11:26:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e206a880 0x5600e2082a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e20764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 --2- 172.21.15.44:0/4088193176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e2061380 0x5600e2066920 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 -- 172.21.15.44:0/4088193176 >> 172.21.15.44:0/4088193176 conn(0x5600e205bab0 msgr2=0x5600e205ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 -- 172.21.15.44:0/4088193176 shutdown_connections 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 -- 172.21.15.44:0/4088193176 wait complete. 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 -- start start 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.049+0000 7f395c190740 1 Processor -- start 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e206e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b1250 con 0x5600e2061ca0 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b0af0 con 0x5600e206a880 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39541adc30 con 0x5600e2061380 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f395c190740 10 client.0 _ll_register_callbacks cb 0x7f393c00ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:26:46.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f3949e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f3952800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e206e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f3953200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f3952800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e206e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:54514/0 (socket says 172.21.15.44:54514) 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.050+0000 7f3952800640 1 -- 172.21.15.44:0/1841104363 learned_addr learned my addr 172.21.15.44:0/1841104363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3952800640 1 -- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 msgr2=0x5600e206f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3952800640 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3952800640 1 -- 172.21.15.44:0/1841104363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 msgr2=0x5600e206eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3952800640 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3952800640 1 -- 172.21.15.44:0/1841104363 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e20e0450 con 0x5600e2061380 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3952800640 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e206e5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39440097a0 tx=0x7f3944000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4733e7c6249ca0de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3937e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3937e00640 1 -- 172.21.15.44:0/1841104363 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394402ca60 con 0x5600e2061380 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3944051910 con 0x5600e2061380 2024-04-20T11:26:46.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.051+0000 7f3937e00640 1 -- 172.21.15.44:0/1841104363 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f394402cc00 con 0x5600e2061380 2024-04-20T11:26:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f3937e00640 1 -- 172.21.15.44:0/1841104363 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3944064050 con 0x5600e2061380 2024-04-20T11:26:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 10 client.26674 fetch_fsmap learned FSMap version 40 2024-04-20T11:26:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5600e20e1720 con 0x5600e2061380 2024-04-20T11:26:46.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f3937e00640 1 -- 172.21.15.44:0/1841104363 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3944051910 con 0x5600e2061380 2024-04-20T11:26:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f3937e00640 1 -- 172.21.15.44:0/1841104363 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f394403f430 con 0x5600e2061380 2024-04-20T11:26:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 10 client.26674 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:26:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 20 client.26674 populate_metadata read hostname 'smithi044' 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e20e19f0 0x5600e20e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 4 client.26674 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.052+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600e1fce9e0 con 0x5600e20e19f0 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f3949e00640 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e20e19f0 0x5600e20e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f3949e00640 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e20e19f0 0x5600e20e3e10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdcfe9ef42d30437 server_cookie=862b860f8e310426 in_seq=0 out_seq=0 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f3937e00640 10 client.26674 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f3937e00640 1 -- 172.21.15.44:0/1841104363 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5600e1fce9e0 con 0x5600e20e19f0 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f3937e00640 10 client.26674 handle_command_reply: tid=0 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f395c190740 1 client.26674 shutdown 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 msgr2=0x5600e206e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.053+0000 7f395c190740 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e206e5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39440097a0 tx=0x7f3944000bc0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 shutdown_connections 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600e20e19f0 0x5600e20e3e10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600e206a880 0x5600e206f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600e2061ca0 0x5600e206eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 --2- 172.21.15.44:0/1841104363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600e2061380 0x5600e206e5d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f3937e00640 0 client.26674 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 >> 172.21.15.44:0/1841104363 conn(0x5600e205bab0 msgr2=0x5600e20679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 shutdown_connections 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 1 -- 172.21.15.44:0/1841104363 wait complete. 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f395c190740 20 client.26674 trim_cache size 0 max 16384 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:26:46.058 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout: "403df977-d5e1-40bb-90ca-3da05e2576bb": { 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "403df977-d5e1-40bb-90ca-3da05e2576bb", 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:26:46.059 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39380a3da0 msgr2=0x7f39380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.054+0000 7f3958a00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39380a3da0 0x7f39380a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f394c009270 tx=0x7f394c009680 comp rx=0 tx=0).stop 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103940 msgr2=0x7f39541a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103940 0x7f39541a0c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e7a0 tx=0x7f393c00ec60 comp rx=0 tx=0).stop 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 shutdown_connections 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f395410c440 0x7f395419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954103940 0x7f39541a0c10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f39380a3da0 0x7f39380a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 --2- 172.21.15.44:0/196286045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3954103020 0x7f39541a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 >> 172.21.15.44:0/196286045 conn(0x7f39540fc880 msgr2=0x7f3954100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 shutdown_connections 2024-04-20T11:26:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:26:46.055+0000 7f3958a00640 1 -- 172.21.15.44:0/196286045 wait complete. 2024-04-20T11:26:46.072 INFO:tasks.cephfs.filesystem:scrub status for tag:403df977-d5e1-40bb-90ca-3da05e2576bb - {'path': '/', 'tag': '403df977-d5e1-40bb-90ca-3da05e2576bb', 'options': 'recursive,force'} 2024-04-20T11:26:46.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:46 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:26:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:46 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:46 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:26:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:46 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:46 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:26:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:46 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:46 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:26:46.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:46 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:46 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:26:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:46 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:46 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:26:46.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:46 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:26:47.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:47 smithi078 ceph-mon[21464]: pgmap v128: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:47.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:47 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:26:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:47 smithi044 ceph-mon[16855]: pgmap v128: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:47 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:26:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:47 smithi165 ceph-mon[21975]: pgmap v128: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:26:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:47 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:26:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:49 smithi078 ceph-mon[21464]: pgmap v129: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:49.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:49 smithi044 ceph-mon[16855]: pgmap v129: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:49 smithi165 ceph-mon[21975]: pgmap v129: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:50.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:50 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:26:50.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:50 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:26:50.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:50 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:26:51.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:51 smithi078 ceph-mon[21464]: pgmap v130: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:51.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:51 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:26:51.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:51 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:26:51.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:51 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:26:51.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:51 smithi044 ceph-mon[16855]: pgmap v130: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:51.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:51 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:26:51.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:51 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:26:51.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:51 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:26:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:51 smithi165 ceph-mon[21975]: pgmap v130: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:51 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:26:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:51 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:26:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:51 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:26:53.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:53 smithi078 ceph-mon[21464]: pgmap v131: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:53.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:53 smithi044 ceph-mon[16855]: pgmap v131: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:53 smithi165 ceph-mon[21975]: pgmap v131: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:55.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:55 smithi078 ceph-mon[21464]: pgmap v132: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:55 smithi044 ceph-mon[16855]: pgmap v132: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:55 smithi165 ceph-mon[21975]: pgmap v132: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:57.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:57 smithi078 ceph-mon[21464]: pgmap v133: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:57.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:57 smithi044 ceph-mon[16855]: pgmap v133: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:57 smithi165 ceph-mon[21975]: pgmap v133: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-20T11:26:59.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:26:59 smithi078 ceph-mon[21464]: pgmap v134: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:59.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:26:59 smithi044 ceph-mon[16855]: pgmap v134: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:26:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:26:59 smithi165 ceph-mon[21975]: pgmap v134: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-20T11:27:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:01 smithi078 ceph-mon[21464]: pgmap v135: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:01.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:01 smithi044 ceph-mon[16855]: pgmap v135: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:01 smithi165 ceph-mon[21975]: pgmap v135: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:03 smithi078 ceph-mon[21464]: pgmap v136: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:03.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:03 smithi044 ceph-mon[16855]: pgmap v136: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:03 smithi165 ceph-mon[21975]: pgmap v136: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:05 smithi078 ceph-mon[21464]: pgmap v137: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:05.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:05 smithi044 ceph-mon[16855]: pgmap v137: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:05 smithi165 ceph-mon[21975]: pgmap v137: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:07.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:07 smithi078 ceph-mon[21464]: pgmap v138: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:07.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:07 smithi044 ceph-mon[16855]: pgmap v138: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:07 smithi165 ceph-mon[21975]: pgmap v138: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:09 smithi078 ceph-mon[21464]: pgmap v139: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:09.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:09 smithi044 ceph-mon[16855]: pgmap v139: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:09.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:09 smithi165 ceph-mon[21975]: pgmap v139: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:11 smithi078 ceph-mon[21464]: pgmap v140: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:11 smithi044 ceph-mon[16855]: pgmap v140: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:11 smithi165 ceph-mon[21975]: pgmap v140: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:27:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:13 smithi078 ceph-mon[21464]: pgmap v141: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:13.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:13 smithi044 ceph-mon[16855]: pgmap v141: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:13 smithi165 ceph-mon[21975]: pgmap v141: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:27:15.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:15 smithi078 ceph-mon[21464]: pgmap v142: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:27:15.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:15 smithi044 ceph-mon[16855]: pgmap v142: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:27:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:15 smithi165 ceph-mon[21975]: pgmap v142: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:27:16.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:27:16.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:27:16.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.301+0000 7f9c10e00640 1 -- 172.21.15.44:0/2234248694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c112ca0 msgr2=0x7f9c0c115090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.301+0000 7f9c10e00640 1 --2- 172.21.15.44:0/2234248694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c115090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009cd0 tx=0x7f9c0402c0e0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.301+0000 7f9c10e00640 1 -- 172.21.15.44:0/2234248694 shutdown_connections 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.302+0000 7f9c10e00640 1 --2- 172.21.15.44:0/2234248694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c115090 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.302+0000 7f9c10e00640 1 --2- 172.21.15.44:0/2234248694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c076db0 0x7f9c0c112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.302+0000 7f9c10e00640 1 --2- 172.21.15.44:0/2234248694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 0x7f9c0c076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.302+0000 7f9c10e00640 1 -- 172.21.15.44:0/2234248694 >> 172.21.15.44:0/2234248694 conn(0x7f9c0c0ffad0 msgr2=0x7f9c0c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.302+0000 7f9c10e00640 1 -- 172.21.15.44:0/2234248694 shutdown_connections 2024-04-20T11:27:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.302+0000 7f9c10e00640 1 -- 172.21.15.44:0/2234248694 wait complete. 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 Processor -- start 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 -- start start 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 0x7f9c0c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c076db0 0x7f9c0c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c1a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c118920 con 0x7f9c0c076490 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1187a0 con 0x7f9c0c076db0 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.303+0000 7f9c10e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c118aa0 con 0x7f9c0c112ca0 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 0x7f9c0c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c076db0 0x7f9c0c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c076db0 0x7f9c0c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48746/0 (socket says 172.21.15.44:48746) 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 0x7f9c0c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55636/0 (socket says 172.21.15.44:55636) 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c03e00640 1 -- 172.21.15.44:0/1632238482 learned_addr learned my addr 172.21.15.44:0/1632238482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0be00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c1a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0be00640 1 -- 172.21.15.44:0/1632238482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c076db0 msgr2=0x7f9c0c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0be00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c076db0 0x7f9c0c19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0be00640 1 -- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 msgr2=0x7f9c0c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0be00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 0x7f9c0c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.304+0000 7f9c0be00640 1 -- 172.21.15.44:0/1632238482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1a3d50 con 0x7f9c0c112ca0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.305+0000 7f9c0be00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c1a3620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9c0402c6f0 tx=0x7f9c04034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.305+0000 7f9c08c00640 1 -- 172.21.15.44:0/1632238482 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04039040 con 0x7f9c0c112ca0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.305+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1a3fe0 con 0x7f9c0c112ca0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.305+0000 7f9c08c00640 1 -- 172.21.15.44:0/1632238482 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c04044070 con 0x7f9c0c112ca0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.305+0000 7f9c08c00640 1 -- 172.21.15.44:0/1632238482 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040042a0 con 0x7f9c0c112ca0 2024-04-20T11:27:16.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.305+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1ab8d0 con 0x7f9c0c112ca0 2024-04-20T11:27:16.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.308+0000 7f9c1505c740 1 Processor -- start 2024-04-20T11:27:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c08c00640 1 -- 172.21.15.44:0/1632238482 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9c04003970 con 0x7f9c0c112ca0 2024-04-20T11:27:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 -- start start 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 0x563d9a151760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151ca0 0x563d9a15a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d9a0b7150 con 0x563d9a151380 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d9a05d370 con 0x563d9a151ca0 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.309+0000 7f9c1505c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d9a0b3110 con 0x563d9a15a880 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 0x563d9a151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 0x563d9a151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55650/0 (socket says 172.21.15.44:55650) 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 -- 172.21.15.44:0/2788659246 learned_addr learned my addr 172.21.15.44:0/2788659246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c0b400640 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151ca0 0x563d9a15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 -- 172.21.15.44:0/2788659246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 msgr2=0x563d9a15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 -- 172.21.15.44:0/2788659246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151ca0 msgr2=0x563d9a15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151ca0 0x563d9a15a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 -- 172.21.15.44:0/2788659246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d99e937d0 con 0x563d9a151380 2024-04-20T11:27:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.310+0000 7f9c03e00640 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 0x563d9a151760 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4013630 tx=0x7f9bf40098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f3f47ab8a05839 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c00c00640 1 -- 172.21.15.44:0/2788659246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4005020 con 0x563d9a151380 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c00c00640 1 -- 172.21.15.44:0/2788659246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9bf40047a0 con 0x563d9a151380 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c08c00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9bf00a3da0 0x7f9bf00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c00c00640 1 -- 172.21.15.44:0/2788659246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400f840 con 0x563d9a151380 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c08c00640 1 -- 172.21.15.44:0/1632238482 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9c040f2880 con 0x7f9c0c112ca0 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c0b400640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9bf00a3da0 0x7f9bf00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c1505c740 1 -- 172.21.15.44:0/2788659246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 msgr2=0x563d9a151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.311+0000 7f9c1505c740 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 0x563d9a151760 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4013630 tx=0x7f9bf40098c0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 -- 172.21.15.44:0/2788659246 shutdown_connections 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151ca0 0x563d9a15a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 --2- 172.21.15.44:0/2788659246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151380 0x563d9a151760 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 -- 172.21.15.44:0/2788659246 >> 172.21.15.44:0/2788659246 conn(0x563d9a14bab0 msgr2=0x563d9a14be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 -- 172.21.15.44:0/2788659246 shutdown_connections 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 -- 172.21.15.44:0/2788659246 wait complete. 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c0b400640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9bf00a3da0 0x7f9bf00a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x563d9a150e50 tx=0x7f9bfc00a2d0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 -- start start 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.312+0000 7f9c1505c740 1 Processor -- start 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a157850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a151ca0 0x563d9a157d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a15a880 0x563d9a1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d9a15d670 con 0x563d9a15a880 2024-04-20T11:27:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d9a070740 con 0x563d9a151380 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d9a15d7f0 con 0x563d9a151ca0 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c1505c740 10 client.0 _ll_register_callbacks cb 0x563d9a06bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c0b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c0b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48762/0 (socket says 172.21.15.44:48762) 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c0b400640 1 -- 172.21.15.44:0/3401215207 learned_addr learned my addr 172.21.15.44:0/3401215207 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c0be00640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a15a880 0x563d9a1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.313+0000 7f9c03e00640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a151ca0 0x563d9a157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c0b400640 1 -- 172.21.15.44:0/3401215207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a151ca0 msgr2=0x563d9a157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c0b400640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a151ca0 0x563d9a157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c0b400640 1 -- 172.21.15.44:0/3401215207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a15a880 msgr2=0x563d9a1582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c0b400640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a15a880 0x563d9a1582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c0b400640 1 -- 172.21.15.44:0/3401215207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d99dc93f0 con 0x563d9a151380 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c0b400640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a157850 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc01ca00 tx=0x7f9bfc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e25e69f7c24b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9bf9600640 1 -- 172.21.15.44:0/3401215207 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc005020 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9bf9600640 1 -- 172.21.15.44:0/3401215207 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9bfc016030 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.314+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf40047a0 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9bf9600640 1 -- 172.21.15.44:0/3401215207 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9bfc0051c0 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 10 client.26693 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563d99d890f0 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9bf9600640 1 -- 172.21.15.44:0/3401215207 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9bf40047a0 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9bf9600640 1 -- 172.21.15.44:0/3401215207 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f9bfc02f030 con 0x563d9a151380 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 10 client.26693 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 20 client.26693 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a16ad10 0x563d9a16d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 4 client.26693 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.315+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9a0c4b70 con 0x563d9a16ad10 2024-04-20T11:27:16.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.316+0000 7f9c03e00640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a16ad10 0x563d9a16d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.316+0000 7f9c03e00640 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a16ad10 0x563d9a16d180 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb375f6e53dae4da server_cookie=15ef9610f10c6c11 in_seq=0 out_seq=0 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.316+0000 7f9bf9600640 10 client.26693 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.317+0000 7f9bf9600640 1 -- 172.21.15.44:0/3401215207 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d9a0c4b70 con 0x563d9a16ad10 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.317+0000 7f9bf9600640 10 client.26693 handle_command_reply: tid=0 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.317+0000 7f9c1505c740 1 client.26693 shutdown 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 msgr2=0x563d9a157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a157850 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc01ca00 tx=0x7f9bfc004730 comp rx=0 tx=0).stop 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 shutdown_connections 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a16ad10 0x563d9a16d180 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a15a880 0x563d9a1582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a151ca0 0x563d9a157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9bf9600640 0 client.26693 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3401215207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a157850 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 >> 172.21.15.44:0/3401215207 conn(0x563d9a14bab0 msgr2=0x563d9a14d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 shutdown_connections 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 1 -- 172.21.15.44:0/3401215207 wait complete. 2024-04-20T11:27:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.318+0000 7f9c1505c740 20 client.26693 trim_cache size 0 max 16384 2024-04-20T11:27:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.337+0000 7f9c1505c740 1 Processor -- start 2024-04-20T11:27:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 -- start start 2024-04-20T11:27:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a166530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a14d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b3fe0 con 0x563d9a151ca0 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b5d00 con 0x563d9a151380 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c1505c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b46b0 con 0x563d9a15a880 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a166530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55664/0 (socket says 172.21.15.44:55664) 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.338+0000 7f9c03e00640 1 -- 172.21.15.44:0/633591559 learned_addr learned my addr 172.21.15.44:0/633591559 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c0b400640 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c0be00640 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a14d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c03e00640 1 -- 172.21.15.44:0/633591559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 msgr2=0x563d9a14d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c03e00640 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a14d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c03e00640 1 -- 172.21.15.44:0/633591559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 msgr2=0x563d9a159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c03e00640 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c03e00640 1 -- 172.21.15.44:0/633591559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9a151ad0 con 0x563d9a151ca0 2024-04-20T11:27:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9c03e00640 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a166530 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009f60 tx=0x7f9bf4009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac4101f70ef217d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9bfbe00640 1 -- 172.21.15.44:0/633591559 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400f600 con 0x563d9a151ca0 2024-04-20T11:27:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9bfbe00640 1 -- 172.21.15.44:0/633591559 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9bf400fca0 con 0x563d9a151ca0 2024-04-20T11:27:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.339+0000 7f9bfbe00640 1 -- 172.21.15.44:0/633591559 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf402b580 con 0x563d9a151ca0 2024-04-20T11:27:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 -- 172.21.15.44:0/633591559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 msgr2=0x563d9a166530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a166530 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009f60 tx=0x7f9bf4009950 comp rx=0 tx=0).stop 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 -- 172.21.15.44:0/633591559 shutdown_connections 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a166530 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 --2- 172.21.15.44:0/633591559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 -- 172.21.15.44:0/633591559 >> 172.21.15.44:0/633591559 conn(0x563d9a14bab0 msgr2=0x563d9a156790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 -- 172.21.15.44:0/633591559 shutdown_connections 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.340+0000 7f9c1505c740 1 -- 172.21.15.44:0/633591559 wait complete. 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 -- start start 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 Processor -- start 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b5570 con 0x563d9a151ca0 2024-04-20T11:27:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b4e10 con 0x563d9a151380 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1b1f50 con 0x563d9a15a880 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.341+0000 7f9c1505c740 10 client.0 _ll_register_callbacks cb 0x7f9bfc01c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a15e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55676/0 (socket says 172.21.15.44:55676) 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c0b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c0be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 -- 172.21.15.44:0/3220739914 learned_addr learned my addr 172.21.15.44:0/3220739914 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 -- 172.21.15.44:0/3220739914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 msgr2=0x563d9a15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 -- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 msgr2=0x563d9a162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 -- 172.21.15.44:0/3220739914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9a15ef30 con 0x563d9a151ca0 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9c03e00640 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a15e1c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9bf402b260 tx=0x7f9bf4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321ebb3e36b390a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.342+0000 7f9bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9bf9600640 1 -- 172.21.15.44:0/3220739914 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400f800 con 0x563d9a151ca0 2024-04-20T11:27:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9bf9600640 1 -- 172.21.15.44:0/3220739914 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f9bf400f9a0 con 0x563d9a151ca0 2024-04-20T11:27:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf400fca0 con 0x563d9a151ca0 2024-04-20T11:27:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9bf9600640 1 -- 172.21.15.44:0/3220739914 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9bf400fe50 con 0x563d9a151ca0 2024-04-20T11:27:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 10 client.16563 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:16.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563d9a1d1330 con 0x563d9a151ca0 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9bf9600640 1 -- 172.21.15.44:0/3220739914 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9bf400fca0 con 0x563d9a151ca0 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9bf9600640 1 -- 172.21.15.44:0/3220739914 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f9bf403b020 con 0x563d9a151ca0 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 10 client.16563 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 20 client.16563 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.343+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a1d1760 0x563d9a1d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.344+0000 7f9c1505c740 4 client.16563 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.344+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d9a153450 con 0x563d9a1d1760 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.344+0000 7f9c0b400640 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a1d1760 0x563d9a1d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c0b400640 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a1d1760 0x563d9a1d3b80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc3433d05da06d0 server_cookie=d2fa043fec756366 in_seq=0 out_seq=0 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9bf9600640 10 client.16563 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9bf9600640 1 -- 172.21.15.44:0/3220739914 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d9a153450 con 0x563d9a1d1760 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9bf9600640 10 client.16563 handle_command_reply: tid=0 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 client.16563 shutdown 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 msgr2=0x563d9a15e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a15e1c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9bf402b260 tx=0x7f9bf4005710 comp rx=0 tx=0).stop 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 shutdown_connections 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563d9a1d1760 0x563d9a1d3b80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9a15a880 0x563d9a15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d9a151ca0 0x563d9a15e1c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 --2- 172.21.15.44:0/3220739914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d9a151380 0x563d9a162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 >> 172.21.15.44:0/3220739914 conn(0x563d9a14bab0 msgr2=0x563d9a158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9bf9600640 0 client.16563 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.345+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 shutdown_connections 2024-04-20T11:27:16.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c1505c740 1 -- 172.21.15.44:0/3220739914 wait complete. 2024-04-20T11:27:16.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c1505c740 20 client.16563 trim_cache size 0 max 16384 2024-04-20T11:27:16.349 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:27:16.349 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:27:16.349 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:27:16.349 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9bf00a3da0 msgr2=0x7f9bf00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9bf00a3da0 0x7f9bf00a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x563d9a150e50 tx=0x7f9bfc00a2d0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c112ca0 msgr2=0x7f9c0c1a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c1a3620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9c0402c6f0 tx=0x7f9c04034670 comp rx=0 tx=0).stop 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 shutdown_connections 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c0c112ca0 0x7f9c0c1a3620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c0c076db0 0x7f9c0c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9bf00a3da0 0x7f9bf00a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 --2- 172.21.15.44:0/1632238482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c0c076490 0x7f9c0c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.346+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 >> 172.21.15.44:0/1632238482 conn(0x7f9c0c0ffad0 msgr2=0x7f9c0c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.347+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 shutdown_connections 2024-04-20T11:27:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.347+0000 7f9c10e00640 1 -- 172.21.15.44:0/1632238482 wait complete. 2024-04-20T11:27:16.367 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:27:16.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:27:16.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 -- 172.21.15.44:0/1994433603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4074330 msgr2=0x7f1db4074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 --2- 172.21.15.44:0/1994433603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4074330 0x7f1db4074710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009ca0 tx=0x7f1da402c180 comp rx=0 tx=0).stop 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 -- 172.21.15.44:0/1994433603 shutdown_connections 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 --2- 172.21.15.44:0/1994433603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db410e7d0 0x7f1db4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 --2- 172.21.15.44:0/1994433603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074c50 0x7f1db410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 --2- 172.21.15.44:0/1994433603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4074330 0x7f1db4074710 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 -- 172.21.15.44:0/1994433603 >> 172.21.15.44:0/1994433603 conn(0x7f1db4079320 msgr2=0x7f1db4079730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 -- 172.21.15.44:0/1994433603 shutdown_connections 2024-04-20T11:27:16.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.574+0000 7f1db9e00640 1 -- 172.21.15.44:0/1994433603 wait complete. 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 Processor -- start 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 -- start start 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074330 0x7f1db41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 0x7f1db41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db410e7d0 0x7f1db419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4114670 con 0x7f1db4074c50 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41144f0 con 0x7f1db410e7d0 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.575+0000 7f1db9e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41147f0 con 0x7f1db4074330 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 0x7f1db41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 0x7f1db41a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55680/0 (socket says 172.21.15.44:55680) 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 -- 172.21.15.44:0/3608213515 learned_addr learned my addr 172.21.15.44:0/3608213515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:16.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2a00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074330 0x7f1db41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db3400640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db410e7d0 0x7f1db419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 -- 172.21.15.44:0/3608213515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074330 msgr2=0x7f1db41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074330 0x7f1db41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 -- 172.21.15.44:0/3608213515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db410e7d0 msgr2=0x7f1db419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db410e7d0 0x7f1db419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2000640 1 -- 172.21.15.44:0/3608213515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db419b130 con 0x7f1db4074c50 2024-04-20T11:27:16.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.576+0000 7f1db2a00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074330 0x7f1db41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.577+0000 7f1db2000640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 0x7f1db41a0cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00d950 tx=0x7f1d9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.577+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c017070 con 0x7f1db4074c50 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.577+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db419b420 con 0x7f1db4074c50 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.577+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d9c009c30 con 0x7f1db4074c50 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.577+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00f740 con 0x7f1db4074c50 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.577+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db41a7670 con 0x7f1db4074c50 2024-04-20T11:27:16.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.578+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db40691d0 con 0x7f1db4074c50 2024-04-20T11:27:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.580+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1d9c010a80 con 0x7f1db4074c50 2024-04-20T11:27:16.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.583+0000 7f1db8a00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d980a3d20 0x7f1d980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:16.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.583+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1d9c0d0380 con 0x7f1db4074c50 2024-04-20T11:27:16.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.583+0000 7f1db2a00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d980a3d20 0x7f1d980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:16.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.584+0000 7f1db2a00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d980a3d20 0x7f1d980a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1da4000f80 tx=0x7f1da4009880 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:16.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.588+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1d9c009de0 con 0x7f1db4074c50 2024-04-20T11:27:16.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.840+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1db402d5d0 con 0x7f1db4074c50 2024-04-20T11:27:16.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.842+0000 7f1db8a00640 1 -- 172.21.15.44:0/3608213515 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11968 (secure 0 0 0) 0x7f1d9c09c3e0 con 0x7f1db4074c50 2024-04-20T11:27:16.842 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:27:16.842 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:26:43.724238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24494,24550,24596,24562,24506]},"id":1}]} 2024-04-20T11:27:16.842 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 40 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d980a3d20 msgr2=0x7f1d980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d980a3d20 0x7f1d980a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1da4000f80 tx=0x7f1da4009880 comp rx=0 tx=0).stop 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 msgr2=0x7f1db41a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 0x7f1db41a0cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00d950 tx=0x7f1d9c00de10 comp rx=0 tx=0).stop 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 shutdown_connections 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db410e7d0 0x7f1db419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d980a3d20 0x7f1d980a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db4074c50 0x7f1db41a0cf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 --2- 172.21.15.44:0/3608213515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1db4074330 0x7f1db41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 >> 172.21.15.44:0/3608213515 conn(0x7f1db4079320 msgr2=0x7f1db40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 shutdown_connections 2024-04-20T11:27:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:16.845+0000 7f1db9e00640 1 -- 172.21.15.44:0/3608213515 wait complete. 2024-04-20T11:27:16.864 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 -- 172.21.15.44:0/1623099193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f521810c440 msgr2=0x7f521810e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 --2- 172.21.15.44:0/1623099193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f521810c440 0x7f521810e8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5210009ca0 tx=0x7f521002c190 comp rx=0 tx=0).stop 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 -- 172.21.15.44:0/1623099193 shutdown_connections 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 --2- 172.21.15.44:0/1623099193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f521810c440 0x7f521810e8e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 --2- 172.21.15.44:0/1623099193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5218103940 0x7f521810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 --2- 172.21.15.44:0/1623099193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5218103020 0x7f5218103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 -- 172.21.15.44:0/1623099193 >> 172.21.15.44:0/1623099193 conn(0x7f52180fc880 msgr2=0x7f52180feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 -- 172.21.15.44:0/1623099193 shutdown_connections 2024-04-20T11:27:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.081+0000 7f5217200640 1 -- 172.21.15.44:0/1623099193 wait complete. 2024-04-20T11:27:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.082+0000 7f5217200640 1 Processor -- start 2024-04-20T11:27:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 -- start start 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 0x7f5218100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5218103940 0x7f52181011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f521810c440 0x7f52181016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5218112360 con 0x7f5218103940 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52181121e0 con 0x7f521810c440 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.083+0000 7f5217200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52181124e0 con 0x7f5218103020 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 0x7f5218100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f520fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5218103940 0x7f52181011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 0x7f5218100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34624/0 (socket says 172.21.15.44:34624) 2024-04-20T11:27:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 -- 172.21.15.44:0/2464519852 learned_addr learned my addr 172.21.15.44:0/2464519852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5216800640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f521810c440 0x7f52181016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 -- 172.21.15.44:0/2464519852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f521810c440 msgr2=0x7f52181016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f521810c440 0x7f52181016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 -- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5218103940 msgr2=0x7f52181011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5218103940 0x7f52181011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 -- 172.21.15.44:0/2464519852 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52181a6d00 con 0x7f5218103020 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.084+0000 7f5215e00640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 0x7f5218100c70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f520800e9f0 tx=0x7f520800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.085+0000 7f520ea00640 1 -- 172.21.15.44:0/2464519852 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520800cd80 con 0x7f5218103020 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.085+0000 7f520ea00640 1 -- 172.21.15.44:0/2464519852 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5208005520 con 0x7f5218103020 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.085+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52181a6ff0 con 0x7f5218103020 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.085+0000 7f520ea00640 1 -- 172.21.15.44:0/2464519852 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208010690 con 0x7f5218103020 2024-04-20T11:27:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.085+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181a74b0 con 0x7f5218103020 2024-04-20T11:27:17.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 Processor -- start 2024-04-20T11:27:17.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 -- start start 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b32550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b3b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b3da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5586d0a97050 con 0x5586d0b32170 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.089+0000 7f5220016740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586d0a3d490 con 0x5586d0b3b670 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5220016740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5586d0a93010 con 0x5586d0b32a90 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b32550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f520ea00640 1 -- 172.21.15.44:0/2464519852 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f52080046c0 con 0x7f5218103020 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b32550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55714/0 (socket says 172.21.15.44:55714) 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 -- 172.21.15.44:0/3799337162 learned_addr learned my addr 172.21.15.44:0/3799337162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f520fe00640 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b3da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5215e00640 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b3b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 -- 172.21.15.44:0/3799337162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 msgr2=0x5586d0b3b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b3b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 -- 172.21.15.44:0/3799337162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 msgr2=0x5586d0b3da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b3da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f5216800640 1 -- 172.21.15.44:0/3799337162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d0a45070 con 0x5586d0b32170 2024-04-20T11:27:17.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.090+0000 7f520fe00640 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b3da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:17.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.091+0000 7f5216800640 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b32550 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5210009ca0 tx=0x7f5210009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64af898d6e22daed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.091+0000 7f5207400640 1 -- 172.21.15.44:0/3799337162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210043070 con 0x5586d0b32170 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.091+0000 7f5207400640 1 -- 172.21.15.44:0/3799337162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5210032720 con 0x5586d0b32170 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.091+0000 7f5207400640 1 -- 172.21.15.44:0/3799337162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210039040 con 0x5586d0b32170 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.091+0000 7f520ea00640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f51fc0a3da0 0x7f51fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f520ea00640 1 -- 172.21.15.44:0/2464519852 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f52080d0b90 con 0x7f5218103020 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5215e00640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f51fc0a3da0 0x7f51fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 -- 172.21.15.44:0/3799337162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 msgr2=0x5586d0b32550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b32550 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5210009ca0 tx=0x7f5210009900 comp rx=0 tx=0).stop 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 -- 172.21.15.44:0/3799337162 shutdown_connections 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b3da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b3b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 --2- 172.21.15.44:0/3799337162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b32550 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 -- 172.21.15.44:0/3799337162 >> 172.21.15.44:0/3799337162 conn(0x5586d0b2c8a0 msgr2=0x5586d0b2cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 -- 172.21.15.44:0/3799337162 shutdown_connections 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.092+0000 7f5220016740 1 -- 172.21.15.44:0/3799337162 wait complete. 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5215e00640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f51fc0a3da0 0x7f51fc0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5586d0b31c40 tx=0x7f5208004c10 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 -- start start 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 Processor -- start 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b38bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5586d0b3e450 con 0x5586d0b32170 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586d0a50860 con 0x5586d0b3b670 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.093+0000 7f5220016740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5586d0b3e5d0 con 0x5586d0b32a90 2024-04-20T11:27:17.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5220016740 10 client.0 _ll_register_callbacks cb 0x5586d0a1efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b38670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55720/0 (socket says 172.21.15.44:55720) 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 -- 172.21.15.44:0/855024624 learned_addr learned my addr 172.21.15.44:0/855024624 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f520fe00640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b38bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 -- 172.21.15.44:0/855024624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 msgr2=0x5586d0b38bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b38bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 -- 172.21.15.44:0/855024624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 msgr2=0x5586d0b390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 -- 172.21.15.44:0/855024624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d0a54950 con 0x5586d0b32170 2024-04-20T11:27:17.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f5216800640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b38670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f521002c7a0 tx=0x7f52100095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b3e2e2ff1d8abb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.094+0000 7f51fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f51fbe00640 1 -- 172.21.15.44:0/855024624 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210043050 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f51fbe00640 1 -- 172.21.15.44:0/855024624 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f5210003ee0 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5210032720 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f51fbe00640 1 -- 172.21.15.44:0/855024624 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5210058050 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f5220016740 10 client.16581 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5586d0a22820 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f51fbe00640 1 -- 172.21.15.44:0/855024624 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5210032720 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.095+0000 7f51fbe00640 1 -- 172.21.15.44:0/855024624 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f5210048a10 con 0x5586d0b32170 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f5220016740 10 client.16581 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f5220016740 10 client.16581 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f5220016740 20 client.16581 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f5220016740 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0b4b4b0 0x5586d0b4d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f5220016740 4 client.16581 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d0aa4a70 con 0x5586d0b4b4b0 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f520fe00640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0b4b4b0 0x5586d0b4d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f520fe00640 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0b4b4b0 0x5586d0b4d8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f20f9b14f4e25d80 server_cookie=72a9d1ff8b8e3037 in_seq=0 out_seq=0 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.096+0000 7f51fbe00640 10 client.16581 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.097+0000 7f51fbe00640 1 -- 172.21.15.44:0/855024624 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586d0aa4a70 con 0x5586d0b4b4b0 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.097+0000 7f51fbe00640 10 client.16581 handle_command_reply: tid=0 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.097+0000 7f5220016740 1 client.16581 shutdown 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.097+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 msgr2=0x5586d0b38670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.097+0000 7f5220016740 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b38670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f521002c7a0 tx=0x7f52100095e0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 shutdown_connections 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0b4b4b0 0x5586d0b4d8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b3b670 0x5586d0b390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b32a90 0x5586d0b38bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 --2- 172.21.15.44:0/855024624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32170 0x5586d0b38670 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f51fbe00640 0 client.16581 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 >> 172.21.15.44:0/855024624 conn(0x5586d0b2c8a0 msgr2=0x5586d0b2e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 shutdown_connections 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 1 -- 172.21.15.44:0/855024624 wait complete. 2024-04-20T11:27:17.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.098+0000 7f5220016740 20 client.16581 trim_cache size 0 max 16384 2024-04-20T11:27:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 Processor -- start 2024-04-20T11:27:17.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 -- start start 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b37f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52181afbb0 con 0x5586d0b32a90 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52181b18d0 con 0x5586d0b32170 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f5220016740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52181b0280 con 0x5586d0b3b670 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f520fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f520fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b39100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55724/0 (socket says 172.21.15.44:55724) 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.113+0000 7f520fe00640 1 -- 172.21.15.44:0/1861038887 learned_addr learned my addr 172.21.15.44:0/1861038887 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f5216800640 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b37f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f5216800640 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f520fe00640 1 -- 172.21.15.44:0/1861038887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 msgr2=0x5586d0b37f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f520fe00640 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b37f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f520fe00640 1 -- 172.21.15.44:0/1861038887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 msgr2=0x5586d0b47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f520fe00640 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f520fe00640 1 -- 172.21.15.44:0/1861038887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d07a93f0 con 0x5586d0b32a90 2024-04-20T11:27:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f520fe00640 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b39100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5586d0b4ded0 tx=0x7f5200005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e51e171ea2265c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f5206a00640 1 -- 172.21.15.44:0/1861038887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200004250 con 0x5586d0b32a90 2024-04-20T11:27:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f5206a00640 1 -- 172.21.15.44:0/1861038887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5200024070 con 0x5586d0b32a90 2024-04-20T11:27:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.114+0000 7f5206a00640 1 -- 172.21.15.44:0/1861038887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520001f490 con 0x5586d0b32a90 2024-04-20T11:27:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 -- 172.21.15.44:0/1861038887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 msgr2=0x5586d0b39100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b39100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5586d0b4ded0 tx=0x7f5200005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 -- 172.21.15.44:0/1861038887 shutdown_connections 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b37f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b39100 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 --2- 172.21.15.44:0/1861038887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 -- 172.21.15.44:0/1861038887 >> 172.21.15.44:0/1861038887 conn(0x5586d0b2c8a0 msgr2=0x5586d0b2b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 -- 172.21.15.44:0/1861038887 shutdown_connections 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.115+0000 7f5220016740 1 -- 172.21.15.44:0/1861038887 wait complete. 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 -- start start 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 Processor -- start 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b3f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b3fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52181b1140 con 0x5586d0b32a90 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52181b09e0 con 0x5586d0b32170 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52181adb20 con 0x5586d0b3b670 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.116+0000 7f5220016740 10 client.0 _ll_register_callbacks cb 0x7f5210032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b3f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48838/0 (socket says 172.21.15.44:48838) 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 -- 172.21.15.44:0/3210576538 learned_addr learned my addr 172.21.15.44:0/3210576538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b3fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f520fe00640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b3f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 -- 172.21.15.44:0/3210576538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 msgr2=0x5586d0b3fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b3fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 -- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 msgr2=0x5586d0b3f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b3f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.117+0000 7f5216800640 1 -- 172.21.15.44:0/3210576538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d08737d0 con 0x5586d0b32170 2024-04-20T11:27:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f5216800640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b3f230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f52100328a0 tx=0x7f5210036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fa704619db09fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f51fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f51fbe00640 1 -- 172.21.15.44:0/3210576538 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210043040 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f51fbe00640 1 -- 172.21.15.44:0/3210576538 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f5210048450 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5200024070 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f51fbe00640 1 -- 172.21.15.44:0/3210576538 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5210039040 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f5220016740 10 client.26720 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5586d0bb18b0 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.118+0000 7f51fbe00640 1 -- 172.21.15.44:0/3210576538 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5200024070 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f51fbe00640 1 -- 172.21.15.44:0/3210576538 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f521003f740 con 0x5586d0b32170 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f5220016740 10 client.26720 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f5220016740 10 client.26720 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f5220016740 20 client.26720 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f5220016740 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0bb1fe0 0x5586d0bb4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f5220016740 4 client.26720 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586d0a9e8e0 con 0x5586d0bb1fe0 2024-04-20T11:27:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.119+0000 7f520fe00640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0bb1fe0 0x5586d0bb4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f520fe00640 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0bb1fe0 0x5586d0bb4400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b19b2fbbc0570c4 server_cookie=c11e42cecef251c5 in_seq=0 out_seq=0 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f51fbe00640 10 client.26720 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f51fbe00640 1 -- 172.21.15.44:0/3210576538 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586d0a9e8e0 con 0x5586d0bb1fe0 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f51fbe00640 10 client.26720 handle_command_reply: tid=0 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 client.26720 shutdown 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 msgr2=0x5586d0b3f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b3f230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f52100328a0 tx=0x7f5210036040 comp rx=0 tx=0).stop 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 shutdown_connections 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0b3b670 0x5586d0b3fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0b32a90 0x5586d0b3f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x5586d0bb1fe0 0x5586d0bb4400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 --2- 172.21.15.44:0/3210576538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0b32170 0x5586d0b3f230 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f51fbe00640 0 client.26720 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 >> 172.21.15.44:0/3210576538 conn(0x5586d0b2c8a0 msgr2=0x5586d0b2da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 shutdown_connections 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 1 -- 172.21.15.44:0/3210576538 wait complete. 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.120+0000 7f5220016740 20 client.26720 trim_cache size 0 max 16384 2024-04-20T11:27:17.124 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f51fc0a3da0 msgr2=0x7f51fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f51fc0a3da0 0x7f51fc0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5586d0b31c40 tx=0x7f5208004c10 comp rx=0 tx=0).stop 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 msgr2=0x7f5218100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 0x7f5218100c70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f520800e9f0 tx=0x7f520800edf0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 shutdown_connections 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f521810c440 0x7f52181016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f51fc0a3da0 0x7f51fc0a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5218103940 0x7f52181011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 --2- 172.21.15.44:0/2464519852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218103020 0x7f5218100c70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 >> 172.21.15.44:0/2464519852 conn(0x7f52180fc880 msgr2=0x7f52180fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 shutdown_connections 2024-04-20T11:27:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.121+0000 7f5217200640 1 -- 172.21.15.44:0/2464519852 wait complete. 2024-04-20T11:27:17.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 -- 172.21.15.44:0/2714734231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38104dc0 msgr2=0x7efe38108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 --2- 172.21.15.44:0/2714734231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38104dc0 0x7efe38108e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efe20009ca0 tx=0x7efe2002c100 comp rx=0 tx=0).stop 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 -- 172.21.15.44:0/2714734231 shutdown_connections 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 --2- 172.21.15.44:0/2714734231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe381094e0 0x7efe381110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 --2- 172.21.15.44:0/2714734231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38104dc0 0x7efe38108e00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 --2- 172.21.15.44:0/2714734231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe38104410 0x7efe381047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 -- 172.21.15.44:0/2714734231 >> 172.21.15.44:0/2714734231 conn(0x7efe380ffcf0 msgr2=0x7efe38102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 -- 172.21.15.44:0/2714734231 shutdown_connections 2024-04-20T11:27:17.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.354+0000 7efe3f800640 1 -- 172.21.15.44:0/2714734231 wait complete. 2024-04-20T11:27:17.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 Processor -- start 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 -- start start 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe38104410 0x7efe381a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 0x7efe381a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe3819a9f0 0x7efe3819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe381146b0 con 0x7efe381094e0 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe38114530 con 0x7efe3819a9f0 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.355+0000 7efe3f800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe38114830 con 0x7efe38104410 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 0x7efe381a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 0x7efe381a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55766/0 (socket says 172.21.15.44:55766) 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe38104410 0x7efe381a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 -- 172.21.15.44:0/72679107 learned_addr learned my addr 172.21.15.44:0/72679107 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe3d800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe3819a9f0 0x7efe3819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 -- 172.21.15.44:0/72679107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe38104410 msgr2=0x7efe381a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe38104410 0x7efe381a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 -- 172.21.15.44:0/72679107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe3819a9f0 msgr2=0x7efe3819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe3819a9f0 0x7efe3819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 -- 172.21.15.44:0/72679107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3819b510 con 0x7efe381094e0 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe3d800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe3819a9f0 0x7efe3819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.356+0000 7efe37e00640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 0x7efe381a0d80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efe20005cc0 tx=0x7efe200093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.357+0000 7efe35600640 1 -- 172.21.15.44:0/72679107 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20039040 con 0x7efe381094e0 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.357+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3819b7a0 con 0x7efe381094e0 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.357+0000 7efe35600640 1 -- 172.21.15.44:0/72679107 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe20044070 con 0x7efe381094e0 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.357+0000 7efe35600640 1 -- 172.21.15.44:0/72679107 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20004330 con 0x7efe381094e0 2024-04-20T11:27:17.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.357+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381a7990 con 0x7efe381094e0 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.360+0000 7efe425bd740 1 Processor -- start 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.360+0000 7efe425bd740 1 -- start start 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.360+0000 7efe35600640 1 -- 172.21.15.44:0/72679107 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efe20003970 con 0x7efe381094e0 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.360+0000 7efe425bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c60390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.360+0000 7efe425bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 0x559455c68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.360+0000 7efe425bd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c6b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe425bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559455bc5050 con 0x559455c608d0 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe425bd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559455b6b490 con 0x559455c694b0 2024-04-20T11:27:17.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe425bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559455bc1010 con 0x559455c5ffb0 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 0x559455c68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c60390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 0x559455c68f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55768/0 (socket says 172.21.15.44:55768) 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3ce00640 1 -- 172.21.15.44:0/2650713666 learned_addr learned my addr 172.21.15.44:0/2650713666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe37e00640 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c6b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3d800640 1 -- 172.21.15.44:0/2650713666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 msgr2=0x559455c6b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3d800640 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c6b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3d800640 1 -- 172.21.15.44:0/2650713666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 msgr2=0x559455c68f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3d800640 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 0x559455c68f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3d800640 1 -- 172.21.15.44:0/2650713666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559455b73070 con 0x559455c5ffb0 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.361+0000 7efe3ce00640 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 0x559455c68f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.362+0000 7efe3d800640 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c60390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe30009b30 tx=0x7efe300113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff9dbd939bb52d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.362+0000 7efe2e000640 1 -- 172.21.15.44:0/2650713666 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30011e10 con 0x559455c5ffb0 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.362+0000 7efe2e000640 1 -- 172.21.15.44:0/2650713666 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe30005520 con 0x559455c5ffb0 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.362+0000 7efe2e000640 1 -- 172.21.15.44:0/2650713666 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30012690 con 0x559455c5ffb0 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.362+0000 7efe35600640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efe180a3d20 0x7efe180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.362+0000 7efe35600640 1 -- 172.21.15.44:0/72679107 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efe200f2a50 con 0x7efe381094e0 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe3ce00640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efe180a3d20 0x7efe180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 -- 172.21.15.44:0/2650713666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 msgr2=0x559455c60390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c60390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe30009b30 tx=0x7efe300113e0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 -- 172.21.15.44:0/2650713666 shutdown_connections 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c6b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c608d0 0x559455c68f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 --2- 172.21.15.44:0/2650713666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c60390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 -- 172.21.15.44:0/2650713666 >> 172.21.15.44:0/2650713666 conn(0x559455c5a6e0 msgr2=0x559455c5aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 -- 172.21.15.44:0/2650713666 shutdown_connections 2024-04-20T11:27:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe425bd740 1 -- 172.21.15.44:0/2650713666 wait complete. 2024-04-20T11:27:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.363+0000 7efe3ce00640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efe180a3d20 0x7efe180a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x559455c5fa80 tx=0x7efe28005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 -- start start 2024-04-20T11:27:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 Processor -- start 2024-04-20T11:27:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c66530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c608d0 0x559455c66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c66fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559455c6c2a0 con 0x559455c5ffb0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559455b7e860 con 0x559455c694b0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559455c6c420 con 0x559455c608d0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.364+0000 7efe425bd740 10 client.0 _ll_register_callbacks cb 0x559455b4cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c66530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c608d0 0x559455c66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c66530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55770/0 (socket says 172.21.15.44:55770) 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe3ce00640 1 -- 172.21.15.44:0/2862701255 learned_addr learned my addr 172.21.15.44:0/2862701255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe3d800640 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c66fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 -- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 msgr2=0x559455c66fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c66fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 -- 172.21.15.44:0/2862701255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 msgr2=0x559455c66530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c66530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 -- 172.21.15.44:0/2862701255 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559455b82950 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe37e00640 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c608d0 0x559455c66a70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efe200f0290 tx=0x7efe20034840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6155f5911f83db00 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.365+0000 7efe26a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe26a00640 1 -- 172.21.15.44:0/2862701255 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200be470 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe26a00640 1 -- 172.21.15.44:0/2862701255 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efe200bed70 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe30005520 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe26a00640 1 -- 172.21.15.44:0/2862701255 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efe200f0630 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 10 client.26728 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559455b50820 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe26a00640 1 -- 172.21.15.44:0/2862701255 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe30005520 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe26a00640 1 -- 172.21.15.44:0/2862701255 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7efe2003c030 con 0x559455c608d0 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 10 client.26728 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 10 client.26728 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:27:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 20 client.26728 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.366+0000 7efe425bd740 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455c793b0 0x559455c7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.367+0000 7efe425bd740 4 client.26728 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.367+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559455bd2a70 con 0x559455c793b0 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.367+0000 7efe3ce00640 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455c793b0 0x559455c7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.367+0000 7efe3ce00640 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455c793b0 0x559455c7b800 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b73cfa6f2bd3b8e server_cookie=f91a910e99ec2b58 in_seq=0 out_seq=0 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.367+0000 7efe26a00640 10 client.26728 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe26a00640 1 -- 172.21.15.44:0/2862701255 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559455bd2a70 con 0x559455c793b0 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe26a00640 10 client.26728 handle_command_reply: tid=0 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 client.26728 shutdown 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c608d0 msgr2=0x559455c66a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c608d0 0x559455c66a70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efe200f0290 tx=0x7efe20034840 comp rx=0 tx=0).stop 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 shutdown_connections 2024-04-20T11:27:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c694b0 0x559455c66fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c608d0 0x559455c66a70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455c793b0 0x559455c7b800 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 --2- 172.21.15.44:0/2862701255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c66530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe26a00640 0 client.26728 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.368+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 >> 172.21.15.44:0/2862701255 conn(0x559455c5a6e0 msgr2=0x559455c5c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.369+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 shutdown_connections 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.369+0000 7efe425bd740 1 -- 172.21.15.44:0/2862701255 wait complete. 2024-04-20T11:27:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.369+0000 7efe425bd740 20 client.26728 trim_cache size 0 max 16384 2024-04-20T11:27:17.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 Processor -- start 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 -- start start 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c68c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c75190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c694b0 0x559455c810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe381afeb0 con 0x559455c694b0 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b1bd0 con 0x559455c608d0 2024-04-20T11:27:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.383+0000 7efe425bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b0580 con 0x559455c5ffb0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c694b0 0x559455c810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c68c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c68c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34688/0 (socket says 172.21.15.44:34688) 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 -- 172.21.15.44:0/1077705774 learned_addr learned my addr 172.21.15.44:0/1077705774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe37e00640 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c75190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 -- 172.21.15.44:0/1077705774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 msgr2=0x559455c75190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c75190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 -- 172.21.15.44:0/1077705774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c694b0 msgr2=0x559455c810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c694b0 0x559455c810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 -- 172.21.15.44:0/1077705774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559455c60700 con 0x559455c5ffb0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe3ce00640 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c68c60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe280058c0 tx=0x7efe2801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d922036b2f607e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.384+0000 7efe2d600640 1 -- 172.21.15.44:0/1077705774 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2800fdc0 con 0x559455c5ffb0 2024-04-20T11:27:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe2d600640 1 -- 172.21.15.44:0/1077705774 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe2802b930 con 0x559455c5ffb0 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe2d600640 1 -- 172.21.15.44:0/1077705774 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28004ba0 con 0x559455c5ffb0 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 -- 172.21.15.44:0/1077705774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 msgr2=0x559455c68c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c68c60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe280058c0 tx=0x7efe2801b000 comp rx=0 tx=0).stop 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 -- 172.21.15.44:0/1077705774 shutdown_connections 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c694b0 0x559455c810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c75190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 --2- 172.21.15.44:0/1077705774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c5ffb0 0x559455c68c60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 -- 172.21.15.44:0/1077705774 >> 172.21.15.44:0/1077705774 conn(0x559455c5a6e0 msgr2=0x559455c66d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 -- 172.21.15.44:0/1077705774 shutdown_connections 2024-04-20T11:27:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.385+0000 7efe425bd740 1 -- 172.21.15.44:0/1077705774 wait complete. 2024-04-20T11:27:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 -- start start 2024-04-20T11:27:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 Processor -- start 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c6da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c694b0 0x559455cde940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b1440 con 0x559455c5ffb0 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b0ce0 con 0x559455c608d0 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe381ade80 con 0x559455c694b0 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.386+0000 7efe425bd740 10 client.0 _ll_register_callbacks cb 0x7efe200f6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c6d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55784/0 (socket says 172.21.15.44:55784) 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c694b0 0x559455cde940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 -- 172.21.15.44:0/1657531988 learned_addr learned my addr 172.21.15.44:0/1657531988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe37e00640 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c6da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 -- 172.21.15.44:0/1657531988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c694b0 msgr2=0x559455cde940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c694b0 0x559455cde940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 -- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 msgr2=0x559455c6da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c6da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 -- 172.21.15.44:0/1657531988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594558d73f0 con 0x559455c5ffb0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe3ce00640 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c6d4c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe28004650 tx=0x7efe2800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd038a2f72d1c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.387+0000 7efe26a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe26a00640 1 -- 172.21.15.44:0/1657531988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2802b630 con 0x559455c5ffb0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe2802b930 con 0x559455c5ffb0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe26a00640 1 -- 172.21.15.44:0/1657531988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efe28029440 con 0x559455c5ffb0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe26a00640 1 -- 172.21.15.44:0/1657531988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efe28029640 con 0x559455c5ffb0 2024-04-20T11:27:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe425bd740 10 client.16605 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5594559a17d0 con 0x559455c5ffb0 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe26a00640 1 -- 172.21.15.44:0/1657531988 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe2802b930 con 0x559455c5ffb0 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.388+0000 7efe26a00640 1 -- 172.21.15.44:0/1657531988 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7efe2801d8c0 con 0x559455c5ffb0 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe425bd740 10 client.16605 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe425bd740 10 client.16605 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe425bd740 20 client.16605 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe425bd740 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455ce0490 0x559455ce28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe425bd740 4 client.16605 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559455bcc8e0 con 0x559455ce0490 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe37e00640 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455ce0490 0x559455ce28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe37e00640 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455ce0490 0x559455ce28b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bb688378a797d46 server_cookie=f283cb1a8b0b100e in_seq=0 out_seq=0 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.389+0000 7efe26a00640 10 client.16605 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe26a00640 1 -- 172.21.15.44:0/1657531988 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559455bcc8e0 con 0x559455ce0490 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe26a00640 10 client.16605 handle_command_reply: tid=0 2024-04-20T11:27:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 client.16605 shutdown 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 msgr2=0x559455c6d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c6d4c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe28004650 tx=0x7efe2800f6f0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 shutdown_connections 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559455c694b0 0x559455cde940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559455c608d0 0x559455c6da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x559455ce0490 0x559455ce28b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 --2- 172.21.15.44:0/1657531988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559455c5ffb0 0x559455c6d4c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe26a00640 0 client.16605 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 >> 172.21.15.44:0/1657531988 conn(0x559455c5a6e0 msgr2=0x559455c66d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 shutdown_connections 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 1 -- 172.21.15.44:0/1657531988 wait complete. 2024-04-20T11:27:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.390+0000 7efe425bd740 20 client.16605 trim_cache size 0 max 16384 2024-04-20T11:27:17.395 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efe180a3d20 msgr2=0x7efe180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efe180a3d20 0x7efe180a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x559455c5fa80 tx=0x7efe28005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 msgr2=0x7efe381a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 0x7efe381a0d80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efe20005cc0 tx=0x7efe200093d0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 shutdown_connections 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe3819a9f0 0x7efe3819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efe180a3d20 0x7efe180a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe381094e0 0x7efe381a0d80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 --2- 172.21.15.44:0/72679107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe38104410 0x7efe381a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 >> 172.21.15.44:0/72679107 conn(0x7efe380ffcf0 msgr2=0x7efe38102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 shutdown_connections 2024-04-20T11:27:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.391+0000 7efe3f800640 1 -- 172.21.15.44:0/72679107 wait complete. 2024-04-20T11:27:17.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:27:17.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f6278a00640 1 -- 172.21.15.44:0/98049704 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268034440 con 0x7f627c074c30 2024-04-20T11:27:17.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 -- 172.21.15.44:0/98049704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074c30 msgr2=0x7f627c10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 --2- 172.21.15.44:0/98049704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074c30 0x7f627c10e190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6268009900 tx=0x7f626802bcd0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 -- 172.21.15.44:0/98049704 shutdown_connections 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 --2- 172.21.15.44:0/98049704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 0x7f627c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 --2- 172.21.15.44:0/98049704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074c30 0x7f627c10e190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 --2- 172.21.15.44:0/98049704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074310 0x7f627c0746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 -- 172.21.15.44:0/98049704 >> 172.21.15.44:0/98049704 conn(0x7f627c0fd900 msgr2=0x7f627c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 -- 172.21.15.44:0/98049704 shutdown_connections 2024-04-20T11:27:17.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.609+0000 7f627b200640 1 -- 172.21.15.44:0/98049704 wait complete. 2024-04-20T11:27:17.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.610+0000 7f627b200640 1 Processor -- start 2024-04-20T11:27:17.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.610+0000 7f627b200640 1 -- start start 2024-04-20T11:27:17.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074310 0x7f627c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 0x7f627c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 0x7f627c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f627c114560 con 0x7f627c074c30 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1143e0 con 0x7f627c10e6d0 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1146e0 con 0x7f627c074310 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f6279400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 0x7f627c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f6279400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 0x7f627c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55804/0 (socket says 172.21.15.44:55804) 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f6279400640 1 -- 172.21.15.44:0/3539945774 learned_addr learned my addr 172.21.15.44:0/3539945774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f627a800640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 0x7f627c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.611+0000 7f6279e00640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074310 0x7f627c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6279400640 1 -- 172.21.15.44:0/3539945774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074310 msgr2=0x7f627c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6279400640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074310 0x7f627c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6279400640 1 -- 172.21.15.44:0/3539945774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 msgr2=0x7f627c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6279400640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 0x7f627c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6279400640 1 -- 172.21.15.44:0/3539945774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c19aff0 con 0x7f627c074c30 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f627a800640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 0x7f627c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6279400640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 0x7f627c1a0ba0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6268005940 tx=0x7f6268004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6272a00640 1 -- 172.21.15.44:0/3539945774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268043020 con 0x7f627c074c30 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c19b280 con 0x7f627c074c30 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6272a00640 1 -- 172.21.15.44:0/3539945774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6268004250 con 0x7f627c074c30 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f6272a00640 1 -- 172.21.15.44:0/3539945774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268003a80 con 0x7f627c074c30 2024-04-20T11:27:17.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.612+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c1a74b0 con 0x7f627c074c30 2024-04-20T11:27:17.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 Processor -- start 2024-04-20T11:27:17.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6272a00640 1 -- 172.21.15.44:0/3539945774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6268046020 con 0x7f627c074c30 2024-04-20T11:27:17.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 -- start start 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0935360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 0x55f9a0946940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0946e80 0x55f9a0949270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f9a089a050 con 0x55f9a0934f80 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9a0840490 con 0x55f9a0946e80 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.616+0000 7f6283dd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f9a0896010 con 0x55f9a0935930 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f627a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0935360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 0x55f9a0946940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 0x55f9a0946940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34728/0 (socket says 172.21.15.44:34728) 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 -- 172.21.15.44:0/1126290787 learned_addr learned my addr 172.21.15.44:0/1126290787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279400640 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0946e80 0x55f9a0949270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 -- 172.21.15.44:0/1126290787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0946e80 msgr2=0x55f9a0949270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0946e80 0x55f9a0949270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 -- 172.21.15.44:0/1126290787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 msgr2=0x55f9a0935360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0935360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 -- 172.21.15.44:0/1126290787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a0848070 con 0x55f9a0935930 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f627a800640 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0935360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.617+0000 7f6279e00640 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 0x55f9a0946940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f626c009d20 tx=0x7f626c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0f65c50989f593 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.618+0000 7f6272a00640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62600a3da0 0x7f62600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.618+0000 7f6267400640 1 -- 172.21.15.44:0/1126290787 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c004230 con 0x55f9a0935930 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.618+0000 7f6267400640 1 -- 172.21.15.44:0/1126290787 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f626c0043d0 con 0x55f9a0935930 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.618+0000 7f6267400640 1 -- 172.21.15.44:0/1126290787 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c019990 con 0x55f9a0935930 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.618+0000 7f6272a00640 1 -- 172.21.15.44:0/3539945774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f62680f25c0 con 0x7f627c074c30 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.618+0000 7f627a800640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62600a3da0 0x7f62600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 -- 172.21.15.44:0/1126290787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 msgr2=0x55f9a0946940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 0x55f9a0946940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f626c009d20 tx=0x7f626c011810 comp rx=0 tx=0).stop 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f627a800640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62600a3da0 0x7f62600a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6274009fd0 tx=0x7f6274009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 -- 172.21.15.44:0/1126290787 shutdown_connections 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0946e80 0x55f9a0949270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0935930 0x55f9a0946940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1126290787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0935360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 -- 172.21.15.44:0/1126290787 >> 172.21.15.44:0/1126290787 conn(0x55f9a092f8a0 msgr2=0x55f9a092fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 -- 172.21.15.44:0/1126290787 shutdown_connections 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.619+0000 7f6283dd5740 1 -- 172.21.15.44:0/1126290787 wait complete. 2024-04-20T11:27:17.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 -- start start 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 Processor -- start 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0934f80 0x55f9a0943ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 0x55f9a09443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0944920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f9a0949c40 con 0x55f9a0935930 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9a0853860 con 0x55f9a0934f80 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.620+0000 7f6283dd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f9a0949dc0 con 0x55f9a0946e80 2024-04-20T11:27:17.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6283dd5740 10 client.0 _ll_register_callbacks cb 0x55f9a0821fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 0x55f9a09443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f627a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0944920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 0x55f9a09443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55820/0 (socket says 172.21.15.44:55820) 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 -- 172.21.15.44:0/1042006091 learned_addr learned my addr 172.21.15.44:0/1042006091 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279e00640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0934f80 0x55f9a0943ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 -- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 msgr2=0x55f9a0944920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0944920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 -- 172.21.15.44:0/1042006091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0934f80 msgr2=0x55f9a0943ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0934f80 0x55f9a0943ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f6279400640 1 -- 172.21.15.44:0/1042006091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a0857950 con 0x55f9a0935930 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.621+0000 7f627a800640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0944920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f6279400640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 0x55f9a09443e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f62680efe90 tx=0x7f6268009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ba5cfac37722d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f625be00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f625be00640 1 -- 172.21.15.44:0/1042006091 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62680bdfe0 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f625be00640 1 -- 172.21.15.44:0/1042006091 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f62680be8e0 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f626c0043d0 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f625be00640 1 -- 172.21.15.44:0/1042006091 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f62680f01a0 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f6283dd5740 10 client.16617 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f9a0825820 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.622+0000 7f625be00640 1 -- 172.21.15.44:0/1042006091 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f626c0043d0 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f625be00640 1 -- 172.21.15.44:0/1042006091 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f62680c6e60 con 0x55f9a0935930 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6283dd5740 10 client.16617 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6283dd5740 10 client.16617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6283dd5740 20 client.16617 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a0956e40 0x55f9a0959260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6283dd5740 4 client.16617 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a08a7a70 con 0x55f9a0956e40 2024-04-20T11:27:17.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.623+0000 7f6279e00640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a0956e40 0x55f9a0959260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.624+0000 7f6279e00640 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a0956e40 0x55f9a0959260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ca728ad122452c3 server_cookie=91df7a4e8631e75d in_seq=0 out_seq=0 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.624+0000 7f625be00640 10 client.16617 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.624+0000 7f625be00640 1 -- 172.21.15.44:0/1042006091 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9a08a7a70 con 0x55f9a0956e40 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.624+0000 7f625be00640 10 client.16617 handle_command_reply: tid=0 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 client.16617 shutdown 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 msgr2=0x55f9a09443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 0x55f9a09443e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f62680efe90 tx=0x7f6268009620 comp rx=0 tx=0).stop 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 shutdown_connections 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a0956e40 0x55f9a0959260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0944920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0935930 0x55f9a09443e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 --2- 172.21.15.44:0/1042006091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0934f80 0x55f9a0943ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f625be00640 0 client.16617 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 >> 172.21.15.44:0/1042006091 conn(0x55f9a092f8a0 msgr2=0x55f9a09315f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 shutdown_connections 2024-04-20T11:27:17.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 1 -- 172.21.15.44:0/1042006091 wait complete. 2024-04-20T11:27:17.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.625+0000 7f6283dd5740 20 client.16617 trim_cache size 0 max 16384 2024-04-20T11:27:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.640+0000 7f6283dd5740 1 Processor -- start 2024-04-20T11:27:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 -- start start 2024-04-20T11:27:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0934f80 0x55f9a09465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a0952b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 0x55f9a095eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ab810 con 0x55f9a0946e80 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ad530 con 0x55f9a0935930 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6283dd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1abee0 con 0x55f9a0934f80 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f627a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 0x55f9a095eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f627a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 0x55f9a095eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55836/0 (socket says 172.21.15.44:55836) 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f627a800640 1 -- 172.21.15.44:0/3978054871 learned_addr learned my addr 172.21.15.44:0/3978054871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.641+0000 7f6279400640 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a0952b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f6279e00640 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0934f80 0x55f9a09465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f627a800640 1 -- 172.21.15.44:0/3978054871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0934f80 msgr2=0x55f9a09465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f627a800640 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0934f80 0x55f9a09465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f627a800640 1 -- 172.21.15.44:0/3978054871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 msgr2=0x55f9a0952b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f627a800640 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a0952b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f627a800640 1 -- 172.21.15.44:0/3978054871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a05ac3f0 con 0x55f9a0946e80 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.642+0000 7f627a800640 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 0x55f9a095eb50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62740123a0 tx=0x7f627401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1441ef5b98ebc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6266a00640 1 -- 172.21.15.44:0/3978054871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740123d0 con 0x55f9a0946e80 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6266a00640 1 -- 172.21.15.44:0/3978054871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f627401f920 con 0x55f9a0946e80 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6266a00640 1 -- 172.21.15.44:0/3978054871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627401d9d0 con 0x55f9a0946e80 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 -- 172.21.15.44:0/3978054871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 msgr2=0x55f9a095eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 0x55f9a095eb50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62740123a0 tx=0x7f627401a000 comp rx=0 tx=0).stop 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 -- 172.21.15.44:0/3978054871 shutdown_connections 2024-04-20T11:27:17.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0946e80 0x55f9a095eb50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a0952b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 --2- 172.21.15.44:0/3978054871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0934f80 0x55f9a09465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.643+0000 7f6283dd5740 1 -- 172.21.15.44:0/3978054871 >> 172.21.15.44:0/3978054871 conn(0x55f9a092f8a0 msgr2=0x55f9a093f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.644+0000 7f6283dd5740 1 -- 172.21.15.44:0/3978054871 shutdown_connections 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.644+0000 7f6283dd5740 1 -- 172.21.15.44:0/3978054871 wait complete. 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.644+0000 7f6283dd5740 1 -- start start 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.644+0000 7f6283dd5740 1 Processor -- start 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0940c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a09411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0941710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1acda0 con 0x55f9a0934f80 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ac640 con 0x55f9a0935930 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1a97a0 con 0x55f9a0946e80 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6283dd5740 10 client.0 _ll_register_callbacks cb 0x7f6268004b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0940c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0940c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55848/0 (socket says 172.21.15.44:55848) 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 -- 172.21.15.44:0/2528439759 learned_addr learned my addr 172.21.15.44:0/2528439759 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279400640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a09411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f627a800640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0941710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 -- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 msgr2=0x55f9a0941710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0941710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 -- 172.21.15.44:0/2528439759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 msgr2=0x55f9a09411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a09411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.645+0000 7f6279e00640 1 -- 172.21.15.44:0/2528439759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a06767d0 con 0x55f9a0934f80 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f6279e00640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0940c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa8e749832ecaed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f625be00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f625be00640 1 -- 172.21.15.44:0/2528439759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c031060 con 0x55f9a0934f80 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f625be00640 1 -- 172.21.15.44:0/2528439759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f626c02bd50 con 0x55f9a0934f80 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f627401f920 con 0x55f9a0934f80 2024-04-20T11:27:17.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f625be00640 1 -- 172.21.15.44:0/2528439759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f626c02b3a0 con 0x55f9a0934f80 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f6283dd5740 10 client.16623 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f9a09bd590 con 0x55f9a0934f80 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.646+0000 7f625be00640 1 -- 172.21.15.44:0/2528439759 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f627401f920 con 0x55f9a0934f80 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f625be00640 1 -- 172.21.15.44:0/2528439759 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f626c034020 con 0x55f9a0934f80 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6283dd5740 10 client.16623 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6283dd5740 10 client.16623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6283dd5740 20 client.16623 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6283dd5740 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a09bdc00 0x55f9a09c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6283dd5740 4 client.16623 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:17.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9a08a18e0 con 0x55f9a09bdc00 2024-04-20T11:27:17.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.647+0000 7f6279400640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a09bdc00 0x55f9a09c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6279400640 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a09bdc00 0x55f9a09c0020 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbcdc46d3501f00c server_cookie=eb74b5b5c8bc2a9e in_seq=0 out_seq=0 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f625be00640 10 client.16623 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f625be00640 1 -- 172.21.15.44:0/2528439759 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9a08a18e0 con 0x55f9a09bdc00 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f625be00640 10 client.16623 handle_command_reply: tid=0 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 client.16623 shutdown 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 msgr2=0x55f9a0940c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0940c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c019440 comp rx=0 tx=0).stop 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 shutdown_connections 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55f9a09bdc00 0x55f9a09c0020 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f9a0946e80 0x55f9a0941710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9a0935930 0x55f9a09411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f625be00640 0 client.16623 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 --2- 172.21.15.44:0/2528439759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9a0934f80 0x55f9a0940c90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 >> 172.21.15.44:0/2528439759 conn(0x55f9a092f8a0 msgr2=0x55f9a0943e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 shutdown_connections 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 1 -- 172.21.15.44:0/2528439759 wait complete. 2024-04-20T11:27:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.648+0000 7f6283dd5740 20 client.16623 trim_cache size 0 max 16384 2024-04-20T11:27:17.655 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62600a3da0 msgr2=0x7f62600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62600a3da0 0x7f62600a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6274009fd0 tx=0x7f6274009500 comp rx=0 tx=0).stop 2024-04-20T11:27:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 msgr2=0x7f627c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 0x7f627c1a0ba0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6268005940 tx=0x7f6268004970 comp rx=0 tx=0).stop 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 shutdown_connections 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f627c10e6d0 0x7f627c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f62600a3da0 0x7f62600a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f627c074c30 0x7f627c1a0ba0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 --2- 172.21.15.44:0/3539945774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f627c074310 0x7f627c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.649+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 >> 172.21.15.44:0/3539945774 conn(0x7f627c0fd900 msgr2=0x7f627c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.650+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 shutdown_connections 2024-04-20T11:27:17.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.650+0000 7f627b200640 1 -- 172.21.15.44:0/3539945774 wait complete. 2024-04-20T11:27:17.664 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:17 smithi044 ceph-mon[16855]: pgmap v143: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:27:17.664 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:17 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3608213515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:27:17.667 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:27:17.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.863+0000 7f9153e00640 1 -- 172.21.15.44:0/2515103897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c1094e0 msgr2=0x7f914c1110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.863+0000 7f9153e00640 1 --2- 172.21.15.44:0/2515103897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c1094e0 0x7f914c1110b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9144009be0 tx=0x7f914402bef0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.863+0000 7f9153e00640 1 -- 172.21.15.44:0/2515103897 shutdown_connections 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.863+0000 7f9153e00640 1 --2- 172.21.15.44:0/2515103897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c1094e0 0x7f914c1110b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.863+0000 7f9153e00640 1 --2- 172.21.15.44:0/2515103897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c104dc0 0x7f914c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.864+0000 7f9153e00640 1 --2- 172.21.15.44:0/2515103897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c104410 0x7f914c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.864+0000 7f9153e00640 1 -- 172.21.15.44:0/2515103897 >> 172.21.15.44:0/2515103897 conn(0x7f914c0ffcf0 msgr2=0x7f914c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.864+0000 7f9153e00640 1 -- 172.21.15.44:0/2515103897 shutdown_connections 2024-04-20T11:27:17.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.864+0000 7f9153e00640 1 -- 172.21.15.44:0/2515103897 wait complete. 2024-04-20T11:27:17.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 Processor -- start 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 -- start start 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c104410 0x7f914c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 0x7f914c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 0x7f914c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f914c114720 con 0x7f914c1094e0 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1145a0 con 0x7f914c104410 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.865+0000 7f9153e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1148a0 con 0x7f914c19a980 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9150a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 0x7f914c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 0x7f914c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c104410 0x7f914c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9150a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 0x7f914c1a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55862/0 (socket says 172.21.15.44:55862) 2024-04-20T11:27:17.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 0x7f914c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34776/0 (socket says 172.21.15.44:34776) 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9150a00640 1 -- 172.21.15.44:0/386230740 learned_addr learned my addr 172.21.15.44:0/386230740 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 -- 172.21.15.44:0/386230740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c104410 msgr2=0x7f914c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c104410 0x7f914c1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 -- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 msgr2=0x7f914c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 0x7f914c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 -- 172.21.15.44:0/386230740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c19b4a0 con 0x7f914c19a980 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9150a00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 0x7f914c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.866+0000 7f9151e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 0x7f914c19ae30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f914402c430 tx=0x7f9144033720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.867+0000 7f914a000640 1 -- 172.21.15.44:0/386230740 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144043020 con 0x7f914c19a980 2024-04-20T11:27:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.867+0000 7f914a000640 1 -- 172.21.15.44:0/386230740 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9144004040 con 0x7f914c19a980 2024-04-20T11:27:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.867+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c19b730 con 0x7f914c19a980 2024-04-20T11:27:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.867+0000 7f914a000640 1 -- 172.21.15.44:0/386230740 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914403e690 con 0x7f914c19a980 2024-04-20T11:27:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.867+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1a7980 con 0x7f914c19a980 2024-04-20T11:27:17.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.870+0000 7f9156be9740 1 Processor -- start 2024-04-20T11:27:17.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.870+0000 7f914a000640 1 -- 172.21.15.44:0/386230740 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9144046020 con 0x7f914c19a980 2024-04-20T11:27:17.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.870+0000 7f9156be9740 1 -- start start 2024-04-20T11:27:17.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.870+0000 7f9156be9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9156be9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9156be9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9156be9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d96c41a050 con 0x55d96c4be4b0 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9156be9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d96c3c0490 con 0x55d96c4b58d0 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9156be9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d96c416010 con 0x55d96c4b4fb0 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34788/0 (socket says 172.21.15.44:34788) 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 -- 172.21.15.44:0/3721949911 learned_addr learned my addr 172.21.15.44:0/3721949911 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9151400640 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 -- 172.21.15.44:0/3721949911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 msgr2=0x55d96c4bdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 -- 172.21.15.44:0/3721949911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 msgr2=0x55d96c4c08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4c08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.871+0000 7f9150a00640 1 -- 172.21.15.44:0/3721949911 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96c3c8070 con 0x55d96c4b4fb0 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f914a000640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f91300a3d20 0x7f91300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f9150a00640 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4b5390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9138011be0 tx=0x7f9138009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b96dd508cf202d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f914a000640 1 -- 172.21.15.44:0/386230740 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f91440f2b10 con 0x7f914c19a980 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f9142600640 1 -- 172.21.15.44:0/3721949911 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913800fc50 con 0x55d96c4b4fb0 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f9151400640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f91300a3d20 0x7f91300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f9142600640 1 -- 172.21.15.44:0/3721949911 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f913800fdf0 con 0x55d96c4b4fb0 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.872+0000 7f9142600640 1 -- 172.21.15.44:0/3721949911 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913800e7a0 con 0x55d96c4b4fb0 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 -- 172.21.15.44:0/3721949911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 msgr2=0x55d96c4b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4b5390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9138011be0 tx=0x7f9138009530 comp rx=0 tx=0).stop 2024-04-20T11:27:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9151400640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f91300a3d20 0x7f91300a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55d96c4b4a80 tx=0x7f913c009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 -- 172.21.15.44:0/3721949911 shutdown_connections 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4c08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 --2- 172.21.15.44:0/3721949911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4b5390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 -- 172.21.15.44:0/3721949911 >> 172.21.15.44:0/3721949911 conn(0x55d96c4af6e0 msgr2=0x55d96c4afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 -- 172.21.15.44:0/3721949911 shutdown_connections 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 -- 172.21.15.44:0/3721949911 wait complete. 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.873+0000 7f9156be9740 1 -- start start 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 Processor -- start 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4be4b0 0x55d96c4bbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d96c4c12f0 con 0x55d96c4b4fb0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d96c3d3860 con 0x55d96c4b58d0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d96c4c1470 con 0x55d96c4be4b0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9156be9740 10 client.0 _ll_register_callbacks cb 0x55d96c3a1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9151400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9151400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55876/0 (socket says 172.21.15.44:55876) 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9151400640 1 -- 172.21.15.44:0/1724909303 learned_addr learned my addr 172.21.15.44:0/1724909303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9151e00640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4be4b0 0x55d96c4bbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.874+0000 7f9150a00640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151e00640 1 -- 172.21.15.44:0/1724909303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 msgr2=0x55d96c4bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151e00640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151e00640 1 -- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 msgr2=0x55d96c4bb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151e00640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151e00640 1 -- 172.21.15.44:0/1724909303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96c3d7950 con 0x55d96c4be4b0 2024-04-20T11:27:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151400640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9151e00640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4be4b0 0x55d96c4bbfb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91440f0350 tx=0x7f91440be5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38f252087641873 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9136a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9136a00640 1 -- 172.21.15.44:0/1724909303 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440c7840 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9136a00640 1 -- 172.21.15.44:0/1724909303 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f91440df070 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.875+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f913800fdf0 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9136a00640 1 -- 172.21.15.44:0/1724909303 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f91440d80d0 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 10 client.26764 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d96c3a5820 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9136a00640 1 -- 172.21.15.44:0/1724909303 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f913800fdf0 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9136a00640 1 -- 172.21.15.44:0/1724909303 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f91440c73b0 con 0x55d96c4be4b0 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 10 client.26764 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 10 client.26764 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:27:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 20 client.26764 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c4ce580 0x55d96c4d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 4 client.26764 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.876+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96c427a70 con 0x55d96c4ce580 2024-04-20T11:27:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.877+0000 7f9151400640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c4ce580 0x55d96c4d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.877+0000 7f9151400640 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c4ce580 0x55d96c4d09e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcaea0f966de09f2 server_cookie=f2df38c0ee94f60d in_seq=0 out_seq=0 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.877+0000 7f9136a00640 10 client.26764 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9136a00640 1 -- 172.21.15.44:0/1724909303 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d96c427a70 con 0x55d96c4ce580 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9136a00640 10 client.26764 handle_command_reply: tid=0 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 client.26764 shutdown 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4be4b0 msgr2=0x55d96c4bbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4be4b0 0x55d96c4bbfb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91440f0350 tx=0x7f91440be5f0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 shutdown_connections 2024-04-20T11:27:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c4ce580 0x55d96c4d09e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4be4b0 0x55d96c4bbfb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 --2- 172.21.15.44:0/1724909303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9136a00640 0 client.26764 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.878+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 >> 172.21.15.44:0/1724909303 conn(0x55d96c4af6e0 msgr2=0x55d96c4b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.879+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 shutdown_connections 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.879+0000 7f9156be9740 1 -- 172.21.15.44:0/1724909303 wait complete. 2024-04-20T11:27:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.879+0000 7f9156be9740 20 client.26764 trim_cache size 0 max 16384 2024-04-20T11:27:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 Processor -- start 2024-04-20T11:27:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 -- start start 2024-04-20T11:27:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 0x55d96c4ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4be4b0 0x55d96c4ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1afe30 con 0x55d96c4b58d0 2024-04-20T11:27:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1b1b50 con 0x55d96c4be4b0 2024-04-20T11:27:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9156be9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1b0500 con 0x55d96c4b4fb0 2024-04-20T11:27:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9150a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 0x55d96c4ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9150a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 0x55d96c4ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55886/0 (socket says 172.21.15.44:55886) 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9150a00640 1 -- 172.21.15.44:0/2827044896 learned_addr learned my addr 172.21.15.44:0/2827044896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9151e00640 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4be4b0 0x55d96c4ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.894+0000 7f9151400640 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9150a00640 1 -- 172.21.15.44:0/2827044896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 msgr2=0x55d96c4bdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9150a00640 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9150a00640 1 -- 172.21.15.44:0/2827044896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4be4b0 msgr2=0x55d96c4ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9150a00640 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4be4b0 0x55d96c4ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9150a00640 1 -- 172.21.15.44:0/2827044896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96c12c3f0 con 0x55d96c4b58d0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9150a00640 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 0x55d96c4ca1b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f91380058c0 tx=0x7f91380095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0c5ed97a1d794a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9141c00640 1 -- 172.21.15.44:0/2827044896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913802c070 con 0x55d96c4b58d0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9141c00640 1 -- 172.21.15.44:0/2827044896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9138005540 con 0x55d96c4b58d0 2024-04-20T11:27:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.895+0000 7f9141c00640 1 -- 172.21.15.44:0/2827044896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138004d70 con 0x55d96c4b58d0 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 -- 172.21.15.44:0/2827044896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 msgr2=0x55d96c4ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 0x55d96c4ca1b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f91380058c0 tx=0x7f91380095a0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 -- 172.21.15.44:0/2827044896 shutdown_connections 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4be4b0 0x55d96c4ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4b58d0 0x55d96c4ca1b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 --2- 172.21.15.44:0/2827044896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4bdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 -- 172.21.15.44:0/2827044896 >> 172.21.15.44:0/2827044896 conn(0x55d96c4af6e0 msgr2=0x55d96c4b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 -- 172.21.15.44:0/2827044896 shutdown_connections 2024-04-20T11:27:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.896+0000 7f9156be9740 1 -- 172.21.15.44:0/2827044896 wait complete. 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 -- start start 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 Processor -- start 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1b13c0 con 0x55d96c4be4b0 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1b0c60 con 0x55d96c4b58d0 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1addc0 con 0x55d96c4b4fb0 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.897+0000 7f9156be9740 10 client.0 _ll_register_callbacks cb 0x7f91440be710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34818/0 (socket says 172.21.15.44:34818) 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 -- 172.21.15.44:0/408549440 learned_addr learned my addr 172.21.15.44:0/408549440 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9150a00640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 -- 172.21.15.44:0/408549440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 msgr2=0x55d96c4c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 -- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 msgr2=0x55d96c4b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 -- 172.21.15.44:0/408549440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96c1f67d0 con 0x55d96c4b4fb0 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151e00640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9151400640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4c66c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f913c01e9a0 tx=0x7f913c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6d5243deb3f809 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.898+0000 7f9136a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9136a00640 1 -- 172.21.15.44:0/408549440 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c01ee00 con 0x55d96c4b4fb0 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9138005540 con 0x55d96c4b4fb0 2024-04-20T11:27:17.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9136a00640 1 -- 172.21.15.44:0/408549440 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f913c004020 con 0x55d96c4b4fb0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9136a00640 1 -- 172.21.15.44:0/408549440 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f913c004220 con 0x55d96c4b4fb0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9156be9740 10 client.26770 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d96c534640 con 0x55d96c4b4fb0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.899+0000 7f9136a00640 1 -- 172.21.15.44:0/408549440 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9138005540 con 0x55d96c4b4fb0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9136a00640 1 -- 172.21.15.44:0/408549440 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f913c040020 con 0x55d96c4b4fb0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9156be9740 10 client.26770 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9156be9740 10 client.26770 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9156be9740 20 client.26770 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9156be9740 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c534d50 0x55d96c537170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9156be9740 4 client.26770 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d96c4218e0 con 0x55d96c534d50 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9150a00640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c534d50 0x55d96c537170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9150a00640 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c534d50 0x55d96c537170 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6ed25599aad5761 server_cookie=80542618a7d3dbc2 in_seq=0 out_seq=0 2024-04-20T11:27:17.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.900+0000 7f9136a00640 10 client.26770 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9136a00640 1 -- 172.21.15.44:0/408549440 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d96c4218e0 con 0x55d96c534d50 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9136a00640 10 client.26770 handle_command_reply: tid=0 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 client.26770 shutdown 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 msgr2=0x55d96c4c66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4c66c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f913c01e9a0 tx=0x7f913c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 shutdown_connections 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d96c534d50 0x55d96c537170 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d96c4be4b0 0x55d96c4b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d96c4b58d0 0x55d96c4c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9136a00640 0 client.26770 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 --2- 172.21.15.44:0/408549440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d96c4b4fb0 0x55d96c4c66c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 >> 172.21.15.44:0/408549440 conn(0x55d96c4af6e0 msgr2=0x55d96c4b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 shutdown_connections 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 1 -- 172.21.15.44:0/408549440 wait complete. 2024-04-20T11:27:17.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.901+0000 7f9156be9740 20 client.26770 trim_cache size 0 max 16384 2024-04-20T11:27:17.905 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f91300a3d20 msgr2=0x7f91300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f91300a3d20 0x7f91300a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55d96c4b4a80 tx=0x7f913c009260 comp rx=0 tx=0).stop 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 msgr2=0x7f914c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 0x7f914c19ae30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f914402c430 tx=0x7f9144033720 comp rx=0 tx=0).stop 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 shutdown_connections 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f914c19a980 0x7f914c19ae30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f91300a3d20 0x7f91300a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f914c1094e0 0x7f914c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 --2- 172.21.15.44:0/386230740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f914c104410 0x7f914c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 >> 172.21.15.44:0/386230740 conn(0x7f914c0ffcf0 msgr2=0x7f914c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 shutdown_connections 2024-04-20T11:27:17.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:17.902+0000 7f9153e00640 1 -- 172.21.15.44:0/386230740 wait complete. 2024-04-20T11:27:17.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:27:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:17 smithi078 ceph-mon[21464]: pgmap v143: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:27:17.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:17 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3608213515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:27:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:17 smithi165 ceph-mon[21975]: pgmap v143: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:27:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:17 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3608213515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 -- 172.21.15.44:0/2769502473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 msgr2=0x7ff76010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 --2- 172.21.15.44:0/2769502473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff76010dc00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff758009860 tx=0x7ff75802bb70 comp rx=0 tx=0).stop 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 -- 172.21.15.44:0/2769502473 shutdown_connections 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 --2- 172.21.15.44:0/2769502473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff76010dc00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 --2- 172.21.15.44:0/2769502473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff760101910 0x7ff760105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 --2- 172.21.15.44:0/2769502473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff760100f60 0x7ff760101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 -- 172.21.15.44:0/2769502473 >> 172.21.15.44:0/2769502473 conn(0x7ff7600fc7c0 msgr2=0x7ff7600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:18.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 -- 172.21.15.44:0/2769502473 shutdown_connections 2024-04-20T11:27:18.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.128+0000 7ff75fe00640 1 -- 172.21.15.44:0/2769502473 wait complete. 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.129+0000 7ff75fe00640 1 Processor -- start 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.129+0000 7ff75fe00640 1 -- start start 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.129+0000 7ff75fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff760100f60 0x7ff7601a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff7601a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 0x7ff76019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff760111220 con 0x7ff76019a810 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601110a0 con 0x7ff760100f60 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601113a0 con 0x7ff760106030 2024-04-20T11:27:18.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 0x7ff76019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff7601a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 0x7ff76019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55914/0 (socket says 172.21.15.44:55914) 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff7601a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34842/0 (socket says 172.21.15.44:34842) 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75f400640 1 -- 172.21.15.44:0/3978496498 learned_addr learned my addr 172.21.15.44:0/3978496498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff760100f60 0x7ff7601a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 -- 172.21.15.44:0/3978496498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff760100f60 msgr2=0x7ff7601a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff760100f60 0x7ff7601a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 -- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 msgr2=0x7ff76019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 0x7ff76019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75e000640 1 -- 172.21.15.44:0/3978496498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76019b3a0 con 0x7ff760106030 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.130+0000 7ff75f400640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 0x7ff76019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.131+0000 7ff75e000640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff7601a0b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e9c0 tx=0x7ff74c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.131+0000 7ff757400640 1 -- 172.21.15.44:0/3978496498 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00ccc0 con 0x7ff760106030 2024-04-20T11:27:18.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.131+0000 7ff757400640 1 -- 172.21.15.44:0/3978496498 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c005020 con 0x7ff760106030 2024-04-20T11:27:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.131+0000 7ff757400640 1 -- 172.21.15.44:0/3978496498 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004a60 con 0x7ff760106030 2024-04-20T11:27:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.131+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76019b690 con 0x7ff760106030 2024-04-20T11:27:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.131+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7601a77d0 con 0x7ff760106030 2024-04-20T11:27:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.134+0000 7ff7685b0740 1 Processor -- start 2024-04-20T11:27:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.134+0000 7ff7685b0740 1 -- start start 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 0x55738cb95550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb9e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cba0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff7685b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55738cafa050 con 0x55738cb95170 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff7685b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55738caa0490 con 0x55738cb9e670 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff7685b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55738caf6010 con 0x55738cb95a90 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff757400640 1 -- 172.21.15.44:0/3978496498 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff74c004c80 con 0x7ff760106030 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff75f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 0x55738cb95550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff75f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 0x55738cb95550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55930/0 (socket says 172.21.15.44:55930) 2024-04-20T11:27:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff75f400640 1 -- 172.21.15.44:0/2763935195 learned_addr learned my addr 172.21.15.44:0/2763935195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.135+0000 7ff75ea00640 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb9e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75e000640 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cba0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75f400640 1 -- 172.21.15.44:0/2763935195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 msgr2=0x55738cb9e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75f400640 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb9e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75f400640 1 -- 172.21.15.44:0/2763935195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 msgr2=0x55738cba0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75f400640 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cba0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75f400640 1 -- 172.21.15.44:0/2763935195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738caa8070 con 0x55738cb95170 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75e000640 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cba0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff75f400640 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 0x55738cb95550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff758009780 tx=0x7ff758009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e30f07c25364ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff74be00640 1 -- 172.21.15.44:0/2763935195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758043020 con 0x55738cb95170 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff74be00640 1 -- 172.21.15.44:0/2763935195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff758034940 con 0x55738cb95170 2024-04-20T11:27:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.136+0000 7ff74be00640 1 -- 172.21.15.44:0/2763935195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758039040 con 0x55738cb95170 2024-04-20T11:27:18.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 -- 172.21.15.44:0/2763935195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 msgr2=0x55738cb95550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 0x55738cb95550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff758009780 tx=0x7ff758009490 comp rx=0 tx=0).stop 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff757400640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7440a3da0 0x7ff7440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 -- 172.21.15.44:0/2763935195 shutdown_connections 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cba0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb9e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2763935195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95170 0x55738cb95550 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 -- 172.21.15.44:0/2763935195 >> 172.21.15.44:0/2763935195 conn(0x55738cb8f8a0 msgr2=0x55738cb8fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff757400640 1 -- 172.21.15.44:0/3978496498 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff74c0d0770 con 0x7ff760106030 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff7685b0740 1 -- 172.21.15.44:0/2763935195 shutdown_connections 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.137+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7440a3da0 0x7ff7440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.138+0000 7ff7685b0740 1 -- 172.21.15.44:0/2763935195 wait complete. 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.138+0000 7ff7685b0740 1 -- start start 2024-04-20T11:27:18.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.138+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7440a3da0 0x7ff7440a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55738cb94c40 tx=0x7ff750011040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.138+0000 7ff7685b0740 1 Processor -- start 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 0x55738cb9b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cb9bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cb9c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55738cba1400 con 0x55738cb95a90 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55738cab3860 con 0x55738cb9e670 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55738cba1580 con 0x55738cb95170 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff7685b0740 10 client.0 _ll_register_callbacks cb 0x55738ca81fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cb9bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 0x55738cb9b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cb9c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 0x55738cb9b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34860/0 (socket says 172.21.15.44:34860) 2024-04-20T11:27:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cb9bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55944/0 (socket says 172.21.15.44:55944) 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cb9c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:49010/0 (socket says 172.21.15.44:49010) 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.139+0000 7ff75ea00640 1 -- 172.21.15.44:0/2218256873 learned_addr learned my addr 172.21.15.44:0/2218256873 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75ea00640 1 -- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 msgr2=0x55738cb9c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75ea00640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cb9c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75ea00640 1 -- 172.21.15.44:0/2218256873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 msgr2=0x55738cb9bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75ea00640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cb9bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75ea00640 1 -- 172.21.15.44:0/2218256873 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738cab7950 con 0x55738cb95170 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75f400640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cb9c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75e000640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cb9bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff75ea00640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 0x55738cb9b6c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7500043d0 tx=0x7ff750004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0dd6905ccc0de49 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff748c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff748c00640 1 -- 172.21.15.44:0/2218256873 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75002b060 con 0x55738cb95170 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff748c00640 1 -- 172.21.15.44:0/2218256873 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff75001bda0 con 0x55738cb95170 2024-04-20T11:27:18.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.140+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff758034940 con 0x55738cb95170 2024-04-20T11:27:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.141+0000 7ff748c00640 1 -- 172.21.15.44:0/2218256873 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff750030440 con 0x55738cb95170 2024-04-20T11:27:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.141+0000 7ff7685b0740 10 client.26782 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.141+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55738ca85820 con 0x55738cb95170 2024-04-20T11:27:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.141+0000 7ff748c00640 1 -- 172.21.15.44:0/2218256873 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff758034940 con 0x55738cb95170 2024-04-20T11:27:18.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.141+0000 7ff748c00640 1 -- 172.21.15.44:0/2218256873 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7ff75002e020 con 0x55738cb95170 2024-04-20T11:27:18.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff7685b0740 10 client.26782 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:18.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff7685b0740 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:18.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff7685b0740 20 client.26782 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:18.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cbae690 0x55738cbb0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff7685b0740 4 client.26782 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738cb07a70 con 0x55738cbae690 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.142+0000 7ff75e000640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cbae690 0x55738cbb0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.143+0000 7ff75e000640 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cbae690 0x55738cbb0b30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a119e9fcd794cc server_cookie=b19f4277570f206e in_seq=0 out_seq=0 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.143+0000 7ff748c00640 10 client.26782 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.143+0000 7ff748c00640 1 -- 172.21.15.44:0/2218256873 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55738cb07a70 con 0x55738cbae690 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.143+0000 7ff748c00640 10 client.26782 handle_command_reply: tid=0 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.143+0000 7ff7685b0740 1 client.26782 shutdown 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 msgr2=0x55738cb9b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 0x55738cb9b6c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7500043d0 tx=0x7ff750004ab0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 shutdown_connections 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cbae690 0x55738cbb0b30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb9e670 0x55738cb9c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cb9bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff748c00640 0 client.26782 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 --2- 172.21.15.44:0/2218256873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95170 0x55738cb9b6c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 >> 172.21.15.44:0/2218256873 conn(0x55738cb8f8a0 msgr2=0x55738cb8e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 shutdown_connections 2024-04-20T11:27:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 1 -- 172.21.15.44:0/2218256873 wait complete. 2024-04-20T11:27:18.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.144+0000 7ff7685b0740 20 client.26782 trim_cache size 0 max 16384 2024-04-20T11:27:18.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 Processor -- start 2024-04-20T11:27:18.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 -- start start 2024-04-20T11:27:18.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cb9ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cbaa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb9e670 0x55738cb8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601afc50 con 0x55738cb95a90 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601b1970 con 0x55738cb95170 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.159+0000 7ff7685b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601b0320 con 0x55738cb9e670 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cbaa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cb9ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cbaa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55948/0 (socket says 172.21.15.44:55948) 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 -- 172.21.15.44:0/4129296159 learned_addr learned my addr 172.21.15.44:0/4129296159 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75f400640 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb9e670 0x55738cb8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 -- 172.21.15.44:0/4129296159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb9e670 msgr2=0x55738cb8e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb9e670 0x55738cb8e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 -- 172.21.15.44:0/4129296159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 msgr2=0x55738cb9ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cb9ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 -- 172.21.15.44:0/4129296159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738c80c3f0 con 0x55738cb95a90 2024-04-20T11:27:18.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.160+0000 7ff75e000640 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cbaa2c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff74c004770 tx=0x7ff74c0d2bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b061804c60d8fdba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:18.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff74b400640 1 -- 172.21.15.44:0/4129296159 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c01c040 con 0x55738cb95a90 2024-04-20T11:27:18.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff74b400640 1 -- 172.21.15.44:0/4129296159 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c0a5390 con 0x55738cb95a90 2024-04-20T11:27:18.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff74b400640 1 -- 172.21.15.44:0/4129296159 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c09cbe0 con 0x55738cb95a90 2024-04-20T11:27:18.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 -- 172.21.15.44:0/4129296159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 msgr2=0x55738cbaa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cbaa2c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff74c004770 tx=0x7ff74c0d2bc0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 -- 172.21.15.44:0/4129296159 shutdown_connections 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb9e670 0x55738cb8e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb95a90 0x55738cbaa2c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 --2- 172.21.15.44:0/4129296159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cb9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 -- 172.21.15.44:0/4129296159 >> 172.21.15.44:0/4129296159 conn(0x55738cb8f8a0 msgr2=0x55738cb8e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 -- 172.21.15.44:0/4129296159 shutdown_connections 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.161+0000 7ff7685b0740 1 -- 172.21.15.44:0/4129296159 wait complete. 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 -- start start 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 Processor -- start 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cba6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb97fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 0x55738cb984f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601b11e0 con 0x55738cb9e670 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601b0a80 con 0x55738cb95170 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.162+0000 7ff7685b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601adbe0 con 0x55738cb95a90 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff7685b0740 10 client.0 _ll_register_callbacks cb 0x7ff750004050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 0x55738cb984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 0x55738cb984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55950/0 (socket says 172.21.15.44:55950) 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 -- 172.21.15.44:0/3356164038 learned_addr learned my addr 172.21.15.44:0/3356164038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cba6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75e000640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb97fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 -- 172.21.15.44:0/3356164038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 msgr2=0x55738cb97fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb97fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 -- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 msgr2=0x55738cba6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cba6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75f400640 1 -- 172.21.15.44:0/3356164038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738c8d67d0 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75e000640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb97fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.163+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cba6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff75f400640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 0x55738cb984f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ebef4a2fd0d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff748c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff748c00640 1 -- 172.21.15.44:0/3356164038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758056060 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff748c00640 1 -- 172.21.15.44:0/3356164038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7ff758004250 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74c0a5390 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff748c00640 1 -- 172.21.15.44:0/3356164038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff758039040 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff7685b0740 10 client.16659 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55738cc1d290 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.164+0000 7ff748c00640 1 -- 172.21.15.44:0/3356164038 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c0a5390 con 0x55738cb9e670 2024-04-20T11:27:18.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff748c00640 1 -- 172.21.15.44:0/3356164038 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7ff758059020 con 0x55738cb9e670 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff7685b0740 10 client.16659 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff7685b0740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff7685b0740 20 client.16659 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff7685b0740 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cc1d6e0 0x55738cc1fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff7685b0740 4 client.16659 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55738cb018e0 con 0x55738cc1d6e0 2024-04-20T11:27:18.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.165+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cc1d6e0 0x55738cc1fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff75ea00640 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cc1d6e0 0x55738cc1fad0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd629a9ab3c2694 server_cookie=ed2928121d6a909b in_seq=0 out_seq=0 2024-04-20T11:27:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff748c00640 10 client.16659 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff748c00640 1 -- 172.21.15.44:0/3356164038 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55738cb018e0 con 0x55738cc1d6e0 2024-04-20T11:27:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff748c00640 10 client.16659 handle_command_reply: tid=0 2024-04-20T11:27:18.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 client.16659 shutdown 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 msgr2=0x55738cb984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 0x55738cb984f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758005280 comp rx=0 tx=0).stop 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 shutdown_connections 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738cc1d6e0 0x55738cc1fad0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738cb9e670 0x55738cb984f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738cb95a90 0x55738cb97fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 --2- 172.21.15.44:0/3356164038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738cb95170 0x55738cba6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff748c00640 0 client.16659 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 >> 172.21.15.44:0/3356164038 conn(0x55738cb8f8a0 msgr2=0x55738cb8e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 shutdown_connections 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 1 -- 172.21.15.44:0/3356164038 wait complete. 2024-04-20T11:27:18.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.166+0000 7ff7685b0740 20 client.16659 trim_cache size 0 max 16384 2024-04-20T11:27:18.171 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7440a3da0 msgr2=0x7ff7440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7440a3da0 0x7ff7440a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55738cb94c40 tx=0x7ff750011040 comp rx=0 tx=0).stop 2024-04-20T11:27:18.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 msgr2=0x7ff7601a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff7601a0b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e9c0 tx=0x7ff74c00ee80 comp rx=0 tx=0).stop 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 shutdown_connections 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7440a3da0 0x7ff7440a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff76019a810 0x7ff76019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff760106030 0x7ff7601a0b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 --2- 172.21.15.44:0/3978496498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff760100f60 0x7ff7601a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 >> 172.21.15.44:0/3978496498 conn(0x7ff7600fc7c0 msgr2=0x7ff76010cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 shutdown_connections 2024-04-20T11:27:18.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:18.167+0000 7ff75fe00640 1 -- 172.21.15.44:0/3978496498 wait complete. 2024-04-20T11:27:19.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:27:19.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:19.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 -- 172.21.15.44:0/184288393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0508102fc0 msgr2=0x7f05081033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 --2- 172.21.15.44:0/184288393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0508102fc0 0x7f05081033a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009be0 tx=0x7f04f802bf80 comp rx=0 tx=0).stop 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 -- 172.21.15.44:0/184288393 shutdown_connections 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 --2- 172.21.15.44:0/184288393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f050810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 --2- 172.21.15.44:0/184288393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081038e0 0x7f050810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 --2- 172.21.15.44:0/184288393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0508102fc0 0x7f05081033a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 -- 172.21.15.44:0/184288393 >> 172.21.15.44:0/184288393 conn(0x7f05080fc7e0 msgr2=0x7f05080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 -- 172.21.15.44:0/184288393 shutdown_connections 2024-04-20T11:27:19.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.417+0000 7f0507200640 1 -- 172.21.15.44:0/184288393 wait complete. 2024-04-20T11:27:19.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.418+0000 7f0507200640 1 Processor -- start 2024-04-20T11:27:19.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 -- start start 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 0x7f050819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05081038e0 0x7f050819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f0508198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0508112350 con 0x7f050810c3e0 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05081121d0 con 0x7f0508102fc0 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0507200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f05081124d0 con 0x7f05081038e0 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0506800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f0508198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0505e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 0x7f050819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0505e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 0x7f050819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32852/0 (socket says 172.21.15.44:32852) 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0505e00640 1 -- 172.21.15.44:0/3895459296 learned_addr learned my addr 172.21.15.44:0/3895459296 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.419+0000 7f0506800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f0508198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57722/0 (socket says 172.21.15.44:57722) 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505400640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05081038e0 0x7f050819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505e00640 1 -- 172.21.15.44:0/3895459296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05081038e0 msgr2=0x7f050819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505e00640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05081038e0 0x7f050819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505e00640 1 -- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 msgr2=0x7f0508198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505e00640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f0508198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505e00640 1 -- 172.21.15.44:0/3895459296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0508198d90 con 0x7f0508102fc0 2024-04-20T11:27:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0506800640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f0508198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f0505e00640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 0x7f050819e410 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f04f802c590 tx=0x7f04f8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.420+0000 7f04fea00640 1 -- 172.21.15.44:0/3895459296 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8043020 con 0x7f0508102fc0 2024-04-20T11:27:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.421+0000 7f04fea00640 1 -- 172.21.15.44:0/3895459296 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04f80333e0 con 0x7f0508102fc0 2024-04-20T11:27:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.421+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0508199020 con 0x7f0508102fc0 2024-04-20T11:27:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.421+0000 7f04fea00640 1 -- 172.21.15.44:0/3895459296 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f803e6b0 con 0x7f0508102fc0 2024-04-20T11:27:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.421+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a52d0 con 0x7f0508102fc0 2024-04-20T11:27:19.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.424+0000 7f04fea00640 1 -- 172.21.15.44:0/3895459296 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f04f8046020 con 0x7f0508102fc0 2024-04-20T11:27:19.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.424+0000 7f050f98b740 1 Processor -- start 2024-04-20T11:27:19.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.424+0000 7f050f98b740 1 -- start start 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f050f98b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x55998978c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f050f98b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x5599897954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f050f98b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989797df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f050f98b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5599896ecd20 con 0x55998978c500 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f050f98b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5599896933e0 con 0x55998978ce20 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f050f98b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5599896e8ce0 con 0x559989795a00 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0506800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x55998978c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x5599897954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0506800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x55998978c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57730/0 (socket says 172.21.15.44:57730) 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0506800640 1 -- 172.21.15.44:0/3345558231 learned_addr learned my addr 172.21.15.44:0/3345558231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505400640 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989797df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505e00640 1 -- 172.21.15.44:0/3345558231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 msgr2=0x559989797df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505e00640 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989797df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505e00640 1 -- 172.21.15.44:0/3345558231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 msgr2=0x55998978c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505e00640 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x55998978c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.425+0000 7f0505e00640 1 -- 172.21.15.44:0/3345558231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599896b2980 con 0x55998978ce20 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f0505e00640 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x5599897954c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f04f80f0590 tx=0x7f04f80f6df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c22d949d76f107 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f04fea00640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f04ec0a3d20 0x7f04ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f04f7000640 1 -- 172.21.15.44:0/3345558231 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8102070 con 0x55998978ce20 2024-04-20T11:27:19.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f04fea00640 1 -- 172.21.15.44:0/3895459296 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f04f80f2cc0 con 0x7f0508102fc0 2024-04-20T11:27:19.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f04f7000640 1 -- 172.21.15.44:0/3345558231 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04f800b0a0 con 0x55998978ce20 2024-04-20T11:27:19.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f0506800640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f04ec0a3d20 0x7f04ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.426+0000 7f04f7000640 1 -- 172.21.15.44:0/3345558231 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f80bb080 con 0x55998978ce20 2024-04-20T11:27:19.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 -- 172.21.15.44:0/3345558231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 msgr2=0x5599897954c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x5599897954c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f04f80f0590 tx=0x7f04f80f6df0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f0506800640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f04ec0a3d20 0x7f04ec0a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05000126c0 tx=0x7f0500005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 -- 172.21.15.44:0/3345558231 shutdown_connections 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989797df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x5599897954c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 --2- 172.21.15.44:0/3345558231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x55998978c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 -- 172.21.15.44:0/3345558231 >> 172.21.15.44:0/3345558231 conn(0x559989786c30 msgr2=0x559989787010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 -- 172.21.15.44:0/3345558231 shutdown_connections 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.427+0000 7f050f98b740 1 -- 172.21.15.44:0/3345558231 wait complete. 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 -- start start 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 Processor -- start 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x559989792aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x559989792fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989793520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5599896933e0 con 0x55998978c500 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5599896a67a0 con 0x55998978ce20 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5599897985b0 con 0x559989795a00 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.428+0000 7f050f98b740 10 client.0 _ll_register_callbacks cb 0x5599896a8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x559989792aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x559989792aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57738/0 (socket says 172.21.15.44:57738) 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 -- 172.21.15.44:0/1600609450 learned_addr learned my addr 172.21.15.44:0/1600609450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0506800640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989793520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505400640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x559989792fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 -- 172.21.15.44:0/1600609450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 msgr2=0x559989793520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989793520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 -- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 msgr2=0x559989792fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x559989792fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 -- 172.21.15.44:0/1600609450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599894c77e0 con 0x55998978c500 2024-04-20T11:27:19.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f0505e00640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x559989792aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f04f8004af0 tx=0x7f04f80f0500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9443e19d984c147 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.429+0000 7f04eb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f04eb400640 1 -- 172.21.15.44:0/1600609450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8102030 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f04eb400640 1 -- 172.21.15.44:0/1600609450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f04f80f0700 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5599896fa740 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f04eb400640 1 -- 172.21.15.44:0/1600609450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f04f802cc90 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f050f98b740 10 client.16671 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559989527f70 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f04eb400640 1 -- 172.21.15.44:0/1600609450 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x5599896fa740 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.430+0000 7f04eb400640 1 -- 172.21.15.44:0/1600609450 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f04f80bf040 con 0x55998978c500 2024-04-20T11:27:19.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f050f98b740 10 client.16671 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f050f98b740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f050f98b740 20 client.16671 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f050f98b740 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5599897a5820 0x5599897a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f050f98b740 4 client.16671 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599896f45b0 con 0x5599897a5820 2024-04-20T11:27:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.431+0000 7f0505400640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5599897a5820 0x5599897a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.432+0000 7f0505400640 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5599897a5820 0x5599897a7c10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d08f4035975a921 server_cookie=dad2751072c0b937 in_seq=0 out_seq=0 2024-04-20T11:27:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.432+0000 7f04eb400640 10 client.16671 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f04eb400640 1 -- 172.21.15.44:0/1600609450 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599896f45b0 con 0x5599897a5820 2024-04-20T11:27:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f04eb400640 10 client.16671 handle_command_reply: tid=0 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 client.16671 shutdown 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 msgr2=0x559989792aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x559989792aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f04f8004af0 tx=0x7f04f80f0500 comp rx=0 tx=0).stop 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 shutdown_connections 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5599897a5820 0x5599897a7c10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559989795a00 0x559989793520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55998978ce20 0x559989792fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 --2- 172.21.15.44:0/1600609450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55998978c500 0x559989792aa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f04eb400640 0 client.16671 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 >> 172.21.15.44:0/1600609450 conn(0x559989786c30 msgr2=0x5599897880f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 shutdown_connections 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 1 -- 172.21.15.44:0/1600609450 wait complete. 2024-04-20T11:27:19.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.433+0000 7f050f98b740 20 client.16671 trim_cache size 0 max 16384 2024-04-20T11:27:19.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 Processor -- start 2024-04-20T11:27:19.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 -- start start 2024-04-20T11:27:19.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x5599897951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 0x55998977db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599897a1260 0x559989781880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f05081ab7c0 con 0x559989795a00 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05081ad4e0 con 0x5599897a1260 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.453+0000 7f050f98b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f05081abe90 con 0x55998978ce20 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0505400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 0x55998977db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0506800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599897a1260 0x559989781880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0505400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 0x55998977db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57742/0 (socket says 172.21.15.44:57742) 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0505400640 1 -- 172.21.15.44:0/3574388359 learned_addr learned my addr 172.21.15.44:0/3574388359 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0505e00640 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x5599897951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0506800640 1 -- 172.21.15.44:0/3574388359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 msgr2=0x5599897951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0506800640 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x5599897951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0506800640 1 -- 172.21.15.44:0/3574388359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 msgr2=0x55998977db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0506800640 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 0x55998977db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0506800640 1 -- 172.21.15.44:0/3574388359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599893bf170 con 0x5599897a1260 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.454+0000 7f0505400640 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 0x55998977db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.455+0000 7f0506800640 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599897a1260 0x559989781880 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ca0 tx=0x7f050001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed3063bc531d8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.455+0000 7f04f6600640 1 -- 172.21.15.44:0/3574388359 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500004020 con 0x5599897a1260 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.455+0000 7f04f6600640 1 -- 172.21.15.44:0/3574388359 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05000041c0 con 0x5599897a1260 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.455+0000 7f04f6600640 1 -- 172.21.15.44:0/3574388359 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500011890 con 0x5599897a1260 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.455+0000 7f050f98b740 1 -- 172.21.15.44:0/3574388359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599897a1260 msgr2=0x559989781880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.455+0000 7f050f98b740 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599897a1260 0x559989781880 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ca0 tx=0x7f050001a000 comp rx=0 tx=0).stop 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 -- 172.21.15.44:0/3574388359 shutdown_connections 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599897a1260 0x559989781880 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559989795a00 0x55998977db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 --2- 172.21.15.44:0/3574388359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x5599897951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 -- 172.21.15.44:0/3574388359 >> 172.21.15.44:0/3574388359 conn(0x559989786c30 msgr2=0x559989781170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 -- 172.21.15.44:0/3574388359 shutdown_connections 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 -- 172.21.15.44:0/3574388359 wait complete. 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 -- start start 2024-04-20T11:27:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.456+0000 7f050f98b740 1 Processor -- start 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x559989799890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559989795a00 0x559989799dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 0x55998980abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f05081acd50 con 0x5599897a1260 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05081ac5f0 con 0x559989795a00 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f05081a9730 con 0x55998978ce20 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f050f98b740 10 client.0 _ll_register_callbacks cb 0x7f0500009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f0506800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 0x55998980abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f0506800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 0x55998980abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57744/0 (socket says 172.21.15.44:57744) 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f0506800640 1 -- 172.21.15.44:0/779823995 learned_addr learned my addr 172.21.15.44:0/779823995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f0505e00640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x559989799890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.457+0000 7f0505400640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559989795a00 0x559989799dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0506800640 1 -- 172.21.15.44:0/779823995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 msgr2=0x559989799890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0506800640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x559989799890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0506800640 1 -- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559989795a00 msgr2=0x559989799dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0506800640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559989795a00 0x559989799dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0506800640 1 -- 172.21.15.44:0/779823995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998980b130 con 0x5599897a1260 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0505e00640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x559989799890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f0506800640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 0x55998980abf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f050000a010 tx=0x7f0500003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff91335ead464ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f04eb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f04eb400640 1 -- 172.21.15.44:0/779823995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000037f0 con 0x5599897a1260 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f04eb400640 1 -- 172.21.15.44:0/779823995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f0500003990 con 0x5599897a1260 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.458+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05000041c0 con 0x5599897a1260 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f04eb400640 1 -- 172.21.15.44:0/779823995 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f050003b050 con 0x5599897a1260 2024-04-20T11:27:19.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 10 client.16677 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55998980c440 con 0x5599897a1260 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f04eb400640 1 -- 172.21.15.44:0/779823995 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f05000041c0 con 0x5599897a1260 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f04eb400640 1 -- 172.21.15.44:0/779823995 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f05000049c0 con 0x5599897a1260 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 10 client.16677 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 20 client.16677 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55998980cb70 0x55998980ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 4 client.16677 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.459+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559989670b10 con 0x55998980cb70 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.460+0000 7f0505e00640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55998980cb70 0x55998980ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.460+0000 7f0505e00640 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55998980cb70 0x55998980ef90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d425fa83422f1e9d server_cookie=ac6096b74110847 in_seq=0 out_seq=0 2024-04-20T11:27:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.460+0000 7f04eb400640 10 client.16677 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.461+0000 7f04eb400640 1 -- 172.21.15.44:0/779823995 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559989670b10 con 0x55998980cb70 2024-04-20T11:27:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.461+0000 7f04eb400640 10 client.16677 handle_command_reply: tid=0 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.461+0000 7f050f98b740 1 client.16677 shutdown 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 msgr2=0x55998980abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 0x55998980abf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f050000a010 tx=0x7f0500003ec0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 shutdown_connections 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55998980cb70 0x55998980ef90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5599897a1260 0x55998980abf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559989795a00 0x559989799dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 --2- 172.21.15.44:0/779823995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55998978ce20 0x559989799890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 >> 172.21.15.44:0/779823995 conn(0x559989786c30 msgr2=0x559989792a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f04eb400640 0 client.16677 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 shutdown_connections 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 1 -- 172.21.15.44:0/779823995 wait complete. 2024-04-20T11:27:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.462+0000 7f050f98b740 20 client.16677 trim_cache size 0 max 16384 2024-04-20T11:27:19.465 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:27:19.465 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:27:19.465 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "2c1ee7a4-c56c-42ab-ae13-5b30fce81202", 2024-04-20T11:27:19.465 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:27:19.465 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f04ec0a3d20 msgr2=0x7f04ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f04ec0a3d20 0x7f04ec0a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05000126c0 tx=0x7f0500005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 msgr2=0x7f050819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 0x7f050819e410 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f04f802c590 tx=0x7f04f8004270 comp rx=0 tx=0).stop 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 shutdown_connections 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f04ec0a3d20 0x7f04ec0a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f050810c3e0 0x7f0508198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05081038e0 0x7f050819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 --2- 172.21.15.44:0/3895459296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508102fc0 0x7f050819e410 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 >> 172.21.15.44:0/3895459296 conn(0x7f05080fc7e0 msgr2=0x7f05080fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 shutdown_connections 2024-04-20T11:27:19.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.463+0000 7f0507200640 1 -- 172.21.15.44:0/3895459296 wait complete. 2024-04-20T11:27:19.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:27:19.483 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:27:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 -- 172.21.15.44:0/1587717460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 msgr2=0x7f89e4110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 --2- 172.21.15.44:0/1587717460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e4110f10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009ca0 tx=0x7f89dc02c100 comp rx=0 tx=0).stop 2024-04-20T11:27:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 -- 172.21.15.44:0/1587717460 shutdown_connections 2024-04-20T11:27:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 --2- 172.21.15.44:0/1587717460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e4110f10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 --2- 172.21.15.44:0/1587717460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e4104c20 0x7f89e4108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 --2- 172.21.15.44:0/1587717460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e4104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 -- 172.21.15.44:0/1587717460 >> 172.21.15.44:0/1587717460 conn(0x7f89e40ffad0 msgr2=0x7f89e4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 -- 172.21.15.44:0/1587717460 shutdown_connections 2024-04-20T11:27:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.692+0000 7f89e3200640 1 -- 172.21.15.44:0/1587717460 wait complete. 2024-04-20T11:27:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.693+0000 7f89e3200640 1 Processor -- start 2024-04-20T11:27:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 -- start start 2024-04-20T11:27:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e419a7b0 0x7f89e419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4114530 con 0x7f89e4104270 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41143b0 con 0x7f89e4109340 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41146b0 con 0x7f89e419a7b0 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e41a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57754/0 (socket says 172.21.15.44:57754) 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89e1e00640 1 -- 172.21.15.44:0/1398363644 learned_addr learned my addr 172.21.15.44:0/1398363644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.694+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e2800640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e419a7b0 0x7f89e419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e1e00640 1 -- 172.21.15.44:0/1398363644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e419a7b0 msgr2=0x7f89e419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e1e00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e419a7b0 0x7f89e419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e1e00640 1 -- 172.21.15.44:0/1398363644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 msgr2=0x7f89e41a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e1e00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e41a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e1e00640 1 -- 172.21.15.44:0/1398363644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e419b340 con 0x7f89e4104270 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e41a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.695+0000 7f89e1e00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e41a0640 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e9f0 tx=0x7f89d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.696+0000 7f89daa00640 1 -- 172.21.15.44:0/1398363644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400cd80 con 0x7f89e4104270 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.696+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e419b630 con 0x7f89e4104270 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.696+0000 7f89daa00640 1 -- 172.21.15.44:0/1398363644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89d4010930 con 0x7f89e4104270 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.696+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e41a7860 con 0x7f89e4104270 2024-04-20T11:27:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.696+0000 7f89daa00640 1 -- 172.21.15.44:0/1398363644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4018aa0 con 0x7f89e4104270 2024-04-20T11:27:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.699+0000 7f89eb776740 1 Processor -- start 2024-04-20T11:27:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.699+0000 7f89daa00640 1 -- 172.21.15.44:0/1398363644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f89d4018c40 con 0x7f89e4104270 2024-04-20T11:27:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.699+0000 7f89eb776740 1 -- start start 2024-04-20T11:27:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89eb776740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 0x56021db6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89eb776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89eb776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db75880 0x56021db77c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89eb776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56021dad2150 con 0x56021db6cca0 2024-04-20T11:27:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89eb776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56021da78370 con 0x56021db75880 2024-04-20T11:27:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89eb776740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56021dace110 con 0x56021db6c380 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89e2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 0x56021db6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89e2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 0x56021db6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:55798/0 (socket says 172.21.15.44:55798) 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89e2800640 1 -- 172.21.15.44:0/1139535909 learned_addr learned my addr 172.21.15.44:0/1139535909 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db75880 0x56021db77c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89e2800640 1 -- 172.21.15.44:0/1139535909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db75880 msgr2=0x56021db77c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.700+0000 7f89e2800640 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db75880 0x56021db77c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89e2800640 1 -- 172.21.15.44:0/1139535909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 msgr2=0x56021db75340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89e2800640 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db75340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89e2800640 1 -- 172.21.15.44:0/1139535909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021d8ae7d0 con 0x56021db6c380 2024-04-20T11:27:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db75880 0x56021db77c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89e2800640 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 0x56021db6c760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89dc02c710 tx=0x7f89dc005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dccef7d3bcc3c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89daa00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f89c80a3d20 0x7f89c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89d3000640 1 -- 172.21.15.44:0/1139535909 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc043070 con 0x56021db6c380 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89d3000640 1 -- 172.21.15.44:0/1139535909 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89dc032d40 con 0x56021db6c380 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89d3000640 1 -- 172.21.15.44:0/1139535909 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc039040 con 0x56021db6c380 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89daa00640 1 -- 172.21.15.44:0/1398363644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f89d40d0b00 con 0x7f89e4104270 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.701+0000 7f89e1e00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f89c80a3d20 0x7f89c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 -- 172.21.15.44:0/1139535909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 msgr2=0x56021db6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 0x56021db6c760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89dc02c710 tx=0x7f89dc005f80 comp rx=0 tx=0).stop 2024-04-20T11:27:19.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89e1e00640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f89c80a3d20 0x7f89c80a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x56021db6be50 tx=0x7f89d40d45b0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 -- 172.21.15.44:0/1139535909 shutdown_connections 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db75880 0x56021db77c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 --2- 172.21.15.44:0/1139535909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6c380 0x56021db6c760 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 -- 172.21.15.44:0/1139535909 >> 172.21.15.44:0/1139535909 conn(0x56021db66ab0 msgr2=0x56021db66e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 -- 172.21.15.44:0/1139535909 shutdown_connections 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.702+0000 7f89eb776740 1 -- 172.21.15.44:0/1139535909 wait complete. 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 -- start start 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 Processor -- start 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 0x56021db72900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6cca0 0x56021db72e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db73380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56021db78610 con 0x56021db6c380 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56021da8b740 con 0x56021db6cca0 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.703+0000 7f89eb776740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56021db78790 con 0x56021db75880 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89eb776740 10 client.0 _ll_register_callbacks cb 0x56021da86b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 0x56021db72900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 0x56021db72900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57766/0 (socket says 172.21.15.44:57766) 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 -- 172.21.15.44:0/1825116029 learned_addr learned my addr 172.21.15.44:0/1825116029 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6cca0 0x56021db72e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db73380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 -- 172.21.15.44:0/1825116029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 msgr2=0x56021db73380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db73380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 -- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6cca0 msgr2=0x56021db72e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6cca0 0x56021db72e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89e2800640 1 -- 172.21.15.44:0/1825116029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021d7e43f0 con 0x56021db6c380 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.704+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6cca0 0x56021db72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89e2800640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 0x56021db72900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89dc006010 tx=0x7f89dc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a7ec608e801056 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89c7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89c7400640 1 -- 172.21.15.44:0/1825116029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc053060 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89c7400640 1 -- 172.21.15.44:0/1825116029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f89dc032a50 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89dc032d40 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89c7400640 1 -- 172.21.15.44:0/1825116029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f89dc039040 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89eb776740 10 client.16689 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.705+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56021d7a40f0 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89c7400640 1 -- 172.21.15.44:0/1825116029 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89dc032d40 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89c7400640 1 -- 172.21.15.44:0/1825116029 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f89dc056030 con 0x56021db6c380 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89eb776740 10 client.16689 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89eb776740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89eb776740 20 client.16689 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89eb776740 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021db85a80 0x56021db87e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89eb776740 4 client.16689 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:19.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.706+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021dadfb70 con 0x56021db85a80 2024-04-20T11:27:19.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.707+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021db85a80 0x56021db87e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.707+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021db85a80 0x56021db87e70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede1fd68e803c17f server_cookie=b48a8945944d04f8 in_seq=0 out_seq=0 2024-04-20T11:27:19.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.707+0000 7f89c7400640 10 client.16689 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.708+0000 7f89c7400640 1 -- 172.21.15.44:0/1825116029 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56021dadfb70 con 0x56021db85a80 2024-04-20T11:27:19.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.708+0000 7f89c7400640 10 client.16689 handle_command_reply: tid=0 2024-04-20T11:27:19.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.708+0000 7f89eb776740 1 client.16689 shutdown 2024-04-20T11:27:19.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 msgr2=0x56021db72900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 0x56021db72900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89dc006010 tx=0x7f89dc003f10 comp rx=0 tx=0).stop 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 shutdown_connections 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021db85a80 0x56021db87e70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db73380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6cca0 0x56021db72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 --2- 172.21.15.44:0/1825116029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6c380 0x56021db72900 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89c7400640 0 client.16689 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 >> 172.21.15.44:0/1825116029 conn(0x56021db66ab0 msgr2=0x56021db68800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 shutdown_connections 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 1 -- 172.21.15.44:0/1825116029 wait complete. 2024-04-20T11:27:19.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.709+0000 7f89eb776740 20 client.16689 trim_cache size 0 max 16384 2024-04-20T11:27:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 Processor -- start 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 -- start start 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db75030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db71840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41afcd0 con 0x56021db6cca0 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41b19f0 con 0x56021db6c380 2024-04-20T11:27:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.731+0000 7f89eb776740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41b03a0 con 0x56021db75880 2024-04-20T11:27:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db75030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db75030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32898/0 (socket says 172.21.15.44:32898) 2024-04-20T11:27:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89e2800640 1 -- 172.21.15.44:0/1418983684 learned_addr learned my addr 172.21.15.44:0/1418983684 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89e2800640 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db71840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 -- 172.21.15.44:0/1418983684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 msgr2=0x56021db71840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db71840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 -- 172.21.15.44:0/1418983684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 msgr2=0x56021db75030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db75030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 -- 172.21.15.44:0/1418983684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021db665e0 con 0x56021db6cca0 2024-04-20T11:27:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89e2800640 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db75030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.732+0000 7f89dbe00640 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db814d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x56021db72160 tx=0x7f89cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7855446e6e77ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89d2600640 1 -- 172.21.15.44:0/1418983684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc012110 con 0x56021db6cca0 2024-04-20T11:27:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89d2600640 1 -- 172.21.15.44:0/1418983684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89cc005520 con 0x56021db6cca0 2024-04-20T11:27:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89d2600640 1 -- 172.21.15.44:0/1418983684 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc022170 con 0x56021db6cca0 2024-04-20T11:27:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 -- 172.21.15.44:0/1418983684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 msgr2=0x56021db814d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db814d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x56021db72160 tx=0x7f89cc005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 -- 172.21.15.44:0/1418983684 shutdown_connections 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db75880 0x56021db71840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db6cca0 0x56021db814d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 --2- 172.21.15.44:0/1418983684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db75030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.733+0000 7f89eb776740 1 -- 172.21.15.44:0/1418983684 >> 172.21.15.44:0/1418983684 conn(0x56021db66ab0 msgr2=0x56021db73150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.734+0000 7f89eb776740 1 -- 172.21.15.44:0/1418983684 shutdown_connections 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.734+0000 7f89eb776740 1 -- 172.21.15.44:0/1418983684 wait complete. 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.734+0000 7f89eb776740 1 -- start start 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.734+0000 7f89eb776740 1 Processor -- start 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db7de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6cca0 0x56021db6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db75880 0x56021db6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41b1260 con 0x56021db75880 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41b0b00 con 0x56021db6c380 2024-04-20T11:27:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41adc40 con 0x56021db6cca0 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89eb776740 10 client.0 _ll_register_callbacks cb 0x7f89d40d4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db7de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db7de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32902/0 (socket says 172.21.15.44:32902) 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89e2800640 1 -- 172.21.15.44:0/2559989562 learned_addr learned my addr 172.21.15.44:0/2559989562 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89e2800640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db75880 0x56021db6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.735+0000 7f89dbe00640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6cca0 0x56021db6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89e2800640 1 -- 172.21.15.44:0/2559989562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6cca0 msgr2=0x56021db6f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89e2800640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6cca0 0x56021db6f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89e2800640 1 -- 172.21.15.44:0/2559989562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db75880 msgr2=0x56021db6f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89e2800640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db75880 0x56021db6f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89e2800640 1 -- 172.21.15.44:0/2559989562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021db700a0 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89e2800640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db7de40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f89dc048b40 tx=0x7f89dc003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982c14068aff8503 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89c7400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89c7400640 1 -- 172.21.15.44:0/2559989562 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc043040 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89c7400640 1 -- 172.21.15.44:0/2559989562 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f89dc032850 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.736+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89cc005520 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89c7400640 1 -- 172.21.15.44:0/2559989562 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f89dc053030 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 10 client.26825 fetch_fsmap learned FSMap version 40 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56021dbebd80 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89c7400640 1 -- 172.21.15.44:0/2559989562 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89cc005520 con 0x56021db6c380 2024-04-20T11:27:19.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89c7400640 1 -- 172.21.15.44:0/2559989562 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) ==== 8419+0+0 (secure 0 0 0) 0x7f89dc060040 con 0x56021db6c380 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 10 client.26825 fetch_fsmap finished waiting for FSMap version 40 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 20 client.26825 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021dbec470 0x56021dbee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 4 client.26825 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56021dad99e0 con 0x56021dbec470 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.737+0000 7f89dbe00640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021dbec470 0x56021dbee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.738+0000 7f89dbe00640 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021dbec470 0x56021dbee890 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4226a248e4374ce8 server_cookie=19a13a21695a4e20 in_seq=0 out_seq=0 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.738+0000 7f89c7400640 10 client.26825 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.738+0000 7f89c7400640 1 -- 172.21.15.44:0/2559989562 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56021dad99e0 con 0x56021dbec470 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.738+0000 7f89c7400640 10 client.26825 handle_command_reply: tid=0 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.738+0000 7f89eb776740 1 client.26825 shutdown 2024-04-20T11:27:19.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 msgr2=0x56021db7de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db7de40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f89dc048b40 tx=0x7f89dc003e70 comp rx=0 tx=0).stop 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 shutdown_connections 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x56021dbec470 0x56021dbee890 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56021db75880 0x56021db6f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56021db6cca0 0x56021db6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89c7400640 0 client.26825 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 --2- 172.21.15.44:0/2559989562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56021db6c380 0x56021db7de40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 >> 172.21.15.44:0/2559989562 conn(0x56021db66ab0 msgr2=0x56021db65280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 shutdown_connections 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 1 -- 172.21.15.44:0/2559989562 wait complete. 2024-04-20T11:27:19.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.739+0000 7f89eb776740 20 client.26825 trim_cache size 0 max 16384 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: "2c1ee7a4-c56c-42ab-ae13-5b30fce81202": { 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "2c1ee7a4-c56c-42ab-ae13-5b30fce81202", 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:27:19.743 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:27:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f89c80a3d20 msgr2=0x7f89c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f89c80a3d20 0x7f89c80a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x56021db6be50 tx=0x7f89d40d45b0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 msgr2=0x7f89e41a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e41a0640 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e9f0 tx=0x7f89d400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 shutdown_connections 2024-04-20T11:27:19.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e419a7b0 0x7f89e419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e4109340 0x7f89e41a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f89c80a3d20 0x7f89c80a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 --2- 172.21.15.44:0/1398363644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89e4104270 0x7f89e41a0640 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 >> 172.21.15.44:0/1398363644 conn(0x7f89e40ffad0 msgr2=0x7f89e4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 shutdown_connections 2024-04-20T11:27:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:19.740+0000 7f89e3200640 1 -- 172.21.15.44:0/1398363644 wait complete. 2024-04-20T11:27:19.760 INFO:tasks.cephfs.filesystem:scrub status for tag:2c1ee7a4-c56c-42ab-ae13-5b30fce81202 - {'path': '/', 'tag': '2c1ee7a4-c56c-42ab-ae13-5b30fce81202', 'options': 'recursive,force'} 2024-04-20T11:27:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:19 smithi078 ceph-mon[21464]: pgmap v144: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-20T11:27:19.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:19 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:27:19.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:19 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:19.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:19 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:27:19.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:19 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:19 smithi044 ceph-mon[16855]: pgmap v144: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-20T11:27:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:19 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:27:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:19 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:19 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:27:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:19 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:19 smithi165 ceph-mon[21975]: pgmap v144: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-20T11:27:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:19 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:27:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:19 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:19 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:27:19.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:19 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:21 smithi078 ceph-mon[21464]: pgmap v145: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:27:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:21 smithi044 ceph-mon[16855]: pgmap v145: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:27:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:21 smithi165 ceph-mon[21975]: pgmap v145: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:27:22.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:22 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:27:22.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:22 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:27:22.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:22 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:27:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:23 smithi078 ceph-mon[21464]: pgmap v146: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:27:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:23 smithi044 ceph-mon[16855]: pgmap v146: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:27:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:23 smithi165 ceph-mon[21975]: pgmap v146: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:27:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:25 smithi078 ceph-mon[21464]: pgmap v147: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:25 smithi044 ceph-mon[16855]: pgmap v147: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:25 smithi165 ceph-mon[21975]: pgmap v147: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:27 smithi078 ceph-mon[21464]: pgmap v148: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:27 smithi044 ceph-mon[16855]: pgmap v148: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:27 smithi165 ceph-mon[21975]: pgmap v148: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:29 smithi078 ceph-mon[21464]: pgmap v149: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:29 smithi044 ceph-mon[16855]: pgmap v149: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:29 smithi165 ceph-mon[21975]: pgmap v149: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:30.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:30 smithi078 ceph-mon[21464]: pgmap v150: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:30.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:30 smithi044 ceph-mon[16855]: pgmap v150: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:30.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:30 smithi165 ceph-mon[21975]: pgmap v150: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:27:33.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:33 smithi078 ceph-mon[21464]: pgmap v151: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:33.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:33 smithi044 ceph-mon[16855]: pgmap v151: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:33.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:33 smithi165 ceph-mon[21975]: pgmap v151: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:35 smithi078 ceph-mon[21464]: pgmap v152: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:35.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:35 smithi044 ceph-mon[16855]: pgmap v152: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:35.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:35 smithi165 ceph-mon[21975]: pgmap v152: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:37.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:37 smithi078 ceph-mon[21464]: pgmap v153: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:37 smithi044 ceph-mon[16855]: pgmap v153: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:37 smithi165 ceph-mon[21975]: pgmap v153: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:39.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:39 smithi078 ceph-mon[21464]: pgmap v154: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:39.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:39 smithi044 ceph-mon[16855]: pgmap v154: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:39 smithi165 ceph-mon[21975]: pgmap v154: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:41 smithi078 ceph-mon[21464]: pgmap v155: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:41.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:41 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:41.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:41 smithi044 ceph-mon[16855]: pgmap v155: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:41.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:41 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:41 smithi165 ceph-mon[21975]: pgmap v155: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:41 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:42.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:42 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:42.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:42 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:42.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:42 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:43.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:43 smithi078 ceph-mon[21464]: pgmap v156: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:43 smithi044 ceph-mon[16855]: pgmap v156: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:43 smithi165 ceph-mon[21975]: pgmap v156: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:45.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:45 smithi078 ceph-mon[21464]: pgmap v157: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:45.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:45 smithi044 ceph-mon[16855]: pgmap v157: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:45 smithi165 ceph-mon[21975]: pgmap v157: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:46.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:46 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:46.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:46 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:46 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:47.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:47 smithi078 ceph-mon[21464]: pgmap v158: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:47 smithi044 ceph-mon[16855]: pgmap v158: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:47 smithi165 ceph-mon[21975]: pgmap v158: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:49 smithi078 ceph-mon[21464]: pgmap v159: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:49.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:49 smithi044 ceph-mon[16855]: pgmap v159: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:49 smithi165 ceph-mon[21975]: pgmap v159: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:49.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:27:49.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 -- 172.21.15.44:0/2203627821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3408104280 msgr2=0x7f3408104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 --2- 172.21.15.44:0/2203627821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3408104280 0x7f3408104660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009900 tx=0x7f33fc02bc30 comp rx=0 tx=0).stop 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 -- 172.21.15.44:0/2203627821 shutdown_connections 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 --2- 172.21.15.44:0/2203627821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 0x7f3408110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 --2- 172.21.15.44:0/2203627821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104c30 0x7f3408108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 --2- 172.21.15.44:0/2203627821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3408104280 0x7f3408104660 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 -- 172.21.15.44:0/2203627821 >> 172.21.15.44:0/2203627821 conn(0x7f34080ffaa0 msgr2=0x7f3408101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 -- 172.21.15.44:0/2203627821 shutdown_connections 2024-04-20T11:27:49.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.989+0000 7f340e800640 1 -- 172.21.15.44:0/2203627821 wait complete. 2024-04-20T11:27:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 Processor -- start 2024-04-20T11:27:49.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 -- start start 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104280 0x7f34081a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 0x7f34081a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f340819a8d0 0x7f340819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34081144d0 con 0x7f340819a8d0 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3408114350 con 0x7f3408109350 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.990+0000 7f340e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3408114650 con 0x7f3408104280 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f340819a8d0 0x7f340819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f340819a8d0 0x7f340819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56604/0 (socket says 172.21.15.44:56604) 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104280 0x7f34081a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340de00640 1 -- 172.21.15.44:0/752830839 learned_addr learned my addr 172.21.15.44:0/752830839 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340ca00640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 0x7f34081a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340d400640 1 -- 172.21.15.44:0/752830839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 msgr2=0x7f34081a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340d400640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 0x7f34081a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340d400640 1 -- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f340819a8d0 msgr2=0x7f340819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340d400640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f340819a8d0 0x7f340819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340d400640 1 -- 172.21.15.44:0/752830839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340819b3f0 con 0x7f3408104280 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.991+0000 7f340ca00640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 0x7f34081a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.992+0000 7f340d400640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104280 0x7f34081a0720 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f33fc036040 tx=0x7f33fc02dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.992+0000 7f3406000640 1 -- 172.21.15.44:0/752830839 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc043020 con 0x7f3408104280 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.992+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340819b680 con 0x7f3408104280 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.992+0000 7f3406000640 1 -- 172.21.15.44:0/752830839 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33fc004e50 con 0x7f3408104280 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.992+0000 7f3406000640 1 -- 172.21.15.44:0/752830839 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003970 con 0x7f3408104280 2024-04-20T11:27:49.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.992+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34081a78d0 con 0x7f3408104280 2024-04-20T11:27:49.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.995+0000 7f3406000640 1 -- 172.21.15.44:0/752830839 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f33fc046020 con 0x7f3408104280 2024-04-20T11:27:49.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 Processor -- start 2024-04-20T11:27:49.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 -- start start 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf130760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf139340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf13bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ebf096150 con 0x558ebf130ca0 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ebf03c370 con 0x558ebf139880 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.996+0000 7f3412bcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ebf092110 con 0x558ebf130380 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56616/0 (socket says 172.21.15.44:56616) 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 -- 172.21.15.44:0/2167913114 learned_addr learned my addr 172.21.15.44:0/2167913114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340de00640 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf130760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340ca00640 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf13bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 -- 172.21.15.44:0/2167913114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 msgr2=0x558ebf130760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf130760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 -- 172.21.15.44:0/2167913114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 msgr2=0x558ebf13bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf13bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 -- 172.21.15.44:0/2167913114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebee727d0 con 0x558ebf130ca0 2024-04-20T11:27:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f3406000640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f33f00a3d20 0x7f33f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:49.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.997+0000 7f340d400640 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf139340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0eff20 tx=0x7f33fc0f8a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e102c0b17381552 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.998+0000 7f3406000640 1 -- 172.21.15.44:0/752830839 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f33fc0f2770 con 0x7f3408104280 2024-04-20T11:27:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.998+0000 7f340de00640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f33f00a3d20 0x7f33f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.998+0000 7f33fa600640 1 -- 172.21.15.44:0/2167913114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0baad0 con 0x558ebf130ca0 2024-04-20T11:27:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.998+0000 7f33fa600640 1 -- 172.21.15.44:0/2167913114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33fc0bef30 con 0x558ebf130ca0 2024-04-20T11:27:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.998+0000 7f33fa600640 1 -- 172.21.15.44:0/2167913114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0be190 con 0x558ebf130ca0 2024-04-20T11:27:50.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.998+0000 7f3412bcf740 1 -- 172.21.15.44:0/2167913114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 msgr2=0x558ebf139340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf139340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0eff20 tx=0x7f33fc0f8a60 comp rx=0 tx=0).stop 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 -- 172.21.15.44:0/2167913114 shutdown_connections 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf13bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf139340 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 --2- 172.21.15.44:0/2167913114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf130760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 -- 172.21.15.44:0/2167913114 >> 172.21.15.44:0/2167913114 conn(0x558ebf12aab0 msgr2=0x558ebf12ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 -- 172.21.15.44:0/2167913114 shutdown_connections 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f340de00640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f33f00a3d20 0x7f33f00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34000099a0 tx=0x7f3400009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 -- 172.21.15.44:0/2167913114 wait complete. 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:49.999+0000 7f3412bcf740 1 -- start start 2024-04-20T11:27:50.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 Processor -- start 2024-04-20T11:27:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf136950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf130ca0 0x558ebf136e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf139880 0x558ebf1373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ebf13c430 con 0x558ebf130380 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ebf04f740 con 0x558ebf130ca0 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ebf13c5b0 con 0x558ebf139880 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f3412bcf740 10 client.0 _ll_register_callbacks cb 0x558ebf04ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.000+0000 7f340d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf136950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56628/0 (socket says 172.21.15.44:56628) 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 -- 172.21.15.44:0/1515201210 learned_addr learned my addr 172.21.15.44:0/1515201210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340ca00640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf130ca0 0x558ebf136e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340de00640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf139880 0x558ebf1373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 -- 172.21.15.44:0/1515201210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf139880 msgr2=0x558ebf1373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf139880 0x558ebf1373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 -- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf130ca0 msgr2=0x558ebf136e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf130ca0 0x558ebf136e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 -- 172.21.15.44:0/1515201210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebeda83f0 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340ca00640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf130ca0 0x558ebf136e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f340d400640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf136950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0be9f0 tx=0x7f33fc0f8370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9058099eca03a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.001+0000 7f33eea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f33eea00640 1 -- 172.21.15.44:0/1515201210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0be370 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f33eea00640 1 -- 172.21.15.44:0/1515201210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f33fc005390 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33fc0bef30 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f33eea00640 1 -- 172.21.15.44:0/1515201210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f33fc0be600 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f3412bcf740 10 client.16707 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558ebed680f0 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f33eea00640 1 -- 172.21.15.44:0/1515201210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33fc0bef30 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f33eea00640 1 -- 172.21.15.44:0/1515201210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7f33fc0f0150 con 0x558ebf130380 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f3412bcf740 10 client.16707 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:50.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f3412bcf740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.002+0000 7f3412bcf740 20 client.16707 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.003+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf149850 0x558ebf14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.003+0000 7f3412bcf740 4 client.16707 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.003+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ebf0a3b70 con 0x558ebf149850 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.003+0000 7f340ca00640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf149850 0x558ebf14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.004+0000 7f340ca00640 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf149850 0x558ebf14bc60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da4a9d534826ccf server_cookie=ae64d183805d9a96 in_seq=0 out_seq=0 2024-04-20T11:27:50.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.004+0000 7f33eea00640 10 client.16707 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f33eea00640 1 -- 172.21.15.44:0/1515201210 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ebf0a3b70 con 0x558ebf149850 2024-04-20T11:27:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f33eea00640 10 client.16707 handle_command_reply: tid=0 2024-04-20T11:27:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 client.16707 shutdown 2024-04-20T11:27:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 msgr2=0x558ebf136950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf136950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0be9f0 tx=0x7f33fc0f8370 comp rx=0 tx=0).stop 2024-04-20T11:27:50.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 shutdown_connections 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf149850 0x558ebf14bc60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf139880 0x558ebf1373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf130ca0 0x558ebf136e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1515201210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf136950 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f33eea00640 0 client.16707 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 >> 172.21.15.44:0/1515201210 conn(0x558ebf12aab0 msgr2=0x558ebf12c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 shutdown_connections 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 1 -- 172.21.15.44:0/1515201210 wait complete. 2024-04-20T11:27:50.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.005+0000 7f3412bcf740 20 client.16707 trim_cache size 0 max 16384 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 Processor -- start 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 -- start start 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf139080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf1451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf12c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34081afc40 con 0x558ebf130ca0 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34081b18d0 con 0x558ebf139880 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.025+0000 7f3412bcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34081b0310 con 0x558ebf130380 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf1451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf12c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf1451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56632/0 (socket says 172.21.15.44:56632) 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 -- 172.21.15.44:0/4163374378 learned_addr learned my addr 172.21.15.44:0/4163374378 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340d400640 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 -- 172.21.15.44:0/4163374378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 msgr2=0x558ebf139080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf139080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 -- 172.21.15.44:0/4163374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 msgr2=0x558ebf12c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf12c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340ca00640 1 -- 172.21.15.44:0/4163374378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebf134ec0 con 0x558ebf130ca0 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340de00640 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf12c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.026+0000 7f340d400640 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.027+0000 7f340ca00640 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf1451f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009540 tx=0x7f33f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14ff385aabd7ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.027+0000 7f33f9c00640 1 -- 172.21.15.44:0/4163374378 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f400fc20 con 0x558ebf130ca0 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.027+0000 7f33f9c00640 1 -- 172.21.15.44:0/4163374378 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33f400fdc0 con 0x558ebf130ca0 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.027+0000 7f33f9c00640 1 -- 172.21.15.44:0/4163374378 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4004e10 con 0x558ebf130ca0 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.027+0000 7f3412bcf740 1 -- 172.21.15.44:0/4163374378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 msgr2=0x558ebf1451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf1451f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009540 tx=0x7f33f4005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 -- 172.21.15.44:0/4163374378 shutdown_connections 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf12c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130ca0 0x558ebf1451f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 --2- 172.21.15.44:0/4163374378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130380 0x558ebf139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 -- 172.21.15.44:0/4163374378 >> 172.21.15.44:0/4163374378 conn(0x558ebf12aab0 msgr2=0x558ebf12be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 -- 172.21.15.44:0/4163374378 shutdown_connections 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 -- 172.21.15.44:0/4163374378 wait complete. 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 -- start start 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.028+0000 7f3412bcf740 1 Processor -- start 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf133840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130ca0 0x558ebf133d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf1342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34081b1140 con 0x558ebf130380 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34081b0a40 con 0x558ebf139880 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34081adbd0 con 0x558ebf130ca0 2024-04-20T11:27:50.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f3412bcf740 10 client.0 _ll_register_callbacks cb 0x7f33f4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf133840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56648/0 (socket says 172.21.15.44:56648) 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 -- 172.21.15.44:0/1613152857 learned_addr learned my addr 172.21.15.44:0/1613152857 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf1342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340ca00640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130ca0 0x558ebf133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 -- 172.21.15.44:0/1613152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130ca0 msgr2=0x558ebf133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130ca0 0x558ebf133d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 -- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 msgr2=0x558ebf1342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.029+0000 7f340d400640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf1342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f340d400640 1 -- 172.21.15.44:0/1613152857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebf1aefe0 con 0x558ebf130380 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f340ca00640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130ca0 0x558ebf133d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f340de00640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf1342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f340d400640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf133840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006010 tx=0x7f33fc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edd625d5554fae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f33eea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f33eea00640 1 -- 172.21.15.44:0/1613152857 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0be370 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f33eea00640 1 -- 172.21.15.44:0/1613152857 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f33fc0c7010 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f400fdc0 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f33eea00640 1 -- 172.21.15.44:0/1613152857 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f33fc0be190 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f3412bcf740 10 client.16713 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.030+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558ebf1aefe0 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f33eea00640 1 -- 172.21.15.44:0/1613152857 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33f400fdc0 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f33eea00640 1 -- 172.21.15.44:0/1613152857 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7f33fc0f00c0 con 0x558ebf130380 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f3412bcf740 10 client.16713 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f3412bcf740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f3412bcf740 20 client.16713 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf1b07c0 0x558ebf1b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f3412bcf740 4 client.16713 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:27:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ebf09d9e0 con 0x558ebf1b07c0 2024-04-20T11:27:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.031+0000 7f340ca00640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf1b07c0 0x558ebf1b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f340ca00640 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf1b07c0 0x558ebf1b2be0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31a11a01c91c3bf server_cookie=9c4b3de02e6c3042 in_seq=0 out_seq=0 2024-04-20T11:27:50.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f33eea00640 10 client.16713 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f33eea00640 1 -- 172.21.15.44:0/1613152857 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558ebf09d9e0 con 0x558ebf1b07c0 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f33eea00640 10 client.16713 handle_command_reply: tid=0 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 client.16713 shutdown 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 msgr2=0x558ebf133840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf133840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006010 tx=0x7f33fc005be0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 shutdown_connections 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558ebf1b07c0 0x558ebf1b2be0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ebf139880 0x558ebf1342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ebf130ca0 0x558ebf133d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 --2- 172.21.15.44:0/1613152857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ebf130380 0x558ebf133840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f33eea00640 0 client.16713 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 >> 172.21.15.44:0/1613152857 conn(0x558ebf12aab0 msgr2=0x558ebf12c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.032+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 shutdown_connections 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f3412bcf740 1 -- 172.21.15.44:0/1613152857 wait complete. 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f3412bcf740 20 client.16713 trim_cache size 0 max 16384 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:27:50.038 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:27:50.039 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:27:50.039 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:27:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f33f00a3d20 msgr2=0x7f33f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f33f00a3d20 0x7f33f00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34000099a0 tx=0x7f3400009500 comp rx=0 tx=0).stop 2024-04-20T11:27:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104280 msgr2=0x7f34081a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104280 0x7f34081a0720 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f33fc036040 tx=0x7f33fc02dbf0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 shutdown_connections 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f33f00a3d20 0x7f33f00a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f340819a8d0 0x7f340819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3408109350 0x7f34081a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 --2- 172.21.15.44:0/752830839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408104280 0x7f34081a0720 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 >> 172.21.15.44:0/752830839 conn(0x7f34080ffaa0 msgr2=0x7f3408100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.033+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 shutdown_connections 2024-04-20T11:27:50.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.034+0000 7f340e800640 1 -- 172.21.15.44:0/752830839 wait complete. 2024-04-20T11:27:50.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:27:50.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 -- 172.21.15.44:0/2313555369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 msgr2=0x7f1d301045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 --2- 172.21.15.44:0/2313555369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301045b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009900 tx=0x7f1d2802bc50 comp rx=0 tx=0).stop 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 -- 172.21.15.44:0/2313555369 shutdown_connections 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 --2- 172.21.15.44:0/2313555369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d301092a0 0x7f1d30110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 --2- 172.21.15.44:0/2313555369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d30104b80 0x7f1d30108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 --2- 172.21.15.44:0/2313555369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301045b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 -- 172.21.15.44:0/2313555369 >> 172.21.15.44:0/2313555369 conn(0x7f1d300ff9f0 msgr2=0x7f1d30101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 -- 172.21.15.44:0/2313555369 shutdown_connections 2024-04-20T11:27:50.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.259+0000 7f1d35800640 1 -- 172.21.15.44:0/2313555369 wait complete. 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.260+0000 7f1d35800640 1 Processor -- start 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.260+0000 7f1d35800640 1 -- start start 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d35800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d35800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d301092a0 0x7f1d301a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d35800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d3019a750 0x7f1d3019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d35800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30114470 con 0x7f1d301041d0 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d35800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301142f0 con 0x7f1d3019a750 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d35800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301145f0 con 0x7f1d301092a0 2024-04-20T11:27:50.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56668/0 (socket says 172.21.15.44:56668) 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 -- 172.21.15.44:0/3741943062 learned_addr learned my addr 172.21.15.44:0/3741943062 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d34e00640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d3019a750 0x7f1d3019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2f400640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d301092a0 0x7f1d301a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 -- 172.21.15.44:0/3741943062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d301092a0 msgr2=0x7f1d301a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d301092a0 0x7f1d301a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 -- 172.21.15.44:0/3741943062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d3019a750 msgr2=0x7f1d3019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d3019a750 0x7f1d3019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d2fe00640 1 -- 172.21.15.44:0/3741943062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3019b350 con 0x7f1d301041d0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.261+0000 7f1d34e00640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d3019a750 0x7f1d3019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.262+0000 7f1d2fe00640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301a0690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005ab0 tx=0x7f1d2802dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.262+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28043020 con 0x7f1d301041d0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.262+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d28004250 con 0x7f1d301041d0 2024-04-20T11:27:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.262+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3019b5e0 con 0x7f1d301041d0 2024-04-20T11:27:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.262+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28003b30 con 0x7f1d301041d0 2024-04-20T11:27:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.262+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a7780 con 0x7f1d301041d0 2024-04-20T11:27:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.264+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d301056a0 con 0x7f1d301041d0 2024-04-20T11:27:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.265+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1d28046020 con 0x7f1d301041d0 2024-04-20T11:27:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.267+0000 7f1d2cc00640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d140a3d20 0x7f1d140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.267+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f1d280f26a0 con 0x7f1d301041d0 2024-04-20T11:27:50.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.267+0000 7f1d2f400640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d140a3d20 0x7f1d140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.268+0000 7f1d2f400640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d140a3d20 0x7f1d140a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d180095f0 tx=0x7f1d18005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.272+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1d2804d020 con 0x7f1d301041d0 2024-04-20T11:27:50.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.523+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d300690f0 con 0x7f1d301041d0 2024-04-20T11:27:50.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.525+0000 7f1d2cc00640 1 -- 172.21.15.44:0/3741943062 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11962 (secure 0 0 0) 0x7f1d300690f0 con 0x7f1d301041d0 2024-04-20T11:27:50.526 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:27:50.526 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:27:45.082111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:27:50.526 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d140a3d20 msgr2=0x7f1d140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d140a3d20 0x7f1d140a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d180095f0 tx=0x7f1d18005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 msgr2=0x7f1d301a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301a0690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005ab0 tx=0x7f1d2802dd20 comp rx=0 tx=0).stop 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 shutdown_connections 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d3019a750 0x7f1d3019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d301092a0 0x7f1d301a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f1d140a3d20 0x7f1d140a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 --2- 172.21.15.44:0/3741943062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d301041d0 0x7f1d301a0690 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 >> 172.21.15.44:0/3741943062 conn(0x7f1d300ff9f0 msgr2=0x7f1d30101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 shutdown_connections 2024-04-20T11:27:50.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.528+0000 7f1d35800640 1 -- 172.21.15.44:0/3741943062 wait complete. 2024-04-20T11:27:50.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:27:50.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 -- 172.21.15.44:0/2159756979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42481021a0 msgr2=0x7f4248102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 --2- 172.21.15.44:0/2159756979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42481021a0 0x7f4248102580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802bc40 comp rx=0 tx=0).stop 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 -- 172.21.15.44:0/2159756979 shutdown_connections 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 --2- 172.21.15.44:0/2159756979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248107270 0x7f424810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 --2- 172.21.15.44:0/2159756979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4248102b50 0x7f4248106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 --2- 172.21.15.44:0/2159756979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42481021a0 0x7f4248102580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 -- 172.21.15.44:0/2159756979 >> 172.21.15.44:0/2159756979 conn(0x7f42480793b0 msgr2=0x7f42480797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 -- 172.21.15.44:0/2159756979 shutdown_connections 2024-04-20T11:27:50.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.770+0000 7f424f400640 1 -- 172.21.15.44:0/2159756979 wait complete. 2024-04-20T11:27:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.771+0000 7f424f400640 1 Processor -- start 2024-04-20T11:27:50.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.771+0000 7f424f400640 1 -- start start 2024-04-20T11:27:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 0x7f4248074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4248107270 0x7f42480747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4248074ce0 0x7f42481a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4248112460 con 0x7f42481021a0 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424f400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42481122e0 con 0x7f4248074ce0 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424f400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42481125e0 con 0x7f4248107270 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 0x7f4248074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 0x7f4248074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56678/0 (socket says 172.21.15.44:56678) 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424ca00640 1 -- 172.21.15.44:0/959140670 learned_addr learned my addr 172.21.15.44:0/959140670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f424d400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4248074ce0 0x7f42481a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.772+0000 7f4247e00640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4248107270 0x7f42480747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424ca00640 1 -- 172.21.15.44:0/959140670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4248107270 msgr2=0x7f42480747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424ca00640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4248107270 0x7f42480747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424ca00640 1 -- 172.21.15.44:0/959140670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4248074ce0 msgr2=0x7f42481a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424ca00640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4248074ce0 0x7f42481a6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424ca00640 1 -- 172.21.15.44:0/959140670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42481a7210 con 0x7f42481021a0 2024-04-20T11:27:50.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424d400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4248074ce0 0x7f42481a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424ca00640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 0x7f4248074260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c00 tx=0x7f423802dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f4245600640 1 -- 172.21.15.44:0/959140670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238043020 con 0x7f42481021a0 2024-04-20T11:27:50.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42481a74a0 con 0x7f42481021a0 2024-04-20T11:27:50.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f4245600640 1 -- 172.21.15.44:0/959140670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4238004310 con 0x7f42481021a0 2024-04-20T11:27:50.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.773+0000 7f4245600640 1 -- 172.21.15.44:0/959140670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238003c90 con 0x7f42481021a0 2024-04-20T11:27:50.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.774+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481a7a40 con 0x7f42481021a0 2024-04-20T11:27:50.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 Processor -- start 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 -- start start 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4245600640 1 -- 172.21.15.44:0/959140670 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4238046020 con 0x7f42481021a0 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fafb0 0x556d907fb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90803f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d908044b0 0x556d908068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d90760050 con 0x556d908044b0 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556d90706490 con 0x556d907fb8d0 2024-04-20T11:27:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.777+0000 7f4252200740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556d9075c010 con 0x556d907fafb0 2024-04-20T11:27:50.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f4247e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d908044b0 0x556d908068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90803f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fafb0 0x556d907fb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fafb0 0x556d907fb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40908/0 (socket says 172.21.15.44:40908) 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424d400640 1 -- 172.21.15.44:0/2935943953 learned_addr learned my addr 172.21.15.44:0/2935943953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 -- 172.21.15.44:0/2935943953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fafb0 msgr2=0x556d907fb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fafb0 0x556d907fb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 -- 172.21.15.44:0/2935943953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d908044b0 msgr2=0x556d908068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d908044b0 0x556d908068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 -- 172.21.15.44:0/2935943953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9070e070 con 0x556d907fb8d0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f4247e00640 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d908044b0 0x556d908068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.778+0000 7f424ca00640 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90803f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f42380eff80 tx=0x7f42380f8b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3035fcb9901255 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.779+0000 7f423dc00640 1 -- 172.21.15.44:0/2935943953 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238101070 con 0x556d907fb8d0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.779+0000 7f423dc00640 1 -- 172.21.15.44:0/2935943953 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42380fad70 con 0x556d907fb8d0 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.779+0000 7f4245600640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42280a3d50 0x7f42280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.779+0000 7f423dc00640 1 -- 172.21.15.44:0/2935943953 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380fd440 con 0x556d907fb8d0 2024-04-20T11:27:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.779+0000 7f4245600640 1 -- 172.21.15.44:0/959140670 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f42380f26b0 con 0x7f42481021a0 2024-04-20T11:27:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.779+0000 7f424d400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42280a3d50 0x7f42280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 -- 172.21.15.44:0/2935943953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 msgr2=0x556d90803f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90803f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f42380eff80 tx=0x7f42380f8b40 comp rx=0 tx=0).stop 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 -- 172.21.15.44:0/2935943953 shutdown_connections 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d908044b0 0x556d908068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90803f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 --2- 172.21.15.44:0/2935943953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fafb0 0x556d907fb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 -- 172.21.15.44:0/2935943953 >> 172.21.15.44:0/2935943953 conn(0x556d907f56e0 msgr2=0x556d907f5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 -- 172.21.15.44:0/2935943953 shutdown_connections 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f424d400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42280a3d50 0x7f42280a6210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4240011570 tx=0x7f4240005cc0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 -- 172.21.15.44:0/2935943953 wait complete. 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.780+0000 7f4252200740 1 -- start start 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 Processor -- start 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d908014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d908044b0 0x556d90801f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d90807060 con 0x556d907fafb0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556d90719860 con 0x556d907fb8d0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556d908071e0 con 0x556d908044b0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4252200740 10 client.0 _ll_register_callbacks cb 0x556d906e7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f424ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d908014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f4247e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f424ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d908014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56688/0 (socket says 172.21.15.44:56688) 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.781+0000 7f424ca00640 1 -- 172.21.15.44:0/2093044906 learned_addr learned my addr 172.21.15.44:0/2093044906 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424d400640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d908044b0 0x556d90801f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424ca00640 1 -- 172.21.15.44:0/2093044906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d908044b0 msgr2=0x556d90801f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424ca00640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d908044b0 0x556d90801f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424ca00640 1 -- 172.21.15.44:0/2093044906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 msgr2=0x556d90801a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424ca00640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90801a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424ca00640 1 -- 172.21.15.44:0/2093044906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9071d950 con 0x556d907fafb0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f4247e00640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f424ca00640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d908014d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f42380bac30 tx=0x7f4238005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639c2e0936b59805 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f4236000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f4236000640 1 -- 172.21.15.44:0/2093044906 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238101070 con 0x556d907fafb0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.782+0000 7f4236000640 1 -- 172.21.15.44:0/2093044906 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4238005570 con 0x556d907fafb0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42380fad70 con 0x556d907fafb0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4236000640 1 -- 172.21.15.44:0/2093044906 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f42380baa70 con 0x556d907fafb0 2024-04-20T11:27:50.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 10 client.16731 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556d906eb820 con 0x556d907fafb0 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4236000640 1 -- 172.21.15.44:0/2093044906 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42380fad70 con 0x556d907fafb0 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4236000640 1 -- 172.21.15.44:0/2093044906 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7f42380f00f0 con 0x556d907fafb0 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 10 client.16731 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 10 client.16731 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 20 client.16731 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d90814040 0x556d90816430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 4 client.16731 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.783+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9076da70 con 0x556d90814040 2024-04-20T11:27:50.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.784+0000 7f4247e00640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d90814040 0x556d90816430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.784+0000 7f4247e00640 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d90814040 0x556d90816430 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f2b10f04c1fc1b5 server_cookie=6ea90f6f7fc63e7c in_seq=0 out_seq=0 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.784+0000 7f4236000640 10 client.16731 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4236000640 1 -- 172.21.15.44:0/2093044906 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d9076da70 con 0x556d90814040 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4236000640 10 client.16731 handle_command_reply: tid=0 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4252200740 1 client.16731 shutdown 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 msgr2=0x556d908014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4252200740 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d908014d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f42380bac30 tx=0x7f4238005fd0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 shutdown_connections 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4252200740 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d908044b0 0x556d90801f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.785+0000 7f4252200740 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d90814040 0x556d90816430 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4252200740 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d907fb8d0 0x556d90801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4252200740 1 --2- 172.21.15.44:0/2093044906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d908014d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 >> 172.21.15.44:0/2093044906 conn(0x556d907f56e0 msgr2=0x556d907f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4236000640 0 client.16731 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 shutdown_connections 2024-04-20T11:27:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4252200740 1 -- 172.21.15.44:0/2093044906 wait complete. 2024-04-20T11:27:50.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.786+0000 7f4252200740 20 client.16731 trim_cache size 0 max 16384 2024-04-20T11:27:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 Processor -- start 2024-04-20T11:27:50.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 -- start start 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d90803c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d9080fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d907f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42481aff30 con 0x556d907fafb0 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42481b1bc0 con 0x556d908044b0 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.801+0000 7f4252200740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42481b0600 con 0x556d907fb8d0 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d9080fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d9080fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40926/0 (socket says 172.21.15.44:40926) 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 -- 172.21.15.44:0/2122419346 learned_addr learned my addr 172.21.15.44:0/2122419346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f424ca00640 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d90803c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f424d400640 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d907f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 -- 172.21.15.44:0/2122419346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 msgr2=0x556d907f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d907f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 -- 172.21.15.44:0/2122419346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 msgr2=0x556d90803c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d90803c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 -- 172.21.15.44:0/2122419346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d904723f0 con 0x556d907fb8d0 2024-04-20T11:27:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f424d400640 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d907f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.802+0000 7f4247e00640 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d9080fe20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4248075cf0 tx=0x7f42300117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db9e66e78a75a72e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f423d200640 1 -- 172.21.15.44:0/2122419346 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230013e60 con 0x556d907fb8d0 2024-04-20T11:27:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f423d200640 1 -- 172.21.15.44:0/2122419346 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4230023070 con 0x556d907fb8d0 2024-04-20T11:27:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f423d200640 1 -- 172.21.15.44:0/2122419346 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230004570 con 0x556d907fb8d0 2024-04-20T11:27:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 -- 172.21.15.44:0/2122419346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 msgr2=0x556d9080fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d9080fe20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4248075cf0 tx=0x7f42300117b0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 -- 172.21.15.44:0/2122419346 shutdown_connections 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d907f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d9080fe20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 --2- 172.21.15.44:0/2122419346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d90803c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.803+0000 7f4252200740 1 -- 172.21.15.44:0/2122419346 >> 172.21.15.44:0/2122419346 conn(0x556d907f56e0 msgr2=0x556d907f3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.804+0000 7f4252200740 1 -- 172.21.15.44:0/2122419346 shutdown_connections 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.804+0000 7f4252200740 1 -- 172.21.15.44:0/2122419346 wait complete. 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.804+0000 7f4252200740 1 -- start start 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.804+0000 7f4252200740 1 Processor -- start 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d9080c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d90807b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d90808190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42481b1430 con 0x556d907fafb0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42481b0d30 con 0x556d908044b0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42481adec0 con 0x556d907fb8d0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4252200740 10 client.0 _ll_register_callbacks cb 0x7f42380f8900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d9080c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d9080c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56706/0 (socket says 172.21.15.44:56706) 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 -- 172.21.15.44:0/3222268593 learned_addr learned my addr 172.21.15.44:0/3222268593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f4247e00640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d90807b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424d400640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d90808190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 -- 172.21.15.44:0/3222268593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 msgr2=0x556d90807b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d90807b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 -- 172.21.15.44:0/3222268593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 msgr2=0x556d90808190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d90808190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424ca00640 1 -- 172.21.15.44:0/3222268593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9053c7d0 con 0x556d907fafb0 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.805+0000 7f424d400640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d90808190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f424ca00640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d9080c810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42380bb1a0 tx=0x7f4238005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e8d7b4e7b25c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4236000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4236000640 1 -- 172.21.15.44:0/3222268593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238101070 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4236000640 1 -- 172.21.15.44:0/3222268593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f42380f8240 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4230023070 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4236000640 1 -- 172.21.15.44:0/3222268593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f42380be130 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4252200740 10 client.16737 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556d9053c7d0 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.806+0000 7f4236000640 1 -- 172.21.15.44:0/3222268593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4230023070 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4236000640 1 -- 172.21.15.44:0/3222268593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7f42380c6fb0 con 0x556d907fafb0 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4252200740 10 client.16737 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4252200740 10 client.16737 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4252200740 20 client.16737 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4252200740 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d9087a970 0x556d9087cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4252200740 4 client.16737 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:50.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d907678e0 con 0x556d9087a970 2024-04-20T11:27:50.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.807+0000 7f4247e00640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d9087a970 0x556d9087cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4247e00640 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d9087a970 0x556d9087cd90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=126e56d74a55b55a server_cookie=41ca46588678e92a in_seq=0 out_seq=0 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4236000640 10 client.16737 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4236000640 1 -- 172.21.15.44:0/3222268593 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d907678e0 con 0x556d9087a970 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4236000640 10 client.16737 handle_command_reply: tid=0 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 client.16737 shutdown 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 msgr2=0x556d9080c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d9080c810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42380bb1a0 tx=0x7f4238005e20 comp rx=0 tx=0).stop 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 shutdown_connections 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x556d9087a970 0x556d9087cd90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d908044b0 0x556d90808190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d907fb8d0 0x556d90807b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4236000640 0 client.16737 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 --2- 172.21.15.44:0/3222268593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d907fafb0 0x556d9080c810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 >> 172.21.15.44:0/3222268593 conn(0x556d907f56e0 msgr2=0x556d907f4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 shutdown_connections 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 1 -- 172.21.15.44:0/3222268593 wait complete. 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.808+0000 7f4252200740 20 client.16737 trim_cache size 0 max 16384 2024-04-20T11:27:50.813 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42280a3d50 msgr2=0x7f42280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42280a3d50 0x7f42280a6210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4240011570 tx=0x7f4240005cc0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 msgr2=0x7f4248074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 0x7f4248074260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c00 tx=0x7f423802dde0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 shutdown_connections 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4248074ce0 0x7f42481a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4248107270 0x7f42480747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42280a3d50 0x7f42280a6210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 --2- 172.21.15.44:0/959140670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42481021a0 0x7f4248074260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 >> 172.21.15.44:0/959140670 conn(0x7f42480793b0 msgr2=0x7f42480ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:50.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 shutdown_connections 2024-04-20T11:27:50.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:50.809+0000 7f424f400640 1 -- 172.21.15.44:0/959140670 wait complete. 2024-04-20T11:27:50.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:27:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 -- 172.21.15.44:0/1116947517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8104dc0 msgr2=0x7fd8d8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 --2- 172.21.15.44:0/1116947517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8104dc0 0x7fd8d8108e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009900 tx=0x7fd8c002bc30 comp rx=0 tx=0).stop 2024-04-20T11:27:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 -- 172.21.15.44:0/1116947517 shutdown_connections 2024-04-20T11:27:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 --2- 172.21.15.44:0/1116947517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8d81094e0 0x7fd8d81110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 --2- 172.21.15.44:0/1116947517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8104dc0 0x7fd8d8108e00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 --2- 172.21.15.44:0/1116947517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d8104410 0x7fd8d81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 -- 172.21.15.44:0/1116947517 >> 172.21.15.44:0/1116947517 conn(0x7fd8d80ffcf0 msgr2=0x7fd8d8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.044+0000 7fd8dde00640 1 -- 172.21.15.44:0/1116947517 shutdown_connections 2024-04-20T11:27:51.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.045+0000 7fd8dde00640 1 -- 172.21.15.44:0/1116947517 wait complete. 2024-04-20T11:27:51.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.045+0000 7fd8dde00640 1 Processor -- start 2024-04-20T11:27:51.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 -- start start 2024-04-20T11:27:51.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8d8104410 0x7fd8d81a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 0x7fd8d81a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d819a9e0 0x7fd8d819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d8114660 con 0x7fd8d81094e0 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81144e0 con 0x7fd8d819a9e0 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8dde00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81147e0 con 0x7fd8d8104410 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8ce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 0x7fd8d81a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8ce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 0x7fd8d81a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56720/0 (socket says 172.21.15.44:56720) 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8ce000640 1 -- 172.21.15.44:0/2032629164 learned_addr learned my addr 172.21.15.44:0/2032629164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.046+0000 7fd8d7400640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d819a9e0 0x7fd8d819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8d8104410 0x7fd8d81a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8ce000640 1 -- 172.21.15.44:0/2032629164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8d8104410 msgr2=0x7fd8d81a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8ce000640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8d8104410 0x7fd8d81a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8ce000640 1 -- 172.21.15.44:0/2032629164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d819a9e0 msgr2=0x7fd8d819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8ce000640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d819a9e0 0x7fd8d819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8ce000640 1 -- 172.21.15.44:0/2032629164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d819b500 con 0x7fd8d81094e0 2024-04-20T11:27:51.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.047+0000 7fd8ce000640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 0x7fd8d81a0d70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0036040 tx=0x7fd8c002dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.048+0000 7fd8dca00640 1 -- 172.21.15.44:0/2032629164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0043020 con 0x7fd8d81094e0 2024-04-20T11:27:51.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.048+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d819b790 con 0x7fd8d81094e0 2024-04-20T11:27:51.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.048+0000 7fd8dca00640 1 -- 172.21.15.44:0/2032629164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd8c002de10 con 0x7fd8d81094e0 2024-04-20T11:27:51.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.048+0000 7fd8dca00640 1 -- 172.21.15.44:0/2032629164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0003ab0 con 0x7fd8d81094e0 2024-04-20T11:27:51.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.048+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d81a7a30 con 0x7fd8d81094e0 2024-04-20T11:27:51.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.051+0000 7fd8e0a77740 1 Processor -- start 2024-04-20T11:27:51.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.051+0000 7fd8dca00640 1 -- 172.21.15.44:0/2032629164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd8c0046020 con 0x7fd8d81094e0 2024-04-20T11:27:51.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.051+0000 7fd8e0a77740 1 -- start start 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99bd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55deb9922050 con 0x55deb99bcfb0 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55deb98c8490 con 0x55deb99bd8d0 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55deb991e010 con 0x55deb99c64b0 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99bd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8d6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99bd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56728/0 (socket says 172.21.15.44:56728) 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8d7400640 1 -- 172.21.15.44:0/1067283642 learned_addr learned my addr 172.21.15.44:0/1067283642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.052+0000 7fd8ce000640 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d7400640 1 -- 172.21.15.44:0/1067283642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 msgr2=0x55deb99c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d7400640 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d7400640 1 -- 172.21.15.44:0/1067283642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 msgr2=0x55deb99c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d7400640 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d7400640 1 -- 172.21.15.44:0/1067283642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb98d0070 con 0x55deb99bcfb0 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8dca00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd8bc0a3da0 0x7fd8bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d7400640 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99bd390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009c20 tx=0x7fd8d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4c1b95ea1f21a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8dca00640 1 -- 172.21.15.44:0/2032629164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fd8c00f2620 con 0x7fd8d81094e0 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd8bc0a3da0 0x7fd8bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.053+0000 7fd8c7e00640 1 -- 172.21.15.44:0/1067283642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d001e070 con 0x55deb99bcfb0 2024-04-20T11:27:51.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8c7e00640 1 -- 172.21.15.44:0/1067283642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd8d0019b30 con 0x55deb99bcfb0 2024-04-20T11:27:51.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8c7e00640 1 -- 172.21.15.44:0/1067283642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000ea80 con 0x55deb99bcfb0 2024-04-20T11:27:51.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1067283642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 msgr2=0x55deb99bd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99bd390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009c20 tx=0x7fd8d0011810 comp rx=0 tx=0).stop 2024-04-20T11:27:51.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd8bc0a3da0 0x7fd8bc0a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55deb99bca80 tx=0x7fd8c8009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1067283642 shutdown_connections 2024-04-20T11:27:51.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1067283642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99bd390 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.054+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1067283642 >> 172.21.15.44:0/1067283642 conn(0x55deb99b76e0 msgr2=0x55deb99b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.055+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1067283642 shutdown_connections 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.055+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1067283642 wait complete. 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.055+0000 7fd8e0a77740 1 -- start start 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.055+0000 7fd8e0a77740 1 Processor -- start 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bd8d0 0x55deb99c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55deb99c92a0 con 0x55deb99bcfb0 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55deb98db860 con 0x55deb99c64b0 2024-04-20T11:27:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55deb99c9420 con 0x55deb99bd8d0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8e0a77740 10 client.0 _ll_register_callbacks cb 0x55deb98a9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56734/0 (socket says 172.21.15.44:56734) 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3073678698 learned_addr learned my addr 172.21.15.44:0/3073678698 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8ce000640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bd8d0 0x55deb99c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.056+0000 7fd8d7400640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3073678698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bd8d0 msgr2=0x55deb99c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bd8d0 0x55deb99c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 msgr2=0x55deb99c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3073678698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb98df950 con 0x55deb99bcfb0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d7400640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c34e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800fc90 tx=0x7fd8c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ff4ee80d2f35e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.057+0000 7fd8c4c00640 1 -- 172.21.15.44:0/3073678698 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8003e00 con 0x55deb99bcfb0 2024-04-20T11:27:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8c4c00640 1 -- 172.21.15.44:0/3073678698 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fd8c8004530 con 0x55deb99bcfb0 2024-04-20T11:27:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d0019b30 con 0x55deb99bcfb0 2024-04-20T11:27:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8c4c00640 1 -- 172.21.15.44:0/3073678698 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd8c8004730 con 0x55deb99bcfb0 2024-04-20T11:27:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8e0a77740 10 client.16749 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55deb98ad820 con 0x55deb99bcfb0 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8c4c00640 1 -- 172.21.15.44:0/3073678698 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd8d0019b30 con 0x55deb99bcfb0 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8c4c00640 1 -- 172.21.15.44:0/3073678698 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7fd8c8030020 con 0x55deb99bcfb0 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8e0a77740 10 client.16749 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.058+0000 7fd8e0a77740 10 client.16749 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8e0a77740 20 client.16749 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb99d64a0 0x55deb99d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8e0a77740 4 client.16749 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb992fa70 con 0x55deb99d64a0 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8ce000640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb99d64a0 0x55deb99d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8ce000640 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb99d64a0 0x55deb99d8900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63fbdbdc4850d065 server_cookie=494f865f62c107e7 in_seq=0 out_seq=0 2024-04-20T11:27:51.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.059+0000 7fd8c4c00640 10 client.16749 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.060+0000 7fd8c4c00640 1 -- 172.21.15.44:0/3073678698 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55deb992fa70 con 0x55deb99d64a0 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.060+0000 7fd8c4c00640 10 client.16749 handle_command_reply: tid=0 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.060+0000 7fd8e0a77740 1 client.16749 shutdown 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 msgr2=0x55deb99c34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c34e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800fc90 tx=0x7fd8c8004ab0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 shutdown_connections 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bd8d0 0x55deb99c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb99d64a0 0x55deb99d8900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3073678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c34e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 >> 172.21.15.44:0/3073678698 conn(0x55deb99b76e0 msgr2=0x55deb99b9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8c4c00640 0 client.16749 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 shutdown_connections 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3073678698 wait complete. 2024-04-20T11:27:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.061+0000 7fd8e0a77740 20 client.16749 trim_cache size 0 max 16384 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.076+0000 7fd8e0a77740 1 Processor -- start 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.076+0000 7fd8e0a77740 1 -- start start 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81afe60 con 0x55deb99bcfb0 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81b1af0 con 0x55deb99bd8d0 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81b0530 con 0x55deb99c64b0 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56748/0 (socket says 172.21.15.44:56748) 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3791712166 learned_addr learned my addr 172.21.15.44:0/3791712166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8ce000640 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d7400640 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3791712166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 msgr2=0x55deb99c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3791712166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 msgr2=0x55deb99d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.077+0000 7fd8d6a00640 1 -- 172.21.15.44:0/3791712166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb96343f0 con 0x55deb99bcfb0 2024-04-20T11:27:51.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.078+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c5c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8011620 tx=0x7fd8c8011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0d8c551ebcbee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.078+0000 7fd8c7400640 1 -- 172.21.15.44:0/3791712166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c80048d0 con 0x55deb99bcfb0 2024-04-20T11:27:51.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.078+0000 7fd8c7400640 1 -- 172.21.15.44:0/3791712166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd8c8037020 con 0x55deb99bcfb0 2024-04-20T11:27:51.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.078+0000 7fd8c7400640 1 -- 172.21.15.44:0/3791712166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8032830 con 0x55deb99bcfb0 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3791712166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 msgr2=0x55deb99c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c5c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8011620 tx=0x7fd8c8011650 comp rx=0 tx=0).stop 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3791712166 shutdown_connections 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99c64b0 0x55deb99c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99bd8d0 0x55deb99d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/3791712166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bcfb0 0x55deb99c5c10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3791712166 >> 172.21.15.44:0/3791712166 conn(0x55deb99b76e0 msgr2=0x55deb99b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3791712166 shutdown_connections 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 -- 172.21.15.44:0/3791712166 wait complete. 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.079+0000 7fd8e0a77740 1 -- start start 2024-04-20T11:27:51.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 Processor -- start 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bcfb0 0x55deb99ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 0x55deb99ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81b1360 con 0x55deb99bd8d0 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81b0c60 con 0x55deb99c64b0 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d81ade30 con 0x55deb99bcfb0 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.080+0000 7fd8e0a77740 10 client.0 _ll_register_callbacks cb 0x7fd8c800faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 0x55deb99ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 0x55deb99ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56756/0 (socket says 172.21.15.44:56756) 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 -- 172.21.15.44:0/1276553071 learned_addr learned my addr 172.21.15.44:0/1276553071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bcfb0 0x55deb99ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8d7400640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 -- 172.21.15.44:0/1276553071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bcfb0 msgr2=0x55deb99ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bcfb0 0x55deb99ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 -- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 msgr2=0x55deb99c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 -- 172.21.15.44:0/1276553071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb96fe7d0 con 0x55deb99bd8d0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8ce000640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 0x55deb99ce8c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00be3e0 tx=0x7fd8c00f8530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0db8bc6d833bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.081+0000 7fd8c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8c4c00640 1 -- 172.21.15.44:0/1276553071 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00bede0 con 0x55deb99bd8d0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8c4c00640 1 -- 172.21.15.44:0/1276553071 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fd8c0003c50 con 0x55deb99bd8d0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c8037020 con 0x55deb99bd8d0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8c4c00640 1 -- 172.21.15.44:0/1276553071 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd8c0003e50 con 0x55deb99bd8d0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8e0a77740 10 client.16755 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55deb9a44b50 con 0x55deb99bd8d0 2024-04-20T11:27:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8c4c00640 1 -- 172.21.15.44:0/1276553071 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd8c8037020 con 0x55deb99bd8d0 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8c4c00640 1 -- 172.21.15.44:0/1276553071 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7fd8c8011d20 con 0x55deb99bd8d0 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8e0a77740 10 client.16755 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8e0a77740 10 client.16755 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.082+0000 7fd8e0a77740 20 client.16755 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.083+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb9a45380 0x55deb9a477a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.083+0000 7fd8e0a77740 4 client.16755 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.083+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deb99298e0 con 0x55deb9a45380 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.083+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb9a45380 0x55deb9a477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.083+0000 7fd8d6a00640 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb9a45380 0x55deb9a477a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ccadd0341bd1d1c server_cookie=a3fde076d1423b28 in_seq=0 out_seq=0 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.083+0000 7fd8c4c00640 10 client.16755 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8c4c00640 1 -- 172.21.15.44:0/1276553071 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deb99298e0 con 0x55deb9a45380 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8c4c00640 10 client.16755 handle_command_reply: tid=0 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 client.16755 shutdown 2024-04-20T11:27:51.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 msgr2=0x55deb99ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 0x55deb99ce8c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00be3e0 tx=0x7fd8c00f8530 comp rx=0 tx=0).stop 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 shutdown_connections 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55deb99c64b0 0x55deb99c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55deb9a45380 0x55deb9a477a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55deb99bd8d0 0x55deb99ce8c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 --2- 172.21.15.44:0/1276553071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55deb99bcfb0 0x55deb99ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 >> 172.21.15.44:0/1276553071 conn(0x55deb99b76e0 msgr2=0x55deb99c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8c4c00640 0 client.16755 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 shutdown_connections 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 1 -- 172.21.15.44:0/1276553071 wait complete. 2024-04-20T11:27:51.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.084+0000 7fd8e0a77740 20 client.16755 trim_cache size 0 max 16384 2024-04-20T11:27:51.089 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:51.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd8bc0a3da0 msgr2=0x7fd8bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd8bc0a3da0 0x7fd8bc0a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55deb99bca80 tx=0x7fd8c8009500 comp rx=0 tx=0).stop 2024-04-20T11:27:51.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 msgr2=0x7fd8d81a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 0x7fd8d81a0d70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0036040 tx=0x7fd8c002dbf0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 shutdown_connections 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d819a9e0 0x7fd8d819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd8bc0a3da0 0x7fd8bc0a6260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8d81094e0 0x7fd8d81a0d70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 --2- 172.21.15.44:0/2032629164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8d8104410 0x7fd8d81a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 >> 172.21.15.44:0/2032629164 conn(0x7fd8d80ffcf0 msgr2=0x7fd8d8102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 shutdown_connections 2024-04-20T11:27:51.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.085+0000 7fd8dde00640 1 -- 172.21.15.44:0/2032629164 wait complete. 2024-04-20T11:27:51.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:27:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.314+0000 7f14fb200640 1 -- 172.21.15.44:0/1471474564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4102a20 msgr2=0x7f14f4106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.314+0000 7f14fb200640 1 --2- 172.21.15.44:0/1471474564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4102a20 0x7f14f4106a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402bc40 comp rx=0 tx=0).stop 2024-04-20T11:27:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 -- 172.21.15.44:0/1471474564 shutdown_connections 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 --2- 172.21.15.44:0/1471474564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14f4107140 0x7f14f410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 --2- 172.21.15.44:0/1471474564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4102a20 0x7f14f4106a60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 --2- 172.21.15.44:0/1471474564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 0x7f14f4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 -- 172.21.15.44:0/1471474564 >> 172.21.15.44:0/1471474564 conn(0x7f14f40fd930 msgr2=0x7f14f40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 -- 172.21.15.44:0/1471474564 shutdown_connections 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.315+0000 7f14fb200640 1 -- 172.21.15.44:0/1471474564 wait complete. 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.316+0000 7f14fb200640 1 Processor -- start 2024-04-20T11:27:51.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.316+0000 7f14fb200640 1 -- start start 2024-04-20T11:27:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 0x7f14f419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14f4107140 0x7f14f419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4198670 0x7f14f4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41122f0 con 0x7f14f4102070 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4112170 con 0x7f14f4198670 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4112470 con 0x7f14f4107140 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 0x7f14f419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 0x7f14f419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56776/0 (socket says 172.21.15.44:56776) 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14fa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4198670 0x7f14f4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14f9e00640 1 -- 172.21.15.44:0/2743218644 learned_addr learned my addr 172.21.15.44:0/2743218644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.317+0000 7f14f9400640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14f4107140 0x7f14f419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fa800640 1 -- 172.21.15.44:0/2743218644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14f4107140 msgr2=0x7f14f419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fa800640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14f4107140 0x7f14f419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fa800640 1 -- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 msgr2=0x7f14f419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fa800640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 0x7f14f419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fa800640 1 -- 172.21.15.44:0/2743218644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f4199270 con 0x7f14f4198670 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fa800640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4198670 0x7f14f4198af0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f14f000da10 tx=0x7f14f000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14eea00640 1 -- 172.21.15.44:0/2743218644 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f000bd70 con 0x7f14f4198670 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14eea00640 1 -- 172.21.15.44:0/2743218644 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14f0004030 con 0x7f14f4198670 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f41a5350 con 0x7f14f4198670 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14eea00640 1 -- 172.21.15.44:0/2743218644 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f00107e0 con 0x7f14f4198670 2024-04-20T11:27:51.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.318+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f41a5700 con 0x7f14f4198670 2024-04-20T11:27:51.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.321+0000 7f14ff62a740 1 Processor -- start 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14eea00640 1 -- 172.21.15.44:0/2743218644 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f14f00041e0 con 0x7f14f4198670 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 -- start start 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5605bc54d050 con 0x5605bc5f1670 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605bc4f3490 con 0x5605bc5e8a90 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14ff62a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605bc549010 con 0x5605bc5e8170 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.323+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5f1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39546/0 (socket says 172.21.15.44:39546) 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 -- 172.21.15.44:0/2142649607 learned_addr learned my addr 172.21.15.44:0/2142649607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9400640 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 -- 172.21.15.44:0/2142649607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 msgr2=0x5605bc5e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14eea00640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14dc0a3d20 0x7f14dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 -- 172.21.15.44:0/2142649607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 msgr2=0x5605bc5f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 -- 172.21.15.44:0/2142649607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bc4fb070 con 0x5605bc5e8a90 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9400640 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14eea00640 1 -- 172.21.15.44:0/2743218644 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f14f00d0a90 con 0x7f14f4198670 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14fa800640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14dc0a3d20 0x7f14dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.324+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5f1130 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b7e0 tx=0x7f14e800bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a26fa46817cbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.325+0000 7f14e3000640 1 -- 172.21.15.44:0/2142649607 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8003dd0 con 0x5605bc5e8a90 2024-04-20T11:27:51.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.325+0000 7f14e3000640 1 -- 172.21.15.44:0/2142649607 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e8015070 con 0x5605bc5e8a90 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.325+0000 7f14e3000640 1 -- 172.21.15.44:0/2142649607 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8010820 con 0x5605bc5e8a90 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.325+0000 7f14ff62a740 1 -- 172.21.15.44:0/2142649607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 msgr2=0x5605bc5f1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.325+0000 7f14fa800640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14dc0a3d20 0x7f14dc0a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14f000bfd0 tx=0x7f14f0004a40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.325+0000 7f14ff62a740 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5f1130 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b7e0 tx=0x7f14e800bca0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 -- 172.21.15.44:0/2142649607 shutdown_connections 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5f1130 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 --2- 172.21.15.44:0/2142649607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 -- 172.21.15.44:0/2142649607 >> 172.21.15.44:0/2142649607 conn(0x5605bc5e28a0 msgr2=0x5605bc5e2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 -- 172.21.15.44:0/2142649607 shutdown_connections 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 -- 172.21.15.44:0/2142649607 wait complete. 2024-04-20T11:27:51.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 -- start start 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.326+0000 7f14ff62a740 1 Processor -- start 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5605bc5f4460 con 0x5605bc5f1670 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605bc506860 con 0x5605bc5e8a90 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605bc5f45e0 con 0x5605bc5e8170 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14ff62a740 10 client.0 _ll_register_callbacks cb 0x5605bc4d4fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5ef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56794/0 (socket says 172.21.15.44:56794) 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14f9400640 1 -- 172.21.15.44:0/294389252 learned_addr learned my addr 172.21.15.44:0/294389252 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14f9e00640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.327+0000 7f14f9400640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9400640 1 -- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 msgr2=0x5605bc5ee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9400640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5ee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9400640 1 -- 172.21.15.44:0/294389252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 msgr2=0x5605bc5eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9400640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9400640 1 -- 172.21.15.44:0/294389252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bc50a950 con 0x5605bc5f1670 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9e00640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14f9400640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5ef150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f24b88b2b45fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14d7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14d7400640 1 -- 172.21.15.44:0/294389252 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4004020 con 0x5605bc5f1670 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14d7400640 1 -- 172.21.15.44:0/294389252 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f14e40041c0 con 0x5605bc5f1670 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.328+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e8015070 con 0x5605bc5f1670 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14d7400640 1 -- 172.21.15.44:0/294389252 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f14e40043c0 con 0x5605bc5f1670 2024-04-20T11:27:51.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 10 client.16767 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5605bc4d8820 con 0x5605bc5f1670 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14d7400640 1 -- 172.21.15.44:0/294389252 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e8015070 con 0x5605bc5f1670 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14d7400640 1 -- 172.21.15.44:0/294389252 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7f14e4003ab0 con 0x5605bc5f1670 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 10 client.16767 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 20 client.16767 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc6016e0 0x5605bc603b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 4 client.16767 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.329+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605bc55aa70 con 0x5605bc6016e0 2024-04-20T11:27:51.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.330+0000 7f14f9e00640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc6016e0 0x5605bc603b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.331+0000 7f14f9e00640 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc6016e0 0x5605bc603b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=360f328867412fb5 server_cookie=7a9e169cd746a667 in_seq=0 out_seq=0 2024-04-20T11:27:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.331+0000 7f14d7400640 10 client.16767 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.331+0000 7f14d7400640 1 -- 172.21.15.44:0/294389252 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605bc55aa70 con 0x5605bc6016e0 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.331+0000 7f14d7400640 10 client.16767 handle_command_reply: tid=0 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 client.16767 shutdown 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 msgr2=0x5605bc5ef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5ef150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402bc40 comp rx=0 tx=0).stop 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 shutdown_connections 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc6016e0 0x5605bc603b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5f1670 0x5605bc5ef150 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8a90 0x5605bc5eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 --2- 172.21.15.44:0/294389252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5ee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14d7400640 0 client.16767 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 >> 172.21.15.44:0/294389252 conn(0x5605bc5e28a0 msgr2=0x5605bc5e45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 shutdown_connections 2024-04-20T11:27:51.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 1 -- 172.21.15.44:0/294389252 wait complete. 2024-04-20T11:27:51.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.332+0000 7f14ff62a740 20 client.16767 trim_cache size 0 max 16384 2024-04-20T11:27:51.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.347+0000 7f14ff62a740 1 Processor -- start 2024-04-20T11:27:51.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.347+0000 7f14ff62a740 1 -- start start 2024-04-20T11:27:51.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc5fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5f1670 0x5605bc5f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14ff62a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41ada60 con 0x5605bc5e8a90 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14ff62a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41af6f0 con 0x5605bc5f1670 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14ff62a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41ae130 con 0x5605bc5e8170 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc5fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc5fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56806/0 (socket says 172.21.15.44:56806) 2024-04-20T11:27:51.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 -- 172.21.15.44:0/609714439 learned_addr learned my addr 172.21.15.44:0/609714439 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 -- 172.21.15.44:0/609714439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 msgr2=0x5605bc5f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 -- 172.21.15.44:0/609714439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5f1670 msgr2=0x5605bc5f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5f1670 0x5605bc5f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.348+0000 7f14f9e00640 1 -- 172.21.15.44:0/609714439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605bc25f3f0 con 0x5605bc5e8a90 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.349+0000 7f14f9e00640 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc5fd320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f14e8000c00 tx=0x7f14e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d6b4922c8195b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.349+0000 7f14e2600640 1 -- 172.21.15.44:0/609714439 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8005030 con 0x5605bc5e8a90 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.349+0000 7f14e2600640 1 -- 172.21.15.44:0/609714439 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e8015040 con 0x5605bc5e8a90 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.349+0000 7f14e2600640 1 -- 172.21.15.44:0/609714439 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004ba0 con 0x5605bc5e8a90 2024-04-20T11:27:51.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 -- 172.21.15.44:0/609714439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 msgr2=0x5605bc5fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc5fd320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f14e8000c00 tx=0x7f14e8005fd0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 -- 172.21.15.44:0/609714439 shutdown_connections 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5f1670 0x5605bc5f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc5fd320 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 --2- 172.21.15.44:0/609714439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5e8170 0x5605bc5f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 -- 172.21.15.44:0/609714439 >> 172.21.15.44:0/609714439 conn(0x5605bc5e28a0 msgr2=0x5605bc5eef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 -- 172.21.15.44:0/609714439 shutdown_connections 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 -- 172.21.15.44:0/609714439 wait complete. 2024-04-20T11:27:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.350+0000 7f14ff62a740 1 -- start start 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 Processor -- start 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8170 0x5605bc65fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc660160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5f1670 0x5605bc6606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41aef60 con 0x5605bc5e8a90 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41ae860 con 0x5605bc5e8170 2024-04-20T11:27:51.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41aba10 con 0x5605bc5f1670 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.351+0000 7f14ff62a740 10 client.0 _ll_register_callbacks cb 0x7f14e4005520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc660160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc660160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56822/0 (socket says 172.21.15.44:56822) 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 -- 172.21.15.44:0/1323597877 learned_addr learned my addr 172.21.15.44:0/1323597877 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9400640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8170 0x5605bc65fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9400640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5f1670 0x5605bc6606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 -- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5f1670 msgr2=0x5605bc6606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5f1670 0x5605bc6606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 -- 172.21.15.44:0/1323597877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8170 msgr2=0x5605bc65fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8170 0x5605bc65fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.352+0000 7f14f9e00640 1 -- 172.21.15.44:0/1323597877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bc3297d0 con 0x5605bc5e8a90 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14f9e00640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc660160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f14e8011fd0 tx=0x7f14e80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5350aec35ec1eb12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14d7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14d7400640 1 -- 172.21.15.44:0/1323597877 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011440 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14d7400640 1 -- 172.21.15.44:0/1323597877 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f14e80115e0 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e8015040 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14d7400640 1 -- 172.21.15.44:0/1323597877 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f14e80117e0 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14ff62a740 10 client.16773 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5605bc66c030 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.353+0000 7f14d7400640 1 -- 172.21.15.44:0/1323597877 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e8015040 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14d7400640 1 -- 172.21.15.44:0/1323597877 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7f14e8010550 con 0x5605bc5e8a90 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14ff62a740 10 client.16773 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14ff62a740 10 client.16773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14ff62a740 20 client.16773 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14ff62a740 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc66c510 0x5605bc66e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14ff62a740 4 client.16773 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:51.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605bc5548e0 con 0x5605bc66c510 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.354+0000 7f14f9400640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc66c510 0x5605bc66e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14f9400640 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc66c510 0x5605bc66e930 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8271cf0071d52758 server_cookie=ffdf5bbcc286143b in_seq=0 out_seq=0 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14d7400640 10 client.16773 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14d7400640 1 -- 172.21.15.44:0/1323597877 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605bc5548e0 con 0x5605bc66c510 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14d7400640 10 client.16773 handle_command_reply: tid=0 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 client.16773 shutdown 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 msgr2=0x5605bc660160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc660160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f14e8011fd0 tx=0x7f14e80045f0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 shutdown_connections 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5605bc66c510 0x5605bc66e930 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605bc5f1670 0x5605bc6606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605bc5e8a90 0x5605bc660160 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 --2- 172.21.15.44:0/1323597877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605bc5e8170 0x5605bc65fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14d7400640 0 client.16773 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:27:51.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 >> 172.21.15.44:0/1323597877 conn(0x5605bc5e28a0 msgr2=0x5605bc5e3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 shutdown_connections 2024-04-20T11:27:51.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.355+0000 7f14ff62a740 1 -- 172.21.15.44:0/1323597877 wait complete. 2024-04-20T11:27:51.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14ff62a740 20 client.16773 trim_cache size 0 max 16384 2024-04-20T11:27:51.360 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14dc0a3d20 msgr2=0x7f14dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14dc0a3d20 0x7f14dc0a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14f000bfd0 tx=0x7f14f0004a40 comp rx=0 tx=0).stop 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4198670 msgr2=0x7f14f4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4198670 0x7f14f4198af0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f14f000da10 tx=0x7f14f000ded0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 shutdown_connections 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14f4198670 0x7f14f4198af0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14f4107140 0x7f14f419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14dc0a3d20 0x7f14dc0a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 --2- 172.21.15.44:0/2743218644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f4102070 0x7f14f419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 >> 172.21.15.44:0/2743218644 conn(0x7f14f40fd930 msgr2=0x7f14f40ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.356+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 shutdown_connections 2024-04-20T11:27:51.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.357+0000 7f14fb200640 1 -- 172.21.15.44:0/2743218644 wait complete. 2024-04-20T11:27:51.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:27:51.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 -- 172.21.15.44:0/2949927837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde0104410 msgr2=0x7efde01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 --2- 172.21.15.44:0/2949927837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde0104410 0x7efde01047f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009ca0 tx=0x7efdd002c100 comp rx=0 tx=0).stop 2024-04-20T11:27:51.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 -- 172.21.15.44:0/2949927837 shutdown_connections 2024-04-20T11:27:51.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 --2- 172.21.15.44:0/2949927837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde01110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 --2- 172.21.15.44:0/2949927837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0104dc0 0x7efde0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 --2- 172.21.15.44:0/2949927837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde0104410 0x7efde01047f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 -- 172.21.15.44:0/2949927837 >> 172.21.15.44:0/2949927837 conn(0x7efde00ffcf0 msgr2=0x7efde0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 -- 172.21.15.44:0/2949927837 shutdown_connections 2024-04-20T11:27:51.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.575+0000 7efde5200640 1 -- 172.21.15.44:0/2949927837 wait complete. 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.576+0000 7efde5200640 1 Processor -- start 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 -- start start 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde019b300 0x7efde019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde019fbc0 0x7efde01a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efde01146d0 con 0x7efde019b300 2024-04-20T11:27:51.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efde0114550 con 0x7efde01094e0 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efde5200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efde0114850 con 0x7efde019fbc0 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efdd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde019b300 0x7efde019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efddde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efdd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde019b300 0x7efde019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56844/0 (socket says 172.21.15.44:56844) 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efdd5400640 1 -- 172.21.15.44:0/418438321 learned_addr learned my addr 172.21.15.44:0/418438321 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efddde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39608/0 (socket says 172.21.15.44:39608) 2024-04-20T11:27:51.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.577+0000 7efdde800640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde019fbc0 0x7efde01a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efddde00640 1 -- 172.21.15.44:0/418438321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde019fbc0 msgr2=0x7efde01a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efddde00640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde019fbc0 0x7efde01a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efddde00640 1 -- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde019b300 msgr2=0x7efde019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efddde00640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde019b300 0x7efde019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efddde00640 1 -- 172.21.15.44:0/418438321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde01a7930 con 0x7efde01094e0 2024-04-20T11:27:51.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efddde00640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde019adc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdd0008000 tx=0x7efdd0034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.578+0000 7efdd7400640 1 -- 172.21.15.44:0/418438321 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0039040 con 0x7efde01094e0 2024-04-20T11:27:51.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.579+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde01a7b00 con 0x7efde01094e0 2024-04-20T11:27:51.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.579+0000 7efdd7400640 1 -- 172.21.15.44:0/418438321 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdd0034630 con 0x7efde01094e0 2024-04-20T11:27:51.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.579+0000 7efdd7400640 1 -- 172.21.15.44:0/418438321 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00043c0 con 0x7efde01094e0 2024-04-20T11:27:51.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.579+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01a8010 con 0x7efde01094e0 2024-04-20T11:27:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.582+0000 7efdd7400640 1 -- 172.21.15.44:0/418438321 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efdd0046030 con 0x7efde01094e0 2024-04-20T11:27:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.582+0000 7efde7fe4740 1 Processor -- start 2024-04-20T11:27:51.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.582+0000 7efde7fe4740 1 -- start start 2024-04-20T11:27:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x5634483f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 0x5634483fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483fc4b0 0x5634483fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efde7fe4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563448358050 con 0x5634483f2fb0 2024-04-20T11:27:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efde7fe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634482fe490 con 0x5634483f38d0 2024-04-20T11:27:51.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efde7fe4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563448354010 con 0x5634483fc4b0 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efddde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 0x5634483fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efdde800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x5634483f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efdde800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x5634483f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56846/0 (socket says 172.21.15.44:56846) 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efddde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 0x5634483fbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39624/0 (socket says 172.21.15.44:39624) 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efdde800640 1 -- 172.21.15.44:0/4064311082 learned_addr learned my addr 172.21.15.44:0/4064311082 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.583+0000 7efdd5400640 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483fc4b0 0x5634483fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efddde00640 1 -- 172.21.15.44:0/4064311082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483fc4b0 msgr2=0x5634483fe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efddde00640 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483fc4b0 0x5634483fe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efddde00640 1 -- 172.21.15.44:0/4064311082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 msgr2=0x5634483f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efddde00640 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x5634483f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efddde00640 1 -- 172.21.15.44:0/4064311082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563448306070 con 0x5634483f38d0 2024-04-20T11:27:51.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efdd7400640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efdc40a3da0 0x7efdc40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efdd7400640 1 -- 172.21.15.44:0/418438321 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efdd00f1f20 con 0x7efde01094e0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efddde00640 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 0x5634483fbf70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ef7f0 tx=0x7efdd00fa0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd42527e65660783 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efdde800640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efdc40a3da0 0x7efdc40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efdcf400640 1 -- 172.21.15.44:0/4064311082 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ba370 con 0x5634483f38d0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efdcf400640 1 -- 172.21.15.44:0/4064311082 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdd00be7d0 con 0x5634483f38d0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.584+0000 7efdcf400640 1 -- 172.21.15.44:0/4064311082 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00bda30 con 0x5634483f38d0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 -- 172.21.15.44:0/4064311082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 msgr2=0x5634483fbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efdde800640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efdc40a3da0 0x7efdc40a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009e60 tx=0x7efdd8005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 0x5634483fbf70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ef7f0 tx=0x7efdd00fa0e0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 -- 172.21.15.44:0/4064311082 shutdown_connections 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483fc4b0 0x5634483fe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f38d0 0x5634483fbf70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 --2- 172.21.15.44:0/4064311082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x5634483f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 -- 172.21.15.44:0/4064311082 >> 172.21.15.44:0/4064311082 conn(0x5634483ed6e0 msgr2=0x5634483edac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 -- 172.21.15.44:0/4064311082 shutdown_connections 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.585+0000 7efde7fe4740 1 -- 172.21.15.44:0/4064311082 wait complete. 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.586+0000 7efde7fe4740 1 -- start start 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.586+0000 7efde7fe4740 1 Processor -- start 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f2fb0 0x5634483f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 0x5634483fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5634483ff060 con 0x5634483fc4b0 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563448311860 con 0x5634483f2fb0 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5634483ff1e0 con 0x5634483f38d0 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efde7fe4740 10 client.0 _ll_register_callbacks cb 0x5634482dffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 0x5634483fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 0x5634483fa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56854/0 (socket says 172.21.15.44:56854) 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 -- 172.21.15.44:0/2896178374 learned_addr learned my addr 172.21.15.44:0/2896178374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efddde00640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f2fb0 0x5634483f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdd5400640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 -- 172.21.15.44:0/2896178374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 msgr2=0x5634483f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483f9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 -- 172.21.15.44:0/2896178374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f2fb0 msgr2=0x5634483f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f2fb0 0x5634483f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efdde800640 1 -- 172.21.15.44:0/2896178374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563448315950 con 0x5634483fc4b0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.587+0000 7efddde00640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f2fb0 0x5634483f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efdde800640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 0x5634483fa030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efdd8004590 tx=0x7efdd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840ba7d6224358d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efdc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efdc3e00640 1 -- 172.21.15.44:0/2896178374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8012e60 con 0x5634483fc4b0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efdc3e00640 1 -- 172.21.15.44:0/2896178374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efdd8005020 con 0x5634483fc4b0 2024-04-20T11:27:51.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd00be7d0 con 0x5634483fc4b0 2024-04-20T11:27:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efdc3e00640 1 -- 172.21.15.44:0/2896178374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efdd8005220 con 0x5634483fc4b0 2024-04-20T11:27:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efde7fe4740 10 client.16785 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.588+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5634482e3820 con 0x5634483fc4b0 2024-04-20T11:27:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efdc3e00640 1 -- 172.21.15.44:0/2896178374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdd00be7d0 con 0x5634483fc4b0 2024-04-20T11:27:51.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efdc3e00640 1 -- 172.21.15.44:0/2896178374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7efdd8027070 con 0x5634483fc4b0 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efde7fe4740 10 client.16785 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efde7fe4740 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efde7fe4740 20 client.16785 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344840c400 0x56344840e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efde7fe4740 4 client.16785 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563448365a70 con 0x56344840c400 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.589+0000 7efddde00640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344840c400 0x56344840e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.590+0000 7efddde00640 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344840c400 0x56344840e860 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1ca0ca728e4d703 server_cookie=2fea5fe948d4ea72 in_seq=0 out_seq=0 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.590+0000 7efdc3e00640 10 client.16785 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efdc3e00640 1 -- 172.21.15.44:0/2896178374 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563448365a70 con 0x56344840c400 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efdc3e00640 10 client.16785 handle_command_reply: tid=0 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 client.16785 shutdown 2024-04-20T11:27:51.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 msgr2=0x5634483fa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 0x5634483fa030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efdd8004590 tx=0x7efdd8004730 comp rx=0 tx=0).stop 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 shutdown_connections 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344840c400 0x56344840e860 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483fc4b0 0x5634483fa030 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2896178374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483f2fb0 0x5634483f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efdc3e00640 0 client.16785 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 >> 172.21.15.44:0/2896178374 conn(0x5634483ed6e0 msgr2=0x5634483ef430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 shutdown_connections 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 1 -- 172.21.15.44:0/2896178374 wait complete. 2024-04-20T11:27:51.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.591+0000 7efde7fe4740 20 client.16785 trim_cache size 0 max 16384 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.606+0000 7efde7fe4740 1 Processor -- start 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.606+0000 7efde7fe4740 1 -- start start 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f2fb0 0x5634483fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 0x563448407e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634483f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efde7fe4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b03c0 con 0x5634483f38d0 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efde7fe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b2050 con 0x5634483fc4b0 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efde7fe4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b0a90 con 0x5634483f2fb0 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efdd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 0x563448407e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efdd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 0x563448407e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56868/0 (socket says 172.21.15.44:56868) 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efdd5400640 1 -- 172.21.15.44:0/3955842989 learned_addr learned my addr 172.21.15.44:0/3955842989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efddde00640 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f2fb0 0x5634483fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.607+0000 7efdde800640 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634483f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdd5400640 1 -- 172.21.15.44:0/3955842989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f2fb0 msgr2=0x5634483fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdd5400640 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f2fb0 0x5634483fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdd5400640 1 -- 172.21.15.44:0/3955842989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 msgr2=0x5634483f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdd5400640 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634483f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdd5400640 1 -- 172.21.15.44:0/3955842989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344806a3f0 con 0x5634483f38d0 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdde800640 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634483f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdd5400640 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 0x563448407e90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009ea0 tx=0x7efdc8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc1d6210a9bccd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdcea00640 1 -- 172.21.15.44:0/3955842989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8022070 con 0x5634483f38d0 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdcea00640 1 -- 172.21.15.44:0/3955842989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdc80035f0 con 0x5634483f38d0 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.608+0000 7efdcea00640 1 -- 172.21.15.44:0/3955842989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc801d6c0 con 0x5634483f38d0 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 -- 172.21.15.44:0/3955842989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 msgr2=0x563448407e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 0x563448407e90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009ea0 tx=0x7efdc8005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 -- 172.21.15.44:0/3955842989 shutdown_connections 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634483f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f38d0 0x563448407e90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 --2- 172.21.15.44:0/3955842989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f2fb0 0x5634483fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 -- 172.21.15.44:0/3955842989 >> 172.21.15.44:0/3955842989 conn(0x5634483ed6e0 msgr2=0x5634483eeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 -- 172.21.15.44:0/3955842989 shutdown_connections 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.609+0000 7efde7fe4740 1 -- 172.21.15.44:0/3955842989 wait complete. 2024-04-20T11:27:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 -- start start 2024-04-20T11:27:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 Processor -- start 2024-04-20T11:27:51.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x563448404900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634484003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b18c0 con 0x5634483f2fb0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b11c0 con 0x5634483fc4b0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efde01ae390 con 0x5634483f38d0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.610+0000 7efde7fe4740 10 client.0 _ll_register_callbacks cb 0x7efdd80044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x563448404900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x563448404900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56884/0 (socket says 172.21.15.44:56884) 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 -- 172.21.15.44:0/2505304103 learned_addr learned my addr 172.21.15.44:0/2505304103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efdd5400640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efdde800640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634484003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 -- 172.21.15.44:0/2505304103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 msgr2=0x5634483ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 -- 172.21.15.44:0/2505304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 msgr2=0x5634484003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634484003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 -- 172.21.15.44:0/2505304103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634481347d0 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efddde00640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x563448404900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ef8a0 tx=0x7efdd00347e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453361cd4be7fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.611+0000 7efdc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efdc3e00640 1 -- 172.21.15.44:0/2505304103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00bdc10 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efdc3e00640 1 -- 172.21.15.44:0/2505304103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efdd00f5760 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc80035f0 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efdc3e00640 1 -- 172.21.15.44:0/2505304103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efdd00bdea0 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 10 client.16791 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56344847b180 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efdc3e00640 1 -- 172.21.15.44:0/2505304103 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdc80035f0 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efdc3e00640 1 -- 172.21.15.44:0/2505304103 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7efdd003c030 con 0x5634483f2fb0 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 10 client.16791 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 10 client.16791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:27:51.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 20 client.16791 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344847b530 0x56344847d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 4 client.16791 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.612+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56344835f8e0 con 0x56344847b530 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.613+0000 7efdd5400640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344847b530 0x56344847d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.613+0000 7efdd5400640 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344847b530 0x56344847d920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4a2ea3074f13996 server_cookie=7af2e57c55a6d1d9 in_seq=0 out_seq=0 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.613+0000 7efdc3e00640 10 client.16791 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efdc3e00640 1 -- 172.21.15.44:0/2505304103 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56344835f8e0 con 0x56344847b530 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efdc3e00640 10 client.16791 handle_command_reply: tid=0 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 client.16791 shutdown 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 msgr2=0x563448404900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x563448404900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ef8a0 tx=0x7efdd00347e0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 shutdown_connections 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56344847b530 0x56344847d920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634483fc4b0 0x5634484003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634483f38d0 0x5634483ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efdc3e00640 0 client.16791 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 --2- 172.21.15.44:0/2505304103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634483f2fb0 0x563448404900 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 >> 172.21.15.44:0/2505304103 conn(0x5634483ed6e0 msgr2=0x5634483ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 shutdown_connections 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 1 -- 172.21.15.44:0/2505304103 wait complete. 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.614+0000 7efde7fe4740 20 client.16791 trim_cache size 0 max 16384 2024-04-20T11:27:51.619 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:51.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efdc40a3da0 msgr2=0x7efdc40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efdc40a3da0 0x7efdc40a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009e60 tx=0x7efdd8005f90 comp rx=0 tx=0).stop 2024-04-20T11:27:51.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 msgr2=0x7efde019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde019adc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdd0008000 tx=0x7efdd0034440 comp rx=0 tx=0).stop 2024-04-20T11:27:51.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 shutdown_connections 2024-04-20T11:27:51.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde019fbc0 0x7efde01a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efdc40a3da0 0x7efdc40a6260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde019b300 0x7efde019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 --2- 172.21.15.44:0/418438321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efde01094e0 0x7efde019adc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 >> 172.21.15.44:0/418438321 conn(0x7efde00ffcf0 msgr2=0x7efde0101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 shutdown_connections 2024-04-20T11:27:51.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.615+0000 7efde5200640 1 -- 172.21.15.44:0/418438321 wait complete. 2024-04-20T11:27:51.631 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:51 smithi044 ceph-mon[16855]: pgmap v160: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:51.631 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:51 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3741943062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:27:51.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:27:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:51 smithi078 ceph-mon[21464]: pgmap v160: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:51.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:51 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3741943062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:27:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:51 smithi165 ceph-mon[21975]: pgmap v160: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:51 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3741943062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:27:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 -- 172.21.15.44:0/2420134288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1016a0 msgr2=0x7fef0c10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 --2- 172.21.15.44:0/2420134288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1016a0 0x7fef0c10e3f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feef4009900 tx=0x7feef402bc40 comp rx=0 tx=0).stop 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 -- 172.21.15.44:0/2420134288 shutdown_connections 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 --2- 172.21.15.44:0/2420134288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c10e930 0x7fef0c110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 --2- 172.21.15.44:0/2420134288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1016a0 0x7fef0c10e3f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 --2- 172.21.15.44:0/2420134288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c100d80 0x7fef0c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 -- 172.21.15.44:0/2420134288 >> 172.21.15.44:0/2420134288 conn(0x7fef0c077fc0 msgr2=0x7fef0c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.849+0000 7fef11200640 1 -- 172.21.15.44:0/2420134288 shutdown_connections 2024-04-20T11:27:51.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.850+0000 7fef11200640 1 -- 172.21.15.44:0/2420134288 wait complete. 2024-04-20T11:27:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.850+0000 7fef11200640 1 Processor -- start 2024-04-20T11:27:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 -- start start 2024-04-20T11:27:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 0x7fef0c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c1016a0 0x7fef0c1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c10e930 0x7fef0c103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c114680 con 0x7fef0c10e930 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c114500 con 0x7fef0c100d80 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef11200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c114800 con 0x7fef0c1016a0 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef0a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c10e930 0x7fef0c103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef09e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 0x7fef0c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef09e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 0x7fef0c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39668/0 (socket says 172.21.15.44:39668) 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.851+0000 7fef09e00640 1 -- 172.21.15.44:0/2265286995 learned_addr learned my addr 172.21.15.44:0/2265286995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef01400640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c1016a0 0x7fef0c1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef09e00640 1 -- 172.21.15.44:0/2265286995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c1016a0 msgr2=0x7fef0c1035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef09e00640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c1016a0 0x7fef0c1035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef09e00640 1 -- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c10e930 msgr2=0x7fef0c103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef09e00640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c10e930 0x7fef0c103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef09e00640 1 -- 172.21.15.44:0/2265286995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c1ab1b0 con 0x7fef0c100d80 2024-04-20T11:27:51.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.852+0000 7fef09e00640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 0x7fef0c103060 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feefc00ea20 tx=0x7feefc00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.853+0000 7fef03400640 1 -- 172.21.15.44:0/2265286995 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc00cd80 con 0x7fef0c100d80 2024-04-20T11:27:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.853+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c1ab4a0 con 0x7fef0c100d80 2024-04-20T11:27:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.853+0000 7fef03400640 1 -- 172.21.15.44:0/2265286995 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feefc010930 con 0x7fef0c100d80 2024-04-20T11:27:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.853+0000 7fef03400640 1 -- 172.21.15.44:0/2265286995 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc018a30 con 0x7fef0c100d80 2024-04-20T11:27:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.853+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c1aba10 con 0x7fef0c100d80 2024-04-20T11:27:51.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.856+0000 7fef13fe3740 1 Processor -- start 2024-04-20T11:27:51.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.856+0000 7fef13fe3740 1 -- start start 2024-04-20T11:27:51.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef13fe3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f3e21c050 con 0x558f3e2b78d0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef13fe3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f3e1c2490 con 0x558f3e2b6fb0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef13fe3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f3e218010 con 0x558f3e2c04b0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef0a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef03400640 1 -- 172.21.15.44:0/2265286995 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7feefc010ae0 con 0x7fef0c100d80 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef0a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39674/0 (socket says 172.21.15.44:39674) 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef0a800640 1 -- 172.21.15.44:0/3219375534 learned_addr learned my addr 172.21.15.44:0/3219375534 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.857+0000 7fef01400640 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef09e00640 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef0a800640 1 -- 172.21.15.44:0/3219375534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 msgr2=0x558f3e2c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef0a800640 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef0a800640 1 -- 172.21.15.44:0/3219375534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 msgr2=0x558f3e2bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef0a800640 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef0a800640 1 -- 172.21.15.44:0/3219375534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3e1ca070 con 0x558f3e2b6fb0 2024-04-20T11:27:51.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef09e00640 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7fef0a800640 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2b7390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef04009f30 tx=0x7fef040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eacc2de55a8577d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.858+0000 7feefb400640 1 -- 172.21.15.44:0/3219375534 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04004d20 con 0x558f3e2b6fb0 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7feefb400640 1 -- 172.21.15.44:0/3219375534 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fef04004050 con 0x558f3e2b6fb0 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7feefb400640 1 -- 172.21.15.44:0/3219375534 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04005280 con 0x558f3e2b6fb0 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 -- 172.21.15.44:0/3219375534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 msgr2=0x558f3e2b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2b7390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef04009f30 tx=0x7fef040118d0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 -- 172.21.15.44:0/3219375534 shutdown_connections 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3219375534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2b7390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.859+0000 7fef13fe3740 1 -- 172.21.15.44:0/3219375534 >> 172.21.15.44:0/3219375534 conn(0x558f3e2b16e0 msgr2=0x558f3e2b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.860+0000 7fef03400640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feef00a3da0 0x7feef00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.860+0000 7fef13fe3740 1 -- 172.21.15.44:0/3219375534 shutdown_connections 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.860+0000 7fef13fe3740 1 -- 172.21.15.44:0/3219375534 wait complete. 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.860+0000 7fef03400640 1 -- 172.21.15.44:0/2265286995 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7feefc0d0ad0 con 0x7fef0c100d80 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.860+0000 7fef01400640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feef00a3da0 0x7feef00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.860+0000 7fef13fe3740 1 -- start start 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 Processor -- start 2024-04-20T11:27:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef01400640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feef00a3da0 0x7feef00a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feef4005ab0 tx=0x7feef4005a20 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2b78d0 0x558f3e2bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2c04b0 0x558f3e2bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f3e2c32a0 con 0x558f3e2c04b0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f3e1d5860 con 0x558f3e2b6fb0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f3e2c3420 con 0x558f3e2b78d0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef13fe3740 10 client.0 _ll_register_callbacks cb 0x558f3e1a3fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.861+0000 7fef09e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef0a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2c04b0 0x558f3e2bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2bd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39686/0 (socket says 172.21.15.44:39686) 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef01400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2b78d0 0x558f3e2bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 -- 172.21.15.44:0/790405371 learned_addr learned my addr 172.21.15.44:0/790405371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 -- 172.21.15.44:0/790405371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2b78d0 msgr2=0x558f3e2bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2b78d0 0x558f3e2bd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 -- 172.21.15.44:0/790405371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2c04b0 msgr2=0x558f3e2bdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2c04b0 0x558f3e2bdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 -- 172.21.15.44:0/790405371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3e1d9950 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7fef09e00640 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2bd490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feefc09c890 tx=0x7feefc09c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2b2231acf93f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.862+0000 7feeefe00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7feeefe00640 1 -- 172.21.15.44:0/790405371 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0a0e90 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7feeefe00640 1 -- 172.21.15.44:0/790405371 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7feefc018bd0 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef04004050 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7feeefe00640 1 -- 172.21.15.44:0/790405371 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feefc0a5750 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7fef13fe3740 10 client.26930 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558f3e1a7820 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.863+0000 7feeefe00640 1 -- 172.21.15.44:0/790405371 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fef04004050 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7feeefe00640 1 -- 172.21.15.44:0/790405371 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7feefc0988e0 con 0x558f3e2b6fb0 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef13fe3740 10 client.26930 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef13fe3740 10 client.26930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef13fe3740 20 client.26930 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef13fe3740 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e2d0360 0x558f3e2d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef13fe3740 4 client.26930 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3e229a70 con 0x558f3e2d0360 2024-04-20T11:27:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.864+0000 7fef01400640 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e2d0360 0x558f3e2d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.865+0000 7fef01400640 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e2d0360 0x558f3e2d27e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16398b11def68e6 server_cookie=3d715b4ec47e9429 in_seq=0 out_seq=0 2024-04-20T11:27:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.865+0000 7feeefe00640 10 client.26930 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.865+0000 7feeefe00640 1 -- 172.21.15.44:0/790405371 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f3e229a70 con 0x558f3e2d0360 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.865+0000 7feeefe00640 10 client.26930 handle_command_reply: tid=0 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 client.26930 shutdown 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 msgr2=0x558f3e2bd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2bd490 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feefc09c890 tx=0x7feefc09c6d0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 shutdown_connections 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e2d0360 0x558f3e2d27e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2c04b0 0x558f3e2bdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2b78d0 0x558f3e2bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 --2- 172.21.15.44:0/790405371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2bd490 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7feeefe00640 0 client.26930 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 >> 172.21.15.44:0/790405371 conn(0x558f3e2b16e0 msgr2=0x558f3e2b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 shutdown_connections 2024-04-20T11:27:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 1 -- 172.21.15.44:0/790405371 wait complete. 2024-04-20T11:27:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.866+0000 7fef13fe3740 20 client.26930 trim_cache size 0 max 16384 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.882+0000 7fef13fe3740 1 Processor -- start 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.882+0000 7fef13fe3740 1 -- start start 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef13fe3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1b42d0 con 0x558f3e2b78d0 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef13fe3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1b5f60 con 0x558f3e2b6fb0 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef13fe3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1b49a0 con 0x558f3e2c04b0 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef01400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef09e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef09e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39694/0 (socket says 172.21.15.44:39694) 2024-04-20T11:27:51.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef09e00640 1 -- 172.21.15.44:0/3763086725 learned_addr learned my addr 172.21.15.44:0/3763086725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.883+0000 7fef0a800640 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7fef09e00640 1 -- 172.21.15.44:0/3763086725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 msgr2=0x558f3e2d80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7fef09e00640 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2d80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7fef09e00640 1 -- 172.21.15.44:0/3763086725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 msgr2=0x558f3e2bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7fef09e00640 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7fef09e00640 1 -- 172.21.15.44:0/3763086725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3e2b7700 con 0x558f3e2b6fb0 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7fef09e00640 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2cc160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feefc009830 tx=0x7feefc010dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b0676061e45dbf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7feefaa00640 1 -- 172.21.15.44:0/3763086725 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc09c4f0 con 0x558f3e2b6fb0 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7feefaa00640 1 -- 172.21.15.44:0/3763086725 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feefc0988e0 con 0x558f3e2b6fb0 2024-04-20T11:27:51.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.884+0000 7feefaa00640 1 -- 172.21.15.44:0/3763086725 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc09c240 con 0x558f3e2b6fb0 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 -- 172.21.15.44:0/3763086725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 msgr2=0x558f3e2cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2cc160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feefc009830 tx=0x7feefc010dd0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 -- 172.21.15.44:0/3763086725 shutdown_connections 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2d80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b78d0 0x558f3e2bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 --2- 172.21.15.44:0/3763086725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b6fb0 0x558f3e2cc160 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 -- 172.21.15.44:0/3763086725 >> 172.21.15.44:0/3763086725 conn(0x558f3e2b16e0 msgr2=0x558f3e2bdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 -- 172.21.15.44:0/3763086725 shutdown_connections 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.885+0000 7fef13fe3740 1 -- 172.21.15.44:0/3763086725 wait complete. 2024-04-20T11:27:51.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.886+0000 7fef13fe3740 1 -- start start 2024-04-20T11:27:51.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.886+0000 7fef13fe3740 1 Processor -- start 2024-04-20T11:27:51.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b6fb0 0x558f3e2c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 0x558f3e2c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1b57d0 con 0x558f3e2b6fb0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1b50d0 con 0x558f3e2b78d0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1b2260 con 0x558f3e2c04b0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef13fe3740 10 client.0 _ll_register_callbacks cb 0x7feefc098e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef01400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 0x558f3e2c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef01400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 0x558f3e2c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39706/0 (socket says 172.21.15.44:39706) 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef0a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef01400640 1 -- 172.21.15.44:0/529644249 learned_addr learned my addr 172.21.15.44:0/529644249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.887+0000 7fef09e00640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b6fb0 0x558f3e2c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef01400640 1 -- 172.21.15.44:0/529644249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 msgr2=0x558f3e2c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef01400640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef01400640 1 -- 172.21.15.44:0/529644249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b6fb0 msgr2=0x558f3e2c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef01400640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b6fb0 0x558f3e2c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef01400640 1 -- 172.21.15.44:0/529644249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3df2e3f0 con 0x558f3e2b78d0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef0a800640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef01400640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 0x558f3e2c3cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feef4009230 tx=0x7feef402bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f840c16a65b53911 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7feeefe00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7feeefe00640 1 -- 172.21.15.44:0/529644249 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4058060 con 0x558f3e2b78d0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7feeefe00640 1 -- 172.21.15.44:0/529644249 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7feef4052250 con 0x558f3e2b78d0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.888+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feefc0988e0 con 0x558f3e2b78d0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7feeefe00640 1 -- 172.21.15.44:0/529644249 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7feef40044d0 con 0x558f3e2b78d0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7fef13fe3740 10 client.26936 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558f3dff87d0 con 0x558f3e2b78d0 2024-04-20T11:27:51.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7feeefe00640 1 -- 172.21.15.44:0/529644249 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feefc0988e0 con 0x558f3e2b78d0 2024-04-20T11:27:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7feeefe00640 1 -- 172.21.15.44:0/529644249 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7feef405b020 con 0x558f3e2b78d0 2024-04-20T11:27:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7fef13fe3740 10 client.26936 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.889+0000 7fef13fe3740 10 client.26936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7fef13fe3740 20 client.26936 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:51.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7fef13fe3740 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e336d10 0x558f3e339130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7fef13fe3740 4 client.26936 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f3e2238e0 con 0x558f3e336d10 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7fef09e00640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e336d10 0x558f3e339130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7fef09e00640 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e336d10 0x558f3e339130 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbe72e279da8edd server_cookie=b30d56501508a7fa in_seq=0 out_seq=0 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.890+0000 7feeefe00640 10 client.26936 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7feeefe00640 1 -- 172.21.15.44:0/529644249 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f3e2238e0 con 0x558f3e336d10 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7feeefe00640 10 client.26936 handle_command_reply: tid=0 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 client.26936 shutdown 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 msgr2=0x558f3e2c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 0x558f3e2c3cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feef4009230 tx=0x7feef402bc40 comp rx=0 tx=0).stop 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 shutdown_connections 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558f3e336d10 0x558f3e339130 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f3e2c04b0 0x558f3e2c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f3e2b78d0 0x558f3e2c3cd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 --2- 172.21.15.44:0/529644249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f3e2b6fb0 0x558f3e2c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 >> 172.21.15.44:0/529644249 conn(0x558f3e2b16e0 msgr2=0x558f3e2afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7feeefe00640 0 client.26936 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.891+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 shutdown_connections 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef13fe3740 1 -- 172.21.15.44:0/529644249 wait complete. 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef13fe3740 20 client.26936 trim_cache size 0 max 16384 2024-04-20T11:27:51.895 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feef00a3da0 msgr2=0x7feef00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feef00a3da0 0x7feef00a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feef4005ab0 tx=0x7feef4005a20 comp rx=0 tx=0).stop 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 msgr2=0x7fef0c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 0x7fef0c103060 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feefc00ea20 tx=0x7feefc00eee0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 shutdown_connections 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feef00a3da0 0x7feef00a6260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c10e930 0x7fef0c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c1016a0 0x7fef0c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 --2- 172.21.15.44:0/2265286995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef0c100d80 0x7fef0c103060 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.892+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 >> 172.21.15.44:0/2265286995 conn(0x7fef0c077fc0 msgr2=0x7fef0c10fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:51.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.893+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 shutdown_connections 2024-04-20T11:27:51.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:51.893+0000 7fef11200640 1 -- 172.21.15.44:0/2265286995 wait complete. 2024-04-20T11:27:52.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:52 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:27:52.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:52 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:27:52.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:52 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:27:52.912 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:27:52.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:27:52.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:27:53.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.142+0000 7fae4ec00640 1 -- 172.21.15.44:0/2224987254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074330 msgr2=0x7fae48074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.142+0000 7fae4ec00640 1 --2- 172.21.15.44:0/2224987254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074330 0x7fae48074710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fae38009900 tx=0x7fae3802bcd0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 -- 172.21.15.44:0/2224987254 shutdown_connections 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 --2- 172.21.15.44:0/2224987254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4810e6a0 0x7fae48110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 --2- 172.21.15.44:0/2224987254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae48074c50 0x7fae4810e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 --2- 172.21.15.44:0/2224987254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074330 0x7fae48074710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 -- 172.21.15.44:0/2224987254 >> 172.21.15.44:0/2224987254 conn(0x7fae480fd930 msgr2=0x7fae480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 -- 172.21.15.44:0/2224987254 shutdown_connections 2024-04-20T11:27:53.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.143+0000 7fae4ec00640 1 -- 172.21.15.44:0/2224987254 wait complete. 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.144+0000 7fae4ec00640 1 Processor -- start 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.144+0000 7fae4ec00640 1 -- start start 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.144+0000 7fae4ec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae48074330 0x7fae481a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.144+0000 7fae4ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 0x7fae481a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.144+0000 7fae4ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4810e6a0 0x7fae4819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae48114560 con 0x7fae48074c50 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae481143e0 con 0x7fae48074330 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae481146e0 con 0x7fae4810e6a0 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 0x7fae481a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 0x7fae481a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56956/0 (socket says 172.21.15.44:56956) 2024-04-20T11:27:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 -- 172.21.15.44:0/3589940398 learned_addr learned my addr 172.21.15.44:0/3589940398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4d800640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae48074330 0x7fae481a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4e200640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4810e6a0 0x7fae4819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 -- 172.21.15.44:0/3589940398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4810e6a0 msgr2=0x7fae4819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4810e6a0 0x7fae4819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 -- 172.21.15.44:0/3589940398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae48074330 msgr2=0x7fae481a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae48074330 0x7fae481a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.145+0000 7fae4ce00640 1 -- 172.21.15.44:0/3589940398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4819afd0 con 0x7fae48074c50 2024-04-20T11:27:53.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.146+0000 7fae4ce00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 0x7fae481a0bd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fae3400d950 tx=0x7fae3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.146+0000 7fae46000640 1 -- 172.21.15.44:0/3589940398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34017030 con 0x7fae48074c50 2024-04-20T11:27:53.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.146+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4819b2c0 con 0x7fae48074c50 2024-04-20T11:27:53.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.146+0000 7fae46000640 1 -- 172.21.15.44:0/3589940398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae34004020 con 0x7fae48074c50 2024-04-20T11:27:53.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.146+0000 7fae46000640 1 -- 172.21.15.44:0/3589940398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34004a00 con 0x7fae48074c50 2024-04-20T11:27:53.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.146+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae481a74e0 con 0x7fae48074c50 2024-04-20T11:27:53.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.149+0000 7fae52f80740 1 Processor -- start 2024-04-20T11:27:53.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 -- start start 2024-04-20T11:27:53.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e1f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e285f0 0x55eca9e2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eca9d83d20 con 0x55eca9e1f0f0 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eca9d2a3e0 con 0x55eca9e1fa10 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.150+0000 7fae52f80740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eca9d7fce0 con 0x55eca9e285f0 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e1f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e1f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56962/0 (socket says 172.21.15.44:56962) 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39724/0 (socket says 172.21.15.44:39724) 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4e200640 1 -- 172.21.15.44:0/1872737819 learned_addr learned my addr 172.21.15.44:0/1872737819 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4ce00640 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e285f0 0x55eca9e2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 -- 172.21.15.44:0/1872737819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e285f0 msgr2=0x55eca9e2a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e285f0 0x55eca9e2a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 -- 172.21.15.44:0/1872737819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 msgr2=0x55eca9e1f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e1f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4d800640 1 -- 172.21.15.44:0/1872737819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca9d49980 con 0x55eca9e1fa10 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.151+0000 7fae4e200640 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e1f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:53.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.152+0000 7fae4d800640 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e280b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fae38000c00 tx=0x7fae3802c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8817298332a611 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.152+0000 7fae3ea00640 1 -- 172.21.15.44:0/1872737819 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3802c8a0 con 0x55eca9e1fa10 2024-04-20T11:27:53.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.152+0000 7fae3ea00640 1 -- 172.21.15.44:0/1872737819 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae38004040 con 0x55eca9e1fa10 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.152+0000 7fae3ea00640 1 -- 172.21.15.44:0/1872737819 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38004920 con 0x55eca9e1fa10 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 -- 172.21.15.44:0/1872737819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 msgr2=0x55eca9e280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e280b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fae38000c00 tx=0x7fae3802c6b0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 -- 172.21.15.44:0/1872737819 shutdown_connections 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e285f0 0x55eca9e2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e280b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 --2- 172.21.15.44:0/1872737819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e1f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.153+0000 7fae52f80740 1 -- 172.21.15.44:0/1872737819 >> 172.21.15.44:0/1872737819 conn(0x55eca9e19820 msgr2=0x55eca9e19c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.154+0000 7fae46000640 1 -- 172.21.15.44:0/3589940398 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fae340041d0 con 0x7fae48074c50 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.154+0000 7fae52f80740 1 -- 172.21.15.44:0/1872737819 shutdown_connections 2024-04-20T11:27:53.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.154+0000 7fae52f80740 1 -- 172.21.15.44:0/1872737819 wait complete. 2024-04-20T11:27:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.155+0000 7fae52f80740 1 -- start start 2024-04-20T11:27:53.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.155+0000 7fae52f80740 1 Processor -- start 2024-04-20T11:27:53.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e25620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 0x55eca9e25b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e260a0 0x55eca9e347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eca9e2b380 con 0x55eca9e1f0f0 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eca9d3d7a0 con 0x55eca9e260a0 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eca9e2b500 con 0x55eca9e1fa10 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae52f80740 10 client.0 _ll_register_callbacks cb 0x55eca9d3fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae4d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 0x55eca9e25b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae4e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e25620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e260a0 0x55eca9e347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae4d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 0x55eca9e25b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41190/0 (socket says 172.21.15.44:41190) 2024-04-20T11:27:53.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.156+0000 7fae4d800640 1 -- 172.21.15.44:0/1767554207 learned_addr learned my addr 172.21.15.44:0/1767554207 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae46000640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fae2c0a3c30 0x7fae2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4d800640 1 -- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e260a0 msgr2=0x55eca9e347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4d800640 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e260a0 0x55eca9e347f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4d800640 1 -- 172.21.15.44:0/1767554207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 msgr2=0x55eca9e25620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4d800640 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e25620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4d800640 1 -- 172.21.15.44:0/1767554207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca9b5e7e0 con 0x55eca9e1fa10 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae46000640 1 -- 172.21.15.44:0/3589940398 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fae340d5a80 con 0x7fae48074c50 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4e200640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fae2c0a3c30 0x7fae2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae4d800640 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 0x55eca9e25b60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae3802c210 tx=0x7fae38005380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e12d2a5934d85d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.157+0000 7fae33400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae33400640 1 -- 172.21.15.44:0/1767554207 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380035f0 con 0x55eca9e1fa10 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae38004040 con 0x55eca9e1fa10 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae33400640 1 -- 172.21.15.44:0/1767554207 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fae38003790 con 0x55eca9e1fa10 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae4e200640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fae2c0a3c30 0x7fae2c0a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae40017fd0 tx=0x7fae40004ab0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae33400640 1 -- 172.21.15.44:0/1767554207 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fae38003990 con 0x55eca9e1fa10 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae52f80740 10 client.26947 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.158+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eca9bbef70 con 0x55eca9e1fa10 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae33400640 1 -- 172.21.15.44:0/1767554207 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae38004040 con 0x55eca9e1fa10 2024-04-20T11:27:53.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae33400640 1 -- 172.21.15.44:0/1767554207 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7fae38034440 con 0x55eca9e1fa10 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae52f80740 10 client.26947 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae52f80740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae52f80740 20 client.26947 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae52f80740 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e38c10 0x55eca9e3b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae52f80740 4 client.26947 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.159+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eca9d91740 con 0x55eca9e38c10 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.160+0000 7fae4ce00640 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e38c10 0x55eca9e3b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.160+0000 7fae4ce00640 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e38c10 0x55eca9e3b000 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb5d6fb8f933546 server_cookie=79a58ccb60e89a59 in_seq=0 out_seq=0 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.160+0000 7fae33400640 10 client.26947 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae33400640 1 -- 172.21.15.44:0/1767554207 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eca9d91740 con 0x55eca9e38c10 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae33400640 10 client.26947 handle_command_reply: tid=0 2024-04-20T11:27:53.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 client.26947 shutdown 2024-04-20T11:27:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 msgr2=0x55eca9e25b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 0x55eca9e25b60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae3802c210 tx=0x7fae38005380 comp rx=0 tx=0).stop 2024-04-20T11:27:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 shutdown_connections 2024-04-20T11:27:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e38c10 0x55eca9e3b000 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e260a0 0x55eca9e347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e1fa10 0x55eca9e25b60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 --2- 172.21.15.44:0/1767554207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e25620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 >> 172.21.15.44:0/1767554207 conn(0x55eca9e19820 msgr2=0x55eca9e1b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae33400640 0 client.26947 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 shutdown_connections 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 1 -- 172.21.15.44:0/1767554207 wait complete. 2024-04-20T11:27:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.161+0000 7fae52f80740 20 client.26947 trim_cache size 0 max 16384 2024-04-20T11:27:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 Processor -- start 2024-04-20T11:27:53.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 -- start start 2024-04-20T11:27:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e40690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e40bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e34240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae481ab8b0 con 0x55eca9e1f0f0 2024-04-20T11:27:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae481ad540 con 0x55eca9e1fa10 2024-04-20T11:27:53.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae52f80740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae481abf80 con 0x55eca9e260a0 2024-04-20T11:27:53.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.181+0000 7fae4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e40690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e40bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e40690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56980/0 (socket says 172.21.15.44:56980) 2024-04-20T11:27:53.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4d800640 1 -- 172.21.15.44:0/1119629757 learned_addr learned my addr 172.21.15.44:0/1119629757 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4e200640 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e34240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 -- 172.21.15.44:0/1119629757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 msgr2=0x55eca9e34240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e34240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 -- 172.21.15.44:0/1119629757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 msgr2=0x55eca9e40690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e40690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 -- 172.21.15.44:0/1119629757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca9a56170 con 0x55eca9e1fa10 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4d800640 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e40690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.182+0000 7fae4ce00640 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e40bd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fae340d31e0 tx=0x7fae34021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad35ebf0a95503e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae3e000640 1 -- 172.21.15.44:0/1119629757 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae340a1390 con 0x55eca9e1fa10 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae3e000640 1 -- 172.21.15.44:0/1119629757 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae340a2130 con 0x55eca9e1fa10 2024-04-20T11:27:53.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae3e000640 1 -- 172.21.15.44:0/1119629757 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae340c20b0 con 0x55eca9e1fa10 2024-04-20T11:27:53.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 -- 172.21.15.44:0/1119629757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 msgr2=0x55eca9e40bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e40bd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fae340d31e0 tx=0x7fae34021000 comp rx=0 tx=0).stop 2024-04-20T11:27:53.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 -- 172.21.15.44:0/1119629757 shutdown_connections 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e34240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e40bd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 --2- 172.21.15.44:0/1119629757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e40690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.183+0000 7fae52f80740 1 -- 172.21.15.44:0/1119629757 >> 172.21.15.44:0/1119629757 conn(0x55eca9e19820 msgr2=0x55eca9e18a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.184+0000 7fae52f80740 1 -- 172.21.15.44:0/1119629757 shutdown_connections 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.184+0000 7fae52f80740 1 -- 172.21.15.44:0/1119629757 wait complete. 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.184+0000 7fae52f80740 1 -- start start 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.184+0000 7fae52f80740 1 Processor -- start 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e2c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae481acdb0 con 0x55eca9e1f0f0 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae481ac6b0 con 0x55eca9e1fa10 2024-04-20T11:27:53.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae481a9820 con 0x55eca9e260a0 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae52f80740 10 client.0 _ll_register_callbacks cb 0x7fae40017bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e2c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e2c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56986/0 (socket says 172.21.15.44:56986) 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4d800640 1 -- 172.21.15.44:0/3212235807 learned_addr learned my addr 172.21.15.44:0/3212235807 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4e200640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4ce00640 1 -- 172.21.15.44:0/3212235807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 msgr2=0x55eca9e2cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4ce00640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e2cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4ce00640 1 -- 172.21.15.44:0/3212235807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 msgr2=0x55eca9e2c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4ce00640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e2c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4ce00640 1 -- 172.21.15.44:0/3212235807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca9e9df00 con 0x55eca9e1fa10 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.185+0000 7fae4d800640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e2c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae4ce00640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e2c5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fae340d60b0 tx=0x7fae34004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb185f966df9684a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae33400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae33400640 1 -- 172.21.15.44:0/3212235807 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3409fb70 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae340a2130 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae33400640 1 -- 172.21.15.44:0/3212235807 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fae3408c0b0 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae33400640 1 -- 172.21.15.44:0/3212235807 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fae34085050 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae52f80740 10 client.26954 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.186+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eca9e9f180 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae33400640 1 -- 172.21.15.44:0/3212235807 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae340a2130 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae33400640 1 -- 172.21.15.44:0/3212235807 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7fae340a0e80 con 0x55eca9e1fa10 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae52f80740 10 client.26954 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae52f80740 10 client.26954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae52f80740 20 client.26954 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae52f80740 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e9f8d0 0x55eca9ea1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae52f80740 4 client.26954 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:27:53.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eca9d8b5b0 con 0x55eca9e9f8d0 2024-04-20T11:27:53.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.187+0000 7fae4d800640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e9f8d0 0x55eca9ea1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.188+0000 7fae4d800640 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e9f8d0 0x55eca9ea1cf0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7008e462f13600 server_cookie=db1b7b2665f9bdde in_seq=0 out_seq=0 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.188+0000 7fae33400640 10 client.26954 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae33400640 1 -- 172.21.15.44:0/3212235807 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eca9d8b5b0 con 0x55eca9e9f8d0 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae33400640 10 client.26954 handle_command_reply: tid=0 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 client.26954 shutdown 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 msgr2=0x55eca9e2c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e2c5f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fae340d60b0 tx=0x7fae34004540 comp rx=0 tx=0).stop 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 shutdown_connections 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eca9e9f8d0 0x55eca9ea1cf0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eca9e260a0 0x55eca9e2cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eca9e1fa10 0x55eca9e2c5f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 --2- 172.21.15.44:0/3212235807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eca9e1f0f0 0x55eca9e2c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 >> 172.21.15.44:0/3212235807 conn(0x55eca9e19820 msgr2=0x55eca9e24e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.189+0000 7fae33400640 0 client.26954 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.190+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 shutdown_connections 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.190+0000 7fae52f80740 1 -- 172.21.15.44:0/3212235807 wait complete. 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.190+0000 7fae52f80740 20 client.26954 trim_cache size 0 max 16384 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "cf163c3d-a2fc-4c84-a3e3-9c64bbbbddae", 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:27:53.193 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fae2c0a3c30 msgr2=0x7fae2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fae2c0a3c30 0x7fae2c0a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae40017fd0 tx=0x7fae40004ab0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 msgr2=0x7fae481a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 0x7fae481a0bd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fae3400d950 tx=0x7fae3400de10 comp rx=0 tx=0).stop 2024-04-20T11:27:53.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 shutdown_connections 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4810e6a0 0x7fae4819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fae2c0a3c30 0x7fae2c0a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae48074c50 0x7fae481a0bd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 --2- 172.21.15.44:0/3589940398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae48074330 0x7fae481a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 >> 172.21.15.44:0/3589940398 conn(0x7fae480fd930 msgr2=0x7fae480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 shutdown_connections 2024-04-20T11:27:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.191+0000 7fae4ec00640 1 -- 172.21.15.44:0/3589940398 wait complete. 2024-04-20T11:27:53.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:27:53.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 -- 172.21.15.44:0/1880920397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc714104410 msgr2=0x7fc7141047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 --2- 172.21.15.44:0/1880920397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc714104410 0x7fc7141047f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc708009900 tx=0x7fc70802bc50 comp rx=0 tx=0).stop 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 -- 172.21.15.44:0/1880920397 shutdown_connections 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 --2- 172.21.15.44:0/1880920397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7141094e0 0x7fc7141110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 --2- 172.21.15.44:0/1880920397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc714104dc0 0x7fc714108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 --2- 172.21.15.44:0/1880920397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc714104410 0x7fc7141047f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 -- 172.21.15.44:0/1880920397 >> 172.21.15.44:0/1880920397 conn(0x7fc7140ffcf0 msgr2=0x7fc714102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 -- 172.21.15.44:0/1880920397 shutdown_connections 2024-04-20T11:27:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.401+0000 7fc71c600640 1 -- 172.21.15.44:0/1880920397 wait complete. 2024-04-20T11:27:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.402+0000 7fc71c600640 1 Processor -- start 2024-04-20T11:27:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.402+0000 7fc71c600640 1 -- start start 2024-04-20T11:27:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71c600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc714104410 0x7fc7141a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 0x7fc7141a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71c600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc71419a9f0 0x7fc71419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141146d0 con 0x7fc7141094e0 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71c600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc714114550 con 0x7fc71419a9f0 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71c600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc714114850 con 0x7fc714104410 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 0x7fc7141a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 0x7fc7141a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57012/0 (socket says 172.21.15.44:57012) 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 -- 172.21.15.44:0/712060144 learned_addr learned my addr 172.21.15.44:0/712060144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719c00640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc714104410 0x7fc7141a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc71a600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc71419a9f0 0x7fc71419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 -- 172.21.15.44:0/712060144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc714104410 msgr2=0x7fc7141a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc714104410 0x7fc7141a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 -- 172.21.15.44:0/712060144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc71419a9f0 msgr2=0x7fc71419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc71419a9f0 0x7fc71419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.403+0000 7fc719200640 1 -- 172.21.15.44:0/712060144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71419b510 con 0x7fc7141094e0 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.404+0000 7fc719200640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 0x7fc7141a0d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e9f0 tx=0x7fc70000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.404+0000 7fc712000640 1 -- 172.21.15.44:0/712060144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000ccc0 con 0x7fc7141094e0 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.404+0000 7fc712000640 1 -- 172.21.15.44:0/712060144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc700005520 con 0x7fc7141094e0 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.404+0000 7fc712000640 1 -- 172.21.15.44:0/712060144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700010690 con 0x7fc7141094e0 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.404+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71419b800 con 0x7fc7141094e0 2024-04-20T11:27:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.404+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7141a7a50 con 0x7fc7141094e0 2024-04-20T11:27:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.408+0000 7fc71f370740 1 Processor -- start 2024-04-20T11:27:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.408+0000 7fc71f370740 1 -- start start 2024-04-20T11:27:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.408+0000 7fc712000640 1 -- 172.21.15.44:0/712060144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc700004f70 con 0x7fc7141094e0 2024-04-20T11:27:53.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71f370740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbc4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71f370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71f370740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71f370740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5596cfb2a150 con 0x5596cfbcd6a0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71f370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5596cfad0370 con 0x5596cfbc4ac0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71f370740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5596cfb26110 con 0x5596cfbc41a0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbc4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc719c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbc4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41232/0 (socket says 172.21.15.44:41232) 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71a600640 1 -- 172.21.15.44:0/1878865660 learned_addr learned my addr 172.21.15.44:0/1878865660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc719200640 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71a600640 1 -- 172.21.15.44:0/1878865660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 msgr2=0x5596cfbcd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.409+0000 7fc71a600640 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc71a600640 1 -- 172.21.15.44:0/1878865660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 msgr2=0x5596cfbcfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc71a600640 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc71a600640 1 -- 172.21.15.44:0/1878865660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cf9067d0 con 0x5596cfbc41a0 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc719200640 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:27:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc719c00640 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc71a600640 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbc4580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009f50 tx=0x7fc70c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b75eaa7c72b30fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.410+0000 7fc712000640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6fc0a3da0 0x7fc6fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.411+0000 7fc712000640 1 -- 172.21.15.44:0/712060144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc7000d0ab0 con 0x7fc7141094e0 2024-04-20T11:27:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.411+0000 7fc719c00640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6fc0a3da0 0x7fc6fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.411+0000 7fc719c00640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6fc0a3da0 0x7fc6fc0a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc71419bb30 tx=0x7fc708005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.415+0000 7fc706a00640 1 -- 172.21.15.44:0/1878865660 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004d20 con 0x5596cfbc41a0 2024-04-20T11:27:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.415+0000 7fc706a00640 1 -- 172.21.15.44:0/1878865660 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc70c004050 con 0x5596cfbc41a0 2024-04-20T11:27:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.415+0000 7fc706a00640 1 -- 172.21.15.44:0/1878865660 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c005260 con 0x5596cfbc41a0 2024-04-20T11:27:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.415+0000 7fc71f370740 1 -- 172.21.15.44:0/1878865660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 msgr2=0x5596cfbc4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.415+0000 7fc71f370740 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbc4580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009f50 tx=0x7fc70c0118d0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 -- 172.21.15.44:0/1878865660 shutdown_connections 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 --2- 172.21.15.44:0/1878865660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbc4580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 -- 172.21.15.44:0/1878865660 >> 172.21.15.44:0/1878865660 conn(0x5596cfbbe8d0 msgr2=0x5596cfbbecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 -- 172.21.15.44:0/1878865660 shutdown_connections 2024-04-20T11:27:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 -- 172.21.15.44:0/1878865660 wait complete. 2024-04-20T11:27:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.416+0000 7fc71f370740 1 -- start start 2024-04-20T11:27:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 Processor -- start 2024-04-20T11:27:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5596cfbd0490 con 0x5596cfbc41a0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5596cfae3740 con 0x5596cfbc4ac0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5596cfbd0610 con 0x5596cfbcd6a0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc71f370740 10 client.0 _ll_register_callbacks cb 0x5596cfadeb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc719c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc719c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57044/0 (socket says 172.21.15.44:57044) 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.417+0000 7fc719c00640 1 -- 172.21.15.44:0/3451987942 learned_addr learned my addr 172.21.15.44:0/3451987942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719200640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc71a600640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719c00640 1 -- 172.21.15.44:0/3451987942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 msgr2=0x5596cfbcb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719c00640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719c00640 1 -- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 msgr2=0x5596cfbcaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719c00640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719c00640 1 -- 172.21.15.44:0/3451987942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cf83c3f0 con 0x5596cfbc41a0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719200640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc719c00640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbca760 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc708035000 tx=0x7fc70802bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401d3f1affc0572b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.418+0000 7fc6fb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc6fb400640 1 -- 172.21.15.44:0/3451987942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708005600 con 0x5596cfbc41a0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc6fb400640 1 -- 172.21.15.44:0/3451987942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc7080046c0 con 0x5596cfbc41a0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c004050 con 0x5596cfbc41a0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc6fb400640 1 -- 172.21.15.44:0/3451987942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7080044d0 con 0x5596cfbc41a0 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc71f370740 10 client.16836 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5596cf7fc0f0 con 0x5596cfbc41a0 2024-04-20T11:27:53.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc6fb400640 1 -- 172.21.15.44:0/3451987942 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc70c004050 con 0x5596cfbc41a0 2024-04-20T11:27:53.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.419+0000 7fc6fb400640 1 -- 172.21.15.44:0/3451987942 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7fc708003ce0 con 0x5596cfbc41a0 2024-04-20T11:27:53.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc71f370740 10 client.16836 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:53.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc71f370740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:53.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc71f370740 20 client.16836 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc71f370740 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfbddaf0 0x5596cfbdff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc71f370740 4 client.16836 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596cfb37b70 con 0x5596cfbddaf0 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc719200640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfbddaf0 0x5596cfbdff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc719200640 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfbddaf0 0x5596cfbdff20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc17bee4c58fcf72 server_cookie=32b4276a583f9b98 in_seq=0 out_seq=0 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.420+0000 7fc6fb400640 10 client.16836 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc6fb400640 1 -- 172.21.15.44:0/3451987942 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596cfb37b70 con 0x5596cfbddaf0 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc6fb400640 10 client.16836 handle_command_reply: tid=0 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc71f370740 1 client.16836 shutdown 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 msgr2=0x5596cfbca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc71f370740 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbca760 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc708035000 tx=0x7fc70802bc50 comp rx=0 tx=0).stop 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 shutdown_connections 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.421+0000 7fc71f370740 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfbddaf0 0x5596cfbdff20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbcb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbcaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc6fb400640 0 client.16836 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 1 --2- 172.21.15.44:0/3451987942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbca760 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 >> 172.21.15.44:0/3451987942 conn(0x5596cfbbe8d0 msgr2=0x5596cfbc0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 shutdown_connections 2024-04-20T11:27:53.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 1 -- 172.21.15.44:0/3451987942 wait complete. 2024-04-20T11:27:53.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.422+0000 7fc71f370740 20 client.16836 trim_cache size 0 max 16384 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 Processor -- start 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 -- start start 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbd9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbc9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbbd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141afee0 con 0x5596cfbcd6a0 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141b1b70 con 0x5596cfbc4ac0 2024-04-20T11:27:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.441+0000 7fc71f370740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141b05b0 con 0x5596cfbc41a0 2024-04-20T11:27:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc719200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbc9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbbd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc719c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbd9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc719200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbc9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39768/0 (socket says 172.21.15.44:39768) 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbbd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57058/0 (socket says 172.21.15.44:57058) 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc719c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbd9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41238/0 (socket says 172.21.15.44:41238) 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc719200640 1 -- 172.21.15.44:0/2481868189 learned_addr learned my addr 172.21.15.44:0/2481868189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 -- 172.21.15.44:0/2481868189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 msgr2=0x5596cfbd9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbd9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 -- 172.21.15.44:0/2481868189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 msgr2=0x5596cfbc9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbc9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.442+0000 7fc71a600640 1 -- 172.21.15.44:0/2481868189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cfbc9710 con 0x5596cfbcd6a0 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.443+0000 7fc71a600640 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbbd740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000f80 tx=0x7fc70c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29078e1139e2714c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.443+0000 7fc706000640 1 -- 172.21.15.44:0/2481868189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c009bf0 con 0x5596cfbcd6a0 2024-04-20T11:27:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.443+0000 7fc706000640 1 -- 172.21.15.44:0/2481868189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc70c0055c0 con 0x5596cfbcd6a0 2024-04-20T11:27:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.443+0000 7fc706000640 1 -- 172.21.15.44:0/2481868189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00fcd0 con 0x5596cfbcd6a0 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 -- 172.21.15.44:0/2481868189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 msgr2=0x5596cfbbd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbbd740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000f80 tx=0x7fc70c003bd0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 -- 172.21.15.44:0/2481868189 shutdown_connections 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbcd6a0 0x5596cfbbd740 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbc9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 --2- 172.21.15.44:0/2481868189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbc41a0 0x5596cfbd9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 -- 172.21.15.44:0/2481868189 >> 172.21.15.44:0/2481868189 conn(0x5596cfbbe8d0 msgr2=0x5596cfbbd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 -- 172.21.15.44:0/2481868189 shutdown_connections 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 -- 172.21.15.44:0/2481868189 wait complete. 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 -- start start 2024-04-20T11:27:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.444+0000 7fc71f370740 1 Processor -- start 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbd5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbd1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbd1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141b13e0 con 0x5596cfbc41a0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141b0ce0 con 0x5596cfbc4ac0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141adeb0 con 0x5596cfbcd6a0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71f370740 10 client.0 _ll_register_callbacks cb 0x7fc708009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc719200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbd1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc719c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbd5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc719200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbd1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:39772/0 (socket says 172.21.15.44:39772) 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc719200640 1 -- 172.21.15.44:0/4177505972 learned_addr learned my addr 172.21.15.44:0/4177505972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.445+0000 7fc71a600640 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbd1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc719200640 1 -- 172.21.15.44:0/4177505972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 msgr2=0x5596cfbd1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc719200640 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbd1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc719200640 1 -- 172.21.15.44:0/4177505972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 msgr2=0x5596cfbd5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc719200640 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbd5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc719200640 1 -- 172.21.15.44:0/4177505972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cfbd1e70 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc719200640 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbd1060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc70009c760 tx=0x7fc700087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe407d5e7f03c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc6fb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc6fb400640 1 -- 172.21.15.44:0/4177505972 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000ce60 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc6fb400640 1 -- 172.21.15.44:0/4177505972 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fc700004830 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c0055c0 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc6fb400640 1 -- 172.21.15.44:0/4177505972 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7000bd0b0 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc71f370740 10 client.26972 fetch_fsmap learned FSMap version 43 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5596cfc44180 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.446+0000 7fc6fb400640 1 -- 172.21.15.44:0/4177505972 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc70c0055c0 con 0x5596cfbc4ac0 2024-04-20T11:27:53.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc6fb400640 1 -- 172.21.15.44:0/4177505972 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) ==== 8407+0+0 (secure 0 0 0) 0x7fc7000ce490 con 0x5596cfbc4ac0 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc71f370740 10 client.26972 fetch_fsmap finished waiting for FSMap version 43 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc71f370740 10 client.26972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc71f370740 20 client.26972 populate_metadata read hostname 'smithi044' 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc71f370740 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfc449c0 0x5596cfc46de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc71f370740 4 client.26972 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596cfb319e0 con 0x5596cfc449c0 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.447+0000 7fc719c00640 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfc449c0 0x5596cfc46de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc719c00640 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfc449c0 0x5596cfc46de0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38524d2f654a5b4d server_cookie=7c40c3ad0edbdeb4 in_seq=0 out_seq=0 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc6fb400640 10 client.26972 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc6fb400640 1 -- 172.21.15.44:0/4177505972 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5596cfb319e0 con 0x5596cfc449c0 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc6fb400640 10 client.26972 handle_command_reply: tid=0 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 client.26972 shutdown 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 msgr2=0x5596cfbd1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbd1060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc70009c760 tx=0x7fc700087040 comp rx=0 tx=0).stop 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 shutdown_connections 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5596cfc449c0 0x5596cfc46de0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596cfbcd6a0 0x5596cfbd1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596cfbc4ac0 0x5596cfbd1060 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 --2- 172.21.15.44:0/4177505972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5596cfbc41a0 0x5596cfbd5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc6fb400640 0 client.26972 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 >> 172.21.15.44:0/4177505972 conn(0x5596cfbbe8d0 msgr2=0x5596cfbcaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 shutdown_connections 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 1 -- 172.21.15.44:0/4177505972 wait complete. 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.448+0000 7fc71f370740 20 client.26972 trim_cache size 0 max 16384 2024-04-20T11:27:53.453 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: "cf163c3d-a2fc-4c84-a3e3-9c64bbbbddae": { 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "cf163c3d-a2fc-4c84-a3e3-9c64bbbbddae", 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:27:53.454 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6fc0a3da0 msgr2=0x7fc6fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6fc0a3da0 0x7fc6fc0a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc71419bb30 tx=0x7fc708005fd0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 msgr2=0x7fc7141a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 0x7fc7141a0d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e9f0 tx=0x7fc70000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 shutdown_connections 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc71419a9f0 0x7fc71419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc6fc0a3da0 0x7fc6fc0a6260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7141094e0 0x7fc7141a0d80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 --2- 172.21.15.44:0/712060144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc714104410 0x7fc7141a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:27:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 >> 172.21.15.44:0/712060144 conn(0x7fc7140ffcf0 msgr2=0x7fc714101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 shutdown_connections 2024-04-20T11:27:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:27:53.449+0000 7fc71c600640 1 -- 172.21.15.44:0/712060144 wait complete. 2024-04-20T11:27:53.467 INFO:tasks.cephfs.filesystem:scrub status for tag:cf163c3d-a2fc-4c84-a3e3-9c64bbbbddae - {'path': '/', 'tag': 'cf163c3d-a2fc-4c84-a3e3-9c64bbbbddae', 'options': 'recursive,force'} 2024-04-20T11:27:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:53 smithi078 ceph-mon[21464]: pgmap v161: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:53 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:27:53.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:53 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:27:53.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:53 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:27:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:53 smithi044 ceph-mon[16855]: pgmap v161: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:53 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:27:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:53 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:27:53.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:53 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:27:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:53 smithi165 ceph-mon[21975]: pgmap v161: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:27:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:53 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:27:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:53 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:27:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:53 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:27:54.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:54 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:27:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:54 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:54 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:27:54.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:54 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:54.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:54 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:27:54.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:54 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:54.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:54 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:27:54.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:54 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:54 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:27:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:54 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:54 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:27:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:54 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:27:55.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:55 smithi078 ceph-mon[21464]: pgmap v162: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:55.699 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:55 smithi044 ceph-mon[16855]: pgmap v162: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:55 smithi165 ceph-mon[21975]: pgmap v162: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:57.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:57 smithi044 ceph-mon[16855]: pgmap v163: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:57.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:57 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:57 smithi165 ceph-mon[21975]: pgmap v163: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:57 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:57 smithi078 ceph-mon[21464]: pgmap v163: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:27:57.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:57 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:27:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:58 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:27:58.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:58 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:27:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:58 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:27:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:27:59 smithi165 ceph-mon[21975]: pgmap v164: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:59.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:27:59 smithi078 ceph-mon[21464]: pgmap v164: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:27:59.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:27:59 smithi044 ceph-mon[16855]: pgmap v164: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:01 smithi165 ceph-mon[21975]: pgmap v165: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:01.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:01 smithi078 ceph-mon[21464]: pgmap v165: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:01 smithi044 ceph-mon[16855]: pgmap v165: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:03 smithi165 ceph-mon[21975]: pgmap v166: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:03 smithi078 ceph-mon[21464]: pgmap v166: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:03.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:03 smithi044 ceph-mon[16855]: pgmap v166: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:05 smithi165 ceph-mon[21975]: pgmap v167: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:05.730 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:05 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:05.748 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:05 smithi044 ceph-mon[16855]: pgmap v167: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:05.748 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:05 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:05 smithi078 ceph-mon[21464]: pgmap v167: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:05 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:07.740 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:07 smithi078 ceph-mon[21464]: pgmap v168: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:07.740 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:07 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:07 smithi044 ceph-mon[16855]: pgmap v168: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:07.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:07 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:07 smithi165 ceph-mon[21975]: pgmap v168: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:07 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:09.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:09 smithi078 ceph-mon[21464]: pgmap v169: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:09.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:09 smithi044 ceph-mon[16855]: pgmap v169: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:09.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:09 smithi165 ceph-mon[21975]: pgmap v169: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:11 smithi078 ceph-mon[21464]: pgmap v170: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:11 smithi044 ceph-mon[16855]: pgmap v170: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:11 smithi165 ceph-mon[21975]: pgmap v170: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-20T11:28:12.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:12 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:12.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:12 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:12.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:12 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:13 smithi078 ceph-mon[21464]: pgmap v171: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:13.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:13 smithi044 ceph-mon[16855]: pgmap v171: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:13 smithi165 ceph-mon[21975]: pgmap v171: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-20T11:28:15.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:15 smithi078 ceph-mon[21464]: pgmap v172: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:28:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:15 smithi044 ceph-mon[16855]: pgmap v172: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:28:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:15 smithi165 ceph-mon[21975]: pgmap v172: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:28:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:17 smithi078 ceph-mon[21464]: pgmap v173: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:28:17.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:17 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:17 smithi044 ceph-mon[16855]: pgmap v173: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:28:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:17 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:17 smithi165 ceph-mon[21975]: pgmap v173: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-20T11:28:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:17 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:18.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:18 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:18.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:18 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:18.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:18 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:19 smithi078 ceph-mon[21464]: pgmap v174: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-20T11:28:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:19 smithi044 ceph-mon[16855]: pgmap v174: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-20T11:28:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:19 smithi165 ceph-mon[21975]: pgmap v174: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-20T11:28:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:21 smithi078 ceph-mon[21464]: pgmap v175: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:28:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:21 smithi044 ceph-mon[16855]: pgmap v175: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:28:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:21 smithi165 ceph-mon[21975]: pgmap v175: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:28:23.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:28:23.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 -- 172.21.15.44:0/1915111676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f551010d3d0 msgr2=0x7f551010d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 --2- 172.21.15.44:0/1915111676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f551010d3d0 0x7f551010d7b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5500009900 tx=0x7f550002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 -- 172.21.15.44:0/1915111676 shutdown_connections 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 --2- 172.21.15.44:0/1915111676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108ea0 0x7f55101152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 --2- 172.21.15.44:0/1915111676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5510108500 0x7f5510108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 --2- 172.21.15.44:0/1915111676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f551010d3d0 0x7f551010d7b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 -- 172.21.15.44:0/1915111676 >> 172.21.15.44:0/1915111676 conn(0x7f5510077d80 msgr2=0x7f5510106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 -- 172.21.15.44:0/1915111676 shutdown_connections 2024-04-20T11:28:23.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.696+0000 7f550f200640 1 -- 172.21.15.44:0/1915111676 wait complete. 2024-04-20T11:28:23.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.697+0000 7f550f200640 1 Processor -- start 2024-04-20T11:28:23.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.697+0000 7f550f200640 1 -- start start 2024-04-20T11:28:23.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108500 0x7f551019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 0x7f551019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f551010d3d0 0x7f55101a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5510118990 con 0x7f5510108ea0 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5510118810 con 0x7f5510108500 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5510118b10 con 0x7f551010d3d0 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 0x7f551019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 0x7f551019f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60972/0 (socket says 172.21.15.44:60972) 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108500 0x7f551019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f5507e00640 1 -- 172.21.15.44:0/869678427 learned_addr learned my addr 172.21.15.44:0/869678427 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.698+0000 7f550e800640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f551010d3d0 0x7f55101a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550de00640 1 -- 172.21.15.44:0/869678427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f551010d3d0 msgr2=0x7f55101a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550de00640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f551010d3d0 0x7f55101a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550de00640 1 -- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 msgr2=0x7f551019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550de00640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 0x7f551019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550de00640 1 -- 172.21.15.44:0/869678427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55101a3e70 con 0x7f5510108500 2024-04-20T11:28:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f5507e00640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 0x7f551019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550de00640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108500 0x7f551019eed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f550002c210 tx=0x7f5500004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f5506a00640 1 -- 172.21.15.44:0/869678427 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500043020 con 0x7f5510108500 2024-04-20T11:28:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55101a4100 con 0x7f5510108500 2024-04-20T11:28:23.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f5506a00640 1 -- 172.21.15.44:0/869678427 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5500004250 con 0x7f5510108500 2024-04-20T11:28:23.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f5506a00640 1 -- 172.21.15.44:0/869678427 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500003ab0 con 0x7f5510108500 2024-04-20T11:28:23.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.699+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55101ab9f0 con 0x7f5510108500 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 Processor -- start 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 -- start start 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5506a00640 1 -- 172.21.15.44:0/869678427 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5500046020 con 0x7f5510108500 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4530760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4530ca0 0x55cac4539340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 0x55cac453bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cac4496150 con 0x55cac4539880 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cac443c370 con 0x55cac4530ca0 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5517e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cac4492110 con 0x55cac4530380 2024-04-20T11:28:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.703+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 0x55cac453bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 0x55cac453bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60986/0 (socket says 172.21.15.44:60986) 2024-04-20T11:28:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 -- 172.21.15.44:0/3178639430 learned_addr learned my addr 172.21.15.44:0/3178639430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f550e800640 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4530760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f550de00640 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4530ca0 0x55cac4539340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 -- 172.21.15.44:0/3178639430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 msgr2=0x55cac4530760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4530760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 -- 172.21.15.44:0/3178639430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4530ca0 msgr2=0x55cac4539340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4530ca0 0x55cac4539340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 -- 172.21.15.44:0/3178639430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac42727d0 con 0x55cac4539880 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5506a00640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f54f40a3d20 0x7f54f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.704+0000 7f5507e00640 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 0x55cac453bc70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f54f80115f0 tx=0x7f54f8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f8cd20c2e6f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f5506a00640 1 -- 172.21.15.44:0/869678427 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f55000f24c0 con 0x7f5510108500 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f54ff400640 1 -- 172.21.15.44:0/3178639430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f801e070 con 0x55cac4539880 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f550e800640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f54f40a3d20 0x7f54f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f54ff400640 1 -- 172.21.15.44:0/3178639430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54f800fbc0 con 0x55cac4539880 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f54ff400640 1 -- 172.21.15.44:0/3178639430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8019d90 con 0x55cac4539880 2024-04-20T11:28:23.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f5517e11740 1 -- 172.21.15.44:0/3178639430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 msgr2=0x55cac453bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f5517e11740 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 0x55cac453bc70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f54f80115f0 tx=0x7f54f8011ab0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f550e800640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f54f40a3d20 0x7f54f40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5508005fd0 tx=0x7f5508009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f5517e11740 1 -- 172.21.15.44:0/3178639430 shutdown_connections 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f5517e11740 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4539880 0x55cac453bc70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.705+0000 7f5517e11740 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4530ca0 0x55cac4539340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.706+0000 7f5517e11740 1 --2- 172.21.15.44:0/3178639430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4530760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.706+0000 7f5517e11740 1 -- 172.21.15.44:0/3178639430 >> 172.21.15.44:0/3178639430 conn(0x55cac452aab0 msgr2=0x55cac452ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.706+0000 7f5517e11740 1 -- 172.21.15.44:0/3178639430 shutdown_connections 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.706+0000 7f5517e11740 1 -- 172.21.15.44:0/3178639430 wait complete. 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.706+0000 7f5517e11740 1 -- start start 2024-04-20T11:28:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.706+0000 7f5517e11740 1 Processor -- start 2024-04-20T11:28:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac45368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4536e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4537370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cac453c670 con 0x55cac4530ca0 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cac444f740 con 0x55cac4539880 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cac453c7f0 con 0x55cac4530380 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5517e11740 10 client.0 _ll_register_callbacks cb 0x55cac444ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4536e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f550e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4537370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4536e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60996/0 (socket says 172.21.15.44:60996) 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f5507e00640 1 -- 172.21.15.44:0/243427661 learned_addr learned my addr 172.21.15.44:0/243427661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.707+0000 7f550de00640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac45368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5507e00640 1 -- 172.21.15.44:0/243427661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 msgr2=0x55cac45368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5507e00640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac45368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5507e00640 1 -- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 msgr2=0x55cac4537370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5507e00640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4537370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5507e00640 1 -- 172.21.15.44:0/243427661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac41a83f0 con 0x55cac4530ca0 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f550e800640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4537370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5507e00640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4536e30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f54f800ff80 tx=0x7f54f8009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ac68a56764a1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f54f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f54f3e00640 1 -- 172.21.15.44:0/243427661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f801e040 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f54f3e00640 1 -- 172.21.15.44:0/243427661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f54f8019410 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f800fbc0 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.708+0000 7f54f3e00640 1 -- 172.21.15.44:0/243427661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f54f8009da0 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 10 client.16854 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55cac41680f0 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f54f3e00640 1 -- 172.21.15.44:0/243427661 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54f800fbc0 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f54f3e00640 1 -- 172.21.15.44:0/243427661 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f54f8004920 con 0x55cac4530ca0 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 10 client.16854 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 20 client.16854 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac4549b40 0x55cac454bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 4 client.16854 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.709+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac44a3b70 con 0x55cac4549b40 2024-04-20T11:28:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.710+0000 7f550de00640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac4549b40 0x55cac454bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.710+0000 7f550de00640 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac4549b40 0x55cac454bf30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a8531b3d31fcc server_cookie=26b728e7eae13613 in_seq=0 out_seq=0 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.710+0000 7f54f3e00640 10 client.16854 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f54f3e00640 1 -- 172.21.15.44:0/243427661 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cac44a3b70 con 0x55cac4549b40 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f54f3e00640 10 client.16854 handle_command_reply: tid=0 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 client.16854 shutdown 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 msgr2=0x55cac4536e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4536e30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f54f800ff80 tx=0x7f54f8009530 comp rx=0 tx=0).stop 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 shutdown_connections 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac4549b40 0x55cac454bf30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4537370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4536e30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 --2- 172.21.15.44:0/243427661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac45368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 >> 172.21.15.44:0/243427661 conn(0x55cac452aab0 msgr2=0x55cac452c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.711+0000 7f54f3e00640 0 client.16854 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.712+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 shutdown_connections 2024-04-20T11:28:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.712+0000 7f5517e11740 1 -- 172.21.15.44:0/243427661 wait complete. 2024-04-20T11:28:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.712+0000 7f5517e11740 20 client.16854 trim_cache size 0 max 16384 2024-04-20T11:28:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.731+0000 7f5517e11740 1 Processor -- start 2024-04-20T11:28:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.731+0000 7f5517e11740 1 -- start start 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.731+0000 7f5517e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4539020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.731+0000 7f5517e11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4535830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5517e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4545530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5517e11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55101b3ff0 con 0x55cac4530ca0 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5517e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55101b5bc0 con 0x55cac4539880 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5517e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55101b46c0 con 0x55cac4530380 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4535830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4535830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32770/0 (socket says 172.21.15.44:32770) 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 -- 172.21.15.44:0/4100469235 learned_addr learned my addr 172.21.15.44:0/4100469235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f550e800640 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4545530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f550de00640 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4539020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 -- 172.21.15.44:0/4100469235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 msgr2=0x55cac4539020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4539020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 -- 172.21.15.44:0/4100469235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 msgr2=0x55cac4545530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4545530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f5507e00640 1 -- 172.21.15.44:0/4100469235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac454c4c0 con 0x55cac4530ca0 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.732+0000 7f550e800640 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4545530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.733+0000 7f5507e00640 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4535830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005db0 tx=0x7f54f8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323c09b63e53e915 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.733+0000 7f54fea00640 1 -- 172.21.15.44:0/4100469235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8019410 con 0x55cac4530ca0 2024-04-20T11:28:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.733+0000 7f54fea00640 1 -- 172.21.15.44:0/4100469235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54f8009c30 con 0x55cac4530ca0 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.733+0000 7f54fea00640 1 -- 172.21.15.44:0/4100469235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8019d20 con 0x55cac4530ca0 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 -- 172.21.15.44:0/4100469235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 msgr2=0x55cac4535830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4535830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005db0 tx=0x7f54f8011ab0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 -- 172.21.15.44:0/4100469235 shutdown_connections 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac4545530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac4535830 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 --2- 172.21.15.44:0/4100469235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4539020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 -- 172.21.15.44:0/4100469235 >> 172.21.15.44:0/4100469235 conn(0x55cac452aab0 msgr2=0x55cac4537140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 -- 172.21.15.44:0/4100469235 shutdown_connections 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 -- 172.21.15.44:0/4100469235 wait complete. 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.734+0000 7f5517e11740 1 -- start start 2024-04-20T11:28:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 Processor -- start 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4541b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac453d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac453d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55101b54c0 con 0x55cac4530ca0 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55101b4dc0 con 0x55cac4539880 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55101b1fb0 con 0x55cac4530380 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5517e11740 10 client.0 _ll_register_callbacks cb 0x7f54f801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac453d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.735+0000 7f5507e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac453d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32772/0 (socket says 172.21.15.44:32772) 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 -- 172.21.15.44:0/373144962 learned_addr learned my addr 172.21.15.44:0/373144962 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:23.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f550de00640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4541b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f550e800640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac453d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 -- 172.21.15.44:0/373144962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 msgr2=0x55cac4541b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4541b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 -- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 msgr2=0x55cac453d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac453d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 -- 172.21.15.44:0/373144962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac453dab0 con 0x55cac4530ca0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f5507e00640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac453d000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005e00 tx=0x7f54f800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67aad24061240c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.736+0000 7f54f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f54f3e00640 1 -- 172.21.15.44:0/373144962 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8045060 con 0x55cac4530ca0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f54f3e00640 1 -- 172.21.15.44:0/373144962 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f54f8011040 con 0x55cac4530ca0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f8009c30 con 0x55cac4530ca0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f54f3e00640 1 -- 172.21.15.44:0/373144962 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f54f8019880 con 0x55cac4530ca0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f5517e11740 10 client.16860 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55cac45afcf0 con 0x55cac4530ca0 2024-04-20T11:28:23.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f54f3e00640 1 -- 172.21.15.44:0/373144962 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54f8009c30 con 0x55cac4530ca0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.737+0000 7f54f3e00640 1 -- 172.21.15.44:0/373144962 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f54f8048020 con 0x55cac4530ca0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f5517e11740 10 client.16860 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f5517e11740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f5517e11740 20 client.16860 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f5517e11740 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac45b03e0 0x55cac45b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f5517e11740 4 client.16860 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cac449d9e0 con 0x55cac45b03e0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f550de00640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac45b03e0 0x55cac45b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f550de00640 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac45b03e0 0x55cac45b2800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10a27427f09b739 server_cookie=c79b06d887542151 in_seq=0 out_seq=0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.738+0000 7f54f3e00640 10 client.16860 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f54f3e00640 1 -- 172.21.15.44:0/373144962 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cac449d9e0 con 0x55cac45b03e0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f54f3e00640 10 client.16860 handle_command_reply: tid=0 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 client.16860 shutdown 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 msgr2=0x55cac453d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac453d000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005e00 tx=0x7f54f800ff80 comp rx=0 tx=0).stop 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 shutdown_connections 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55cac45b03e0 0x55cac45b2800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cac4539880 0x55cac453d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cac4530ca0 0x55cac453d000 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 --2- 172.21.15.44:0/373144962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cac4530380 0x55cac4541b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 >> 172.21.15.44:0/373144962 conn(0x55cac452aab0 msgr2=0x55cac4529280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f54f3e00640 0 client.16860 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 shutdown_connections 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 1 -- 172.21.15.44:0/373144962 wait complete. 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.739+0000 7f5517e11740 20 client.16860 trim_cache size 0 max 16384 2024-04-20T11:28:23.742 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:28:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:28:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:28:23.743 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f54f40a3d20 msgr2=0x7f54f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f54f40a3d20 0x7f54f40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5508005fd0 tx=0x7f5508009500 comp rx=0 tx=0).stop 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108500 msgr2=0x7f551019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108500 0x7f551019eed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f550002c210 tx=0x7f5500004970 comp rx=0 tx=0).stop 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 shutdown_connections 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f551010d3d0 0x7f55101a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f54f40a3d20 0x7f54f40a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5510108ea0 0x7f551019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 --2- 172.21.15.44:0/869678427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5510108500 0x7f551019eed0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 >> 172.21.15.44:0/869678427 conn(0x7f5510077d80 msgr2=0x7f55100783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 shutdown_connections 2024-04-20T11:28:23.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.740+0000 7f550f200640 1 -- 172.21.15.44:0/869678427 wait complete. 2024-04-20T11:28:23.758 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:28:23.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:23.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:23 smithi044 ceph-mon[16855]: pgmap v176: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:28:23.830 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:23 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:23 smithi078 ceph-mon[21464]: pgmap v176: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:28:23.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:23 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 -- 172.21.15.44:0/1254282394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 msgr2=0x7f4afc10eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 --2- 172.21.15.44:0/1254282394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 0x7f4afc10eca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009900 tx=0x7f4af402bc40 comp rx=0 tx=0).stop 2024-04-20T11:28:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 -- 172.21.15.44:0/1254282394 shutdown_connections 2024-04-20T11:28:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 --2- 172.21.15.44:0/1254282394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 0x7f4afc10eca0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 --2- 172.21.15.44:0/1254282394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc069d10 0x7f4afc1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 --2- 172.21.15.44:0/1254282394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.970+0000 7f4b01600640 1 -- 172.21.15.44:0/1254282394 >> 172.21.15.44:0/1254282394 conn(0x7f4afc0767a0 msgr2=0x7f4afc076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.971+0000 7f4b01600640 1 -- 172.21.15.44:0/1254282394 shutdown_connections 2024-04-20T11:28:23.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.971+0000 7f4b01600640 1 -- 172.21.15.44:0/1254282394 wait complete. 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 Processor -- start 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 -- start start 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 0x7f4afc1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc19a7e0 0x7f4afc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1122c0 con 0x7f4afc19a7e0 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc112140 con 0x7f4afc107100 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b01600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc112440 con 0x7f4afc069360 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.972+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc19a7e0 0x7f4afc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44106/0 (socket says 172.21.15.44:44106) 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 -- 172.21.15.44:0/4146183129 learned_addr learned my addr 172.21.15.44:0/4146183129 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc19a7e0 0x7f4afc19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32790/0 (socket says 172.21.15.44:32790) 2024-04-20T11:28:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afb400640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 0x7f4afc1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 -- 172.21.15.44:0/4146183129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 msgr2=0x7f4afc1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 0x7f4afc1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 -- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc19a7e0 msgr2=0x7f4afc19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc19a7e0 0x7f4afc19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 -- 172.21.15.44:0/4146183129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc19b370 con 0x7f4afc069360 2024-04-20T11:28:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.973+0000 7f4afbe00640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc1a0620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00eb90 tx=0x7f4aec00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.974+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec004d20 con 0x7f4afc069360 2024-04-20T11:28:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.974+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc19b660 con 0x7f4afc069360 2024-04-20T11:28:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.974+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4aec00ee90 con 0x7f4afc069360 2024-04-20T11:28:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.974+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0108b0 con 0x7f4afc069360 2024-04-20T11:28:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.974+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc1a77a0 con 0x7f4afc069360 2024-04-20T11:28:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.975+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4afc1034b0 con 0x7f4afc069360 2024-04-20T11:28:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.977+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4aec005020 con 0x7f4afc069360 2024-04-20T11:28:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.979+0000 7f4af8c00640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4ae00a3d20 0x7f4ae00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.979+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f4aec0d0e20 con 0x7f4afc069360 2024-04-20T11:28:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.979+0000 7f4afb400640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4ae00a3d20 0x7f4ae00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.980+0000 7f4afb400640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4ae00a3d20 0x7f4ae00a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40095f0 tx=0x7f4ae4005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:23 smithi165 ceph-mon[21975]: pgmap v176: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:28:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:23 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:23.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:23.983+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4afc19b660 con 0x7f4afc069360 2024-04-20T11:28:24.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.231+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4afc068ca0 con 0x7f4afc069360 2024-04-20T11:28:24.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.232+0000 7f4af8c00640 1 -- 172.21.15.44:0/4146183129 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11946 (secure 0 0 0) 0x7f4aec09d4d0 con 0x7f4afc069360 2024-04-20T11:28:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:28:24.232 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:28:22.423521+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:28:24.232 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 50 2024-04-20T11:28:24.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4ae00a3d20 msgr2=0x7f4ae00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4ae00a3d20 0x7f4ae00a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40095f0 tx=0x7f4ae4005f90 comp rx=0 tx=0).stop 2024-04-20T11:28:24.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 msgr2=0x7f4afc1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc1a0620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00eb90 tx=0x7f4aec00c6a0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 shutdown_connections 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4ae00a3d20 0x7f4ae00a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4afc19a7e0 0x7f4afc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.234+0000 7f4b01600640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4afc107100 0x7f4afc1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.235+0000 7f4b01600640 1 --2- 172.21.15.44:0/4146183129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4afc069360 0x7f4afc1a0620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.235+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 >> 172.21.15.44:0/4146183129 conn(0x7f4afc0767a0 msgr2=0x7f4afc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.235+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 shutdown_connections 2024-04-20T11:28:24.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.235+0000 7f4b01600640 1 -- 172.21.15.44:0/4146183129 wait complete. 2024-04-20T11:28:24.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:28:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 -- 172.21.15.44:0/4142081292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 msgr2=0x7f43d0108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 --2- 172.21.15.44:0/4142081292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d0108960 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc02bcd0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 -- 172.21.15.44:0/4142081292 shutdown_connections 2024-04-20T11:28:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 --2- 172.21.15.44:0/4142081292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d0108ea0 0x7f43d01152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 --2- 172.21.15.44:0/4142081292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d0108960 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 --2- 172.21.15.44:0/4142081292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010d3d0 0x7f43d010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 -- 172.21.15.44:0/4142081292 >> 172.21.15.44:0/4142081292 conn(0x7f43d0077d80 msgr2=0x7f43d0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 -- 172.21.15.44:0/4142081292 shutdown_connections 2024-04-20T11:28:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.474+0000 7f43d6400640 1 -- 172.21.15.44:0/4142081292 wait complete. 2024-04-20T11:28:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.475+0000 7f43d6400640 1 Processor -- start 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 -- start start 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d0108ea0 0x7f43d019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d010d3d0 0x7f43d01a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0118940 con 0x7f43d0108500 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01187c0 con 0x7f43d010d3d0 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d6400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0118ac0 con 0x7f43d0108ea0 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d0108ea0 0x7f43d019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32810/0 (socket says 172.21.15.44:32810) 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.476+0000 7f43d5000640 1 -- 172.21.15.44:0/380659123 learned_addr learned my addr 172.21.15.44:0/380659123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5a00640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d010d3d0 0x7f43d01a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5000640 1 -- 172.21.15.44:0/380659123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d0108ea0 msgr2=0x7f43d019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5000640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d0108ea0 0x7f43d019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5000640 1 -- 172.21.15.44:0/380659123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d010d3d0 msgr2=0x7f43d01a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5000640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d010d3d0 0x7f43d01a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5000640 1 -- 172.21.15.44:0/380659123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d01a3e40 con 0x7f43d0108500 2024-04-20T11:28:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5a00640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d010d3d0 0x7f43d01a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.477+0000 7f43d5000640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d019eea0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9f0 tx=0x7f43c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.478+0000 7f43cd600640 1 -- 172.21.15.44:0/380659123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400cd80 con 0x7f43d0108500 2024-04-20T11:28:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.478+0000 7f43cd600640 1 -- 172.21.15.44:0/380659123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43c4010930 con 0x7f43d0108500 2024-04-20T11:28:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.478+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d01a4130 con 0x7f43d0108500 2024-04-20T11:28:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.478+0000 7f43cd600640 1 -- 172.21.15.44:0/380659123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4018b00 con 0x7f43d0108500 2024-04-20T11:28:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.478+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01ab9c0 con 0x7f43d0108500 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 Processor -- start 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 -- start start 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43cd600640 1 -- 172.21.15.44:0/380659123 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f43c4018ca0 con 0x7f43d0108500 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 0x55a19430c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194315130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a194315670 0x55a194317a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a194271050 con 0x55a19430ca90 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a194217490 con 0x55a194315670 2024-04-20T11:28:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.481+0000 7f43da799740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a19426d010 con 0x55a19430c170 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 0x55a19430c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 0x55a19430c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44130/0 (socket says 172.21.15.44:44130) 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194315130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 -- 172.21.15.44:0/3649824876 learned_addr learned my addr 172.21.15.44:0/3649824876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43cfe00640 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a194315670 0x55a194317a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 -- 172.21.15.44:0/3649824876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a194315670 msgr2=0x55a194317a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a194315670 0x55a194317a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 -- 172.21.15.44:0/3649824876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 msgr2=0x55a194315130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194315130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.482+0000 7f43d5a00640 1 -- 172.21.15.44:0/3649824876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19421f070 con 0x55a19430c170 2024-04-20T11:28:24.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43d5a00640 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 0x55a19430c550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f43c80113e0 tx=0x7f43c80118a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdffecd5e22fce89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43c2000640 1 -- 172.21.15.44:0/3649824876 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8012770 con 0x55a19430c170 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43c2000640 1 -- 172.21.15.44:0/3649824876 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43c8012e10 con 0x55a19430c170 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43c2000640 1 -- 172.21.15.44:0/3649824876 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c801a3f0 con 0x55a19430c170 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43cd600640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f43b80a3d20 0x7f43b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43cd600640 1 -- 172.21.15.44:0/380659123 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f43c40d0bb0 con 0x7f43d0108500 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.483+0000 7f43d5000640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f43b80a3d20 0x7f43b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 -- 172.21.15.44:0/3649824876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 msgr2=0x55a19430c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 0x55a19430c550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f43c80113e0 tx=0x7f43c80118a0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 -- 172.21.15.44:0/3649824876 shutdown_connections 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a194315670 0x55a194317a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194315130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 --2- 172.21.15.44:0/3649824876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430c170 0x55a19430c550 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 -- 172.21.15.44:0/3649824876 >> 172.21.15.44:0/3649824876 conn(0x55a1943068a0 msgr2=0x55a194306c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 -- 172.21.15.44:0/3649824876 shutdown_connections 2024-04-20T11:28:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43da799740 1 -- 172.21.15.44:0/3649824876 wait complete. 2024-04-20T11:28:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.484+0000 7f43d5000640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f43b80a3d20 0x7f43b80a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55a19430bc40 tx=0x7f43c40d4660 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 -- start start 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 Processor -- start 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194312640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 0x55a194312b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a194315670 0x55a1943130c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a194318460 con 0x55a194315670 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a19422a860 con 0x55a19430c170 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a1943185e0 con 0x55a19430ca90 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.485+0000 7f43da799740 10 client.0 _ll_register_callbacks cb 0x55a1941f8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:24.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 0x55a194312b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43d5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a194315670 0x55a1943130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 0x55a194312b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44132/0 (socket says 172.21.15.44:44132) 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 -- 172.21.15.44:0/158416975 learned_addr learned my addr 172.21.15.44:0/158416975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43d5a00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194312640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 -- 172.21.15.44:0/158416975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 msgr2=0x55a194312640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194312640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 -- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a194315670 msgr2=0x55a1943130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a194315670 0x55a1943130c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43cfe00640 1 -- 172.21.15.44:0/158416975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19422e950 con 0x55a19430ca90 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.486+0000 7f43d5a00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a194315670 0x55a1943130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43cfe00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 0x55a194312b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc02df30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e22f2153813036 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43b6a00640 1 -- 172.21.15.44:0/158416975 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc004250 con 0x55a19430ca90 2024-04-20T11:28:24.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c8012e10 con 0x55a19430ca90 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43b6a00640 1 -- 172.21.15.44:0/158416975 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f43bc0043f0 con 0x55a19430ca90 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43b6a00640 1 -- 172.21.15.44:0/158416975 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f43bc050050 con 0x55a19430ca90 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43da799740 10 client.27007 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.487+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a1941fc820 con 0x55a19430ca90 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43b6a00640 1 -- 172.21.15.44:0/158416975 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43c8012e10 con 0x55a19430ca90 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43b6a00640 1 -- 172.21.15.44:0/158416975 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f43bc0035f0 con 0x55a19430ca90 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43da799740 10 client.27007 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:24.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43da799740 10 client.27007 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43da799740 20 client.27007 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43da799740 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a194325690 0x55a194327ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43da799740 4 client.27007 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19427ea70 con 0x55a194325690 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.488+0000 7f43d5a00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a194325690 0x55a194327ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.489+0000 7f43d5a00640 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a194325690 0x55a194327ad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=745c4eb85ef8ea6f server_cookie=18220d81e6c1c7fc in_seq=0 out_seq=0 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.489+0000 7f43b6a00640 10 client.27007 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43b6a00640 1 -- 172.21.15.44:0/158416975 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a19427ea70 con 0x55a194325690 2024-04-20T11:28:24.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43b6a00640 10 client.27007 handle_command_reply: tid=0 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 client.27007 shutdown 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 msgr2=0x55a194312b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 0x55a194312b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc02df30 comp rx=0 tx=0).stop 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 shutdown_connections 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a194315670 0x55a1943130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a19430ca90 0x55a194312b80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a194325690 0x55a194327ad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 --2- 172.21.15.44:0/158416975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194312640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43b6a00640 0 client.27007 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 >> 172.21.15.44:0/158416975 conn(0x55a1943068a0 msgr2=0x55a1943085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 shutdown_connections 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 1 -- 172.21.15.44:0/158416975 wait complete. 2024-04-20T11:28:24.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.490+0000 7f43da799740 20 client.27007 trim_cache size 0 max 16384 2024-04-20T11:28:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.505+0000 7f43da799740 1 Processor -- start 2024-04-20T11:28:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.505+0000 7f43da799740 1 -- start start 2024-04-20T11:28:24.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43da799740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194321350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43da799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194312ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43da799740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a1943057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43da799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01b40a0 con 0x55a19430ca90 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43da799740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01b5c70 con 0x55a19430c170 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43da799740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01b4770 con 0x55a194315670 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194312ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194312ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32834/0 (socket says 172.21.15.44:32834) 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 -- 172.21.15.44:0/3429908151 learned_addr learned my addr 172.21.15.44:0/3429908151 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43d5a00640 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194321350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43d5a00640 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a1943057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 -- 172.21.15.44:0/3429908151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 msgr2=0x55a1943057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a1943057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 -- 172.21.15.44:0/3429908151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 msgr2=0x55a194321350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194321350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43cfe00640 1 -- 172.21.15.44:0/3429908151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a193f833f0 con 0x55a19430ca90 2024-04-20T11:28:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.506+0000 7f43d5a00640 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194321350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.507+0000 7f43cfe00640 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194312ee0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43bc003a10 tx=0x7f43bc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f54d4a697586612 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.507+0000 7f43c1600640 1 -- 172.21.15.44:0/3429908151 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003c00 con 0x55a19430ca90 2024-04-20T11:28:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.507+0000 7f43c1600640 1 -- 172.21.15.44:0/3429908151 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43bc003da0 con 0x55a19430ca90 2024-04-20T11:28:24.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.507+0000 7f43c1600640 1 -- 172.21.15.44:0/3429908151 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc005610 con 0x55a19430ca90 2024-04-20T11:28:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 -- 172.21.15.44:0/3429908151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 msgr2=0x55a194312ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194312ee0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43bc003a10 tx=0x7f43bc001e70 comp rx=0 tx=0).stop 2024-04-20T11:28:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 -- 172.21.15.44:0/3429908151 shutdown_connections 2024-04-20T11:28:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a1943057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194312ee0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 --2- 172.21.15.44:0/3429908151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a194321350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 -- 172.21.15.44:0/3429908151 >> 172.21.15.44:0/3429908151 conn(0x55a1943068a0 msgr2=0x55a1943050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 -- 172.21.15.44:0/3429908151 shutdown_connections 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 -- 172.21.15.44:0/3429908151 wait complete. 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 -- start start 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.508+0000 7f43da799740 1 Processor -- start 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a19431dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194319010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a194319570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01b5570 con 0x55a19430ca90 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01b4e70 con 0x55a19430c170 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01b2060 con 0x55a194315670 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43da799740 10 client.0 _ll_register_callbacks cb 0x7f43bc02dbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194319010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43d5a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a19431dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194319010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32846/0 (socket says 172.21.15.44:32846) 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.509+0000 7f43cfe00640 1 -- 172.21.15.44:0/2580514804 learned_addr learned my addr 172.21.15.44:0/2580514804 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43d5a00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a194319570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43cfe00640 1 -- 172.21.15.44:0/2580514804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 msgr2=0x55a194319570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43cfe00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a194319570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43cfe00640 1 -- 172.21.15.44:0/2580514804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 msgr2=0x55a19431dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43cfe00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a19431dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43cfe00640 1 -- 172.21.15.44:0/2580514804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19404d7d0 con 0x55a19430ca90 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43d5a00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a19431dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43cfe00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194319010 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f43bc04d040 tx=0x7f43bc02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b492d12153acf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43b6a00640 1 -- 172.21.15.44:0/2580514804 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc02ca70 con 0x55a19430ca90 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43b6a00640 1 -- 172.21.15.44:0/2580514804 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f43bc02cc10 con 0x55a19430ca90 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.510+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1942788e0 con 0x55a19430ca90 2024-04-20T11:28:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43b6a00640 1 -- 172.21.15.44:0/2580514804 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f43bc02ce10 con 0x55a19430ca90 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 10 client.16884 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a19438bac0 con 0x55a19430ca90 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43b6a00640 1 -- 172.21.15.44:0/2580514804 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55a1942788e0 con 0x55a19430ca90 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43b6a00640 1 -- 172.21.15.44:0/2580514804 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f43bc067050 con 0x55a19430ca90 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 10 client.16884 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 10 client.16884 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 20 client.16884 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a19438c130 0x55a19438e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 4 client.16884 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.511+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1941f45a0 con 0x55a19438c130 2024-04-20T11:28:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.512+0000 7f43d5a00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a19438c130 0x55a19438e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.512+0000 7f43b6a00640 10 client.16884 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.512+0000 7f43d5a00640 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a19438c130 0x55a19438e550 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69052c8481485fc5 server_cookie=8076cd026d4197e4 in_seq=0 out_seq=0 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.512+0000 7f43b6a00640 1 -- 172.21.15.44:0/2580514804 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1941f45a0 con 0x55a19438c130 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.512+0000 7f43b6a00640 10 client.16884 handle_command_reply: tid=0 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.512+0000 7f43da799740 1 client.16884 shutdown 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 msgr2=0x55a194319010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194319010 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f43bc04d040 tx=0x7f43bc02bcd0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 shutdown_connections 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a194315670 0x55a194319570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a19430ca90 0x55a194319010 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55a19438c130 0x55a19438e550 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 --2- 172.21.15.44:0/2580514804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19430c170 0x55a19431dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 >> 172.21.15.44:0/2580514804 conn(0x55a1943068a0 msgr2=0x55a194308480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43b6a00640 0 client.16884 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 shutdown_connections 2024-04-20T11:28:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 1 -- 172.21.15.44:0/2580514804 wait complete. 2024-04-20T11:28:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.513+0000 7f43da799740 20 client.16884 trim_cache size 0 max 16384 2024-04-20T11:28:24.517 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f43b80a3d20 msgr2=0x7f43b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f43b80a3d20 0x7f43b80a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55a19430bc40 tx=0x7f43c40d4660 comp rx=0 tx=0).stop 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 msgr2=0x7f43d019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d019eea0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9f0 tx=0x7f43c400eeb0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 shutdown_connections 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43d010d3d0 0x7f43d01a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d0108ea0 0x7f43d019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f43b80a3d20 0x7f43b80a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 --2- 172.21.15.44:0/380659123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0108500 0x7f43d019eea0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 >> 172.21.15.44:0/380659123 conn(0x7f43d0077d80 msgr2=0x7f43d00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 shutdown_connections 2024-04-20T11:28:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.514+0000 7f43d6400640 1 -- 172.21.15.44:0/380659123 wait complete. 2024-04-20T11:28:24.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:28:24.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 -- 172.21.15.44:0/1466486234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c100bc0 msgr2=0x7f5b5c100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 --2- 172.21.15.44:0/1466486234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c100bc0 0x7f5b5c100fa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009860 tx=0x7f5b4c02bb70 comp rx=0 tx=0).stop 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 -- 172.21.15.44:0/1466486234 shutdown_connections 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 --2- 172.21.15.44:0/1466486234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c10e770 0x7f5b5c110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 --2- 172.21.15.44:0/1466486234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b5c101570 0x7f5b5c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 --2- 172.21.15.44:0/1466486234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c100bc0 0x7f5b5c100fa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.752+0000 7f5b61e00640 1 -- 172.21.15.44:0/1466486234 >> 172.21.15.44:0/1466486234 conn(0x7f5b5c0fc7e0 msgr2=0x7f5b5c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.753+0000 7f5b61e00640 1 -- 172.21.15.44:0/1466486234 shutdown_connections 2024-04-20T11:28:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.753+0000 7f5b61e00640 1 -- 172.21.15.44:0/1466486234 wait complete. 2024-04-20T11:28:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 Processor -- start 2024-04-20T11:28:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 -- start start 2024-04-20T11:28:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b5c100bc0 0x7f5b5c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 0x7f5b5c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c10e770 0x7f5b5c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c114530 con 0x7f5b5c101570 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1143b0 con 0x7f5b5c100bc0 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b61e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1146b0 con 0x7f5b5c10e770 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 0x7f5b5c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.754+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 0x7f5b5c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32876/0 (socket says 172.21.15.44:32876) 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 -- 172.21.15.44:0/3822730642 learned_addr learned my addr 172.21.15.44:0/3822730642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b61400640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c10e770 0x7f5b5c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b60a00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b5c100bc0 0x7f5b5c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 -- 172.21.15.44:0/3822730642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c10e770 msgr2=0x7f5b5c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c10e770 0x7f5b5c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 -- 172.21.15.44:0/3822730642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b5c100bc0 msgr2=0x7f5b5c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b5c100bc0 0x7f5b5c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 -- 172.21.15.44:0/3822730642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c19b000 con 0x7f5b5c101570 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b61400640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c10e770 0x7f5b5c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.755+0000 7f5b5be00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 0x7f5b5c1a0c00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9c0 tx=0x7f5b4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.756+0000 7f5b59600640 1 -- 172.21.15.44:0/3822730642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800cde0 con 0x7f5b5c101570 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.756+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c19b2f0 con 0x7f5b5c101570 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.756+0000 7f5b59600640 1 -- 172.21.15.44:0/3822730642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b480035f0 con 0x7f5b5c101570 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.756+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c1a7510 con 0x7f5b5c101570 2024-04-20T11:28:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.756+0000 7f5b59600640 1 -- 172.21.15.44:0/3822730642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48018630 con 0x7f5b5c101570 2024-04-20T11:28:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.760+0000 7f5b6621f740 1 Processor -- start 2024-04-20T11:28:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.760+0000 7f5b6621f740 1 -- start start 2024-04-20T11:28:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198170 0x55d025198550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d0251a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b6621f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d0250fd050 con 0x55d0251a1670 2024-04-20T11:28:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b6621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0250a3490 con 0x55d025198a90 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b6621f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0250f9010 con 0x55d025198170 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b59600640 1 -- 172.21.15.44:0/3822730642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5b480037a0 con 0x7f5b5c101570 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b60a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b60a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59240/0 (socket says 172.21.15.44:59240) 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b60a00640 1 -- 172.21.15.44:0/379333627 learned_addr learned my addr 172.21.15.44:0/379333627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.761+0000 7f5b61400640 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198170 0x55d025198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b5be00640 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d0251a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b60a00640 1 -- 172.21.15.44:0/379333627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198170 msgr2=0x55d025198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b60a00640 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198170 0x55d025198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b60a00640 1 -- 172.21.15.44:0/379333627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 msgr2=0x55d0251a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b60a00640 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d0251a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b60a00640 1 -- 172.21.15.44:0/379333627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0250ab070 con 0x55d025198a90 2024-04-20T11:28:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b61400640 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198170 0x55d025198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.762+0000 7f5b60a00640 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251a1130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009780 tx=0x7f5b4c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ce069fa4f233b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.763+0000 7f5b51c00640 1 -- 172.21.15.44:0/379333627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c043020 con 0x55d025198a90 2024-04-20T11:28:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.763+0000 7f5b51c00640 1 -- 172.21.15.44:0/379333627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b4c034d50 con 0x55d025198a90 2024-04-20T11:28:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.763+0000 7f5b51c00640 1 -- 172.21.15.44:0/379333627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c039040 con 0x55d025198a90 2024-04-20T11:28:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.763+0000 7f5b59600640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5b400a3d20 0x7f5b400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.763+0000 7f5b59600640 1 -- 172.21.15.44:0/3822730642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f5b480d0a80 con 0x7f5b5c101570 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b61400640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5b400a3d20 0x7f5b400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 -- 172.21.15.44:0/379333627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 msgr2=0x55d0251a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251a1130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009780 tx=0x7f5b4c009510 comp rx=0 tx=0).stop 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 -- 172.21.15.44:0/379333627 shutdown_connections 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d0251a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251a1130 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 --2- 172.21.15.44:0/379333627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198170 0x55d025198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 -- 172.21.15.44:0/379333627 >> 172.21.15.44:0/379333627 conn(0x55d0251928a0 msgr2=0x55d025192c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 -- 172.21.15.44:0/379333627 shutdown_connections 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b6621f740 1 -- 172.21.15.44:0/379333627 wait complete. 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.764+0000 7f5b61400640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5b400a3d20 0x7f5b400a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c06a1d0 tx=0x7f5b540096a0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 -- start start 2024-04-20T11:28:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 Processor -- start 2024-04-20T11:28:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d02519e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198a90 0x55d02519ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d0251a4400 con 0x55d025198a90 2024-04-20T11:28:24.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0250b6860 con 0x55d025198170 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.765+0000 7f5b6621f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0251a4580 con 0x55d0251a1670 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b6621f740 10 client.0 _ll_register_callbacks cb 0x55d025084fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198a90 0x55d02519ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198a90 0x55d02519ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32884/0 (socket says 172.21.15.44:32884) 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b5be00640 1 -- 172.21.15.44:0/2266855656 learned_addr learned my addr 172.21.15.44:0/2266855656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b60a00640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d02519e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b61400640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b60a00640 1 -- 172.21.15.44:0/2266855656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 msgr2=0x55d02519f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b60a00640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b60a00640 1 -- 172.21.15.44:0/2266855656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198a90 msgr2=0x55d02519ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b60a00640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198a90 0x55d02519ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b60a00640 1 -- 172.21.15.44:0/2266855656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0250ba950 con 0x55d025198170 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.766+0000 7f5b61400640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:24.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b60a00640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d02519e6c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c02c0b0 tx=0x7f5b4c034920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19093cacf6f639a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b46000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b46000640 1 -- 172.21.15.44:0/2266855656 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c043030 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b46000640 1 -- 172.21.15.44:0/2266855656 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f5b4c004250 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4c034d50 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b46000640 1 -- 172.21.15.44:0/2266855656 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5b4c05a050 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b6621f740 10 client.27026 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d025088820 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.767+0000 7f5b46000640 1 -- 172.21.15.44:0/2266855656 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b4c034d50 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b46000640 1 -- 172.21.15.44:0/2266855656 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f5b4c032980 con 0x55d025198170 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b6621f740 10 client.27026 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b6621f740 10 client.27026 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b6621f740 20 client.27026 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b6621f740 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d0251b14b0 0x55d0251b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b6621f740 4 client.27026 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d02510aa70 con 0x55d0251b14b0 2024-04-20T11:28:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.768+0000 7f5b5be00640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d0251b14b0 0x55d0251b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.769+0000 7f5b5be00640 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d0251b14b0 0x55d0251b38a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c5f33f2eb1abbff server_cookie=3880d09bbffcde5a in_seq=0 out_seq=0 2024-04-20T11:28:24.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.769+0000 7f5b46000640 10 client.27026 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b46000640 1 -- 172.21.15.44:0/2266855656 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d02510aa70 con 0x55d0251b14b0 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b46000640 10 client.27026 handle_command_reply: tid=0 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 client.27026 shutdown 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 msgr2=0x55d02519e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d02519e6c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c02c0b0 tx=0x7f5b4c034920 comp rx=0 tx=0).stop 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 shutdown_connections 2024-04-20T11:28:24.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d0251b14b0 0x55d0251b38a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198a90 0x55d02519ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 --2- 172.21.15.44:0/2266855656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d02519e6c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 >> 172.21.15.44:0/2266855656 conn(0x55d0251928a0 msgr2=0x55d0251945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b46000640 0 client.27026 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 shutdown_connections 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 1 -- 172.21.15.44:0/2266855656 wait complete. 2024-04-20T11:28:24.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.770+0000 7f5b6621f740 20 client.27026 trim_cache size 0 max 16384 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.785+0000 7f5b6621f740 1 Processor -- start 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.785+0000 7f5b6621f740 1 -- start start 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.785+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 0x55d0251a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b6621f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1afcf0 con 0x55d025198170 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b6621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1b18c0 con 0x55d025198a90 2024-04-20T11:28:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b6621f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1b03c0 con 0x55d0251a1670 2024-04-20T11:28:24.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 0x55d0251a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 0x55d0251a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32898/0 (socket says 172.21.15.44:32898) 2024-04-20T11:28:24.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 -- 172.21.15.44:0/564073467 learned_addr learned my addr 172.21.15.44:0/564073467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b5be00640 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b61400640 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 -- 172.21.15.44:0/564073467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 msgr2=0x55d02519d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 -- 172.21.15.44:0/564073467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 msgr2=0x55d0251ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.786+0000 7f5b60a00640 1 -- 172.21.15.44:0/564073467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d024e0f3f0 con 0x55d025198170 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.787+0000 7f5b60a00640 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 0x55d0251a0df0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c032d20 tx=0x7f5b4c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09a97dde869e6a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.787+0000 7f5b51200640 1 -- 172.21.15.44:0/564073467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c057020 con 0x55d025198170 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.787+0000 7f5b51200640 1 -- 172.21.15.44:0/564073467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b4c034d50 con 0x55d025198170 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.787+0000 7f5b51200640 1 -- 172.21.15.44:0/564073467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c043070 con 0x55d025198170 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 -- 172.21.15.44:0/564073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 msgr2=0x55d0251a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 0x55d0251a0df0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c032d20 tx=0x7f5b4c009510 comp rx=0 tx=0).stop 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 -- 172.21.15.44:0/564073467 shutdown_connections 2024-04-20T11:28:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0251a1670 0x55d02519d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198a90 0x55d0251ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 --2- 172.21.15.44:0/564073467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d025198170 0x55d0251a0df0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 -- 172.21.15.44:0/564073467 >> 172.21.15.44:0/564073467 conn(0x55d0251928a0 msgr2=0x55d025193ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 -- 172.21.15.44:0/564073467 shutdown_connections 2024-04-20T11:28:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 -- 172.21.15.44:0/564073467 wait complete. 2024-04-20T11:28:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 -- start start 2024-04-20T11:28:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.788+0000 7f5b6621f740 1 Processor -- start 2024-04-20T11:28:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d0251a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198a90 0x55d0251a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d02519ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1b11c0 con 0x55d0251a1670 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1b0ac0 con 0x55d025198170 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1adcb0 con 0x55d025198a90 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b6621f740 10 client.0 _ll_register_callbacks cb 0x7f5b4c0347b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b61400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d02519ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b61400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d02519ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32910/0 (socket says 172.21.15.44:32910) 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b61400640 1 -- 172.21.15.44:0/39941320 learned_addr learned my addr 172.21.15.44:0/39941320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b60a00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d0251a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.789+0000 7f5b5be00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198a90 0x55d0251a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b5be00640 1 -- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 msgr2=0x55d0251a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b5be00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d0251a97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b5be00640 1 -- 172.21.15.44:0/39941320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 msgr2=0x55d02519ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b5be00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d02519ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b5be00640 1 -- 172.21.15.44:0/39941320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d024ed97d0 con 0x55d025198a90 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b60a00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d0251a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b61400640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d02519ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b5be00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198a90 0x55d0251a9d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c6a0 tx=0x7f5b480d2ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57dcab55d177e995 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b46000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b46000640 1 -- 172.21.15.44:0/39941320 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4809d130 con 0x55d025198a90 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4c034d50 con 0x55d025198a90 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b46000640 1 -- 172.21.15.44:0/39941320 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f5b480990a0 con 0x55d025198a90 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b46000640 1 -- 172.21.15.44:0/39941320 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5b4809ca30 con 0x55d025198a90 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b6621f740 10 client.27031 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.790+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d025217050 con 0x55d025198a90 2024-04-20T11:28:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b46000640 1 -- 172.21.15.44:0/39941320 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b4c034d50 con 0x55d025198a90 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b46000640 1 -- 172.21.15.44:0/39941320 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f5b480ce460 con 0x55d025198a90 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b6621f740 10 client.27031 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b6621f740 10 client.27031 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b6621f740 20 client.27031 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b6621f740 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d025217770 0x55d025219b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b6621f740 4 client.27031 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0251048e0 con 0x55d025217770 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.791+0000 7f5b60a00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d025217770 0x55d025219b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.792+0000 7f5b60a00640 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d025217770 0x55d025219b90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5665d1219373f767 server_cookie=9c4701fd828db79f in_seq=0 out_seq=0 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.792+0000 7f5b46000640 10 client.27031 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.792+0000 7f5b46000640 1 -- 172.21.15.44:0/39941320 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0251048e0 con 0x55d025217770 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.792+0000 7f5b46000640 10 client.27031 handle_command_reply: tid=0 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.792+0000 7f5b6621f740 1 client.27031 shutdown 2024-04-20T11:28:24.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198a90 msgr2=0x55d0251a9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198a90 0x55d0251a9d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c6a0 tx=0x7f5b480d2ed0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 shutdown_connections 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55d025217770 0x55d025219b90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d0251a1670 0x55d02519ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d025198a90 0x55d0251a9d10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b46000640 0 client.27031 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 --2- 172.21.15.44:0/39941320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d025198170 0x55d0251a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 >> 172.21.15.44:0/39941320 conn(0x55d0251928a0 msgr2=0x55d0251910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 shutdown_connections 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 1 -- 172.21.15.44:0/39941320 wait complete. 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b6621f740 20 client.27031 trim_cache size 0 max 16384 2024-04-20T11:28:24.796 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.793+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5b400a3d20 msgr2=0x7f5b400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5b400a3d20 0x7f5b400a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c06a1d0 tx=0x7f5b540096a0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 msgr2=0x7f5b5c1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 0x7f5b5c1a0c00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9c0 tx=0x7f5b4800ee80 comp rx=0 tx=0).stop 2024-04-20T11:28:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 shutdown_connections 2024-04-20T11:28:24.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b5c10e770 0x7f5b5c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5b400a3d20 0x7f5b400a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b5c101570 0x7f5b5c1a0c00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 --2- 172.21.15.44:0/3822730642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b5c100bc0 0x7f5b5c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 >> 172.21.15.44:0/3822730642 conn(0x7f5b5c0fc7e0 msgr2=0x7f5b5c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 shutdown_connections 2024-04-20T11:28:24.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:24.794+0000 7f5b61e00640 1 -- 172.21.15.44:0/3822730642 wait complete. 2024-04-20T11:28:24.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:24 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4146183129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:28:24.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:28:24.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:24 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4146183129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:28:24.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:24 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4146183129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:28:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 -- 172.21.15.44:0/3946694030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104dc0 msgr2=0x7fad44108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 --2- 172.21.15.44:0/3946694030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104dc0 0x7fad44108e00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009ca0 tx=0x7fad2c02c100 comp rx=0 tx=0).stop 2024-04-20T11:28:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 -- 172.21.15.44:0/3946694030 shutdown_connections 2024-04-20T11:28:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 --2- 172.21.15.44:0/3946694030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad441094e0 0x7fad441110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 --2- 172.21.15.44:0/3946694030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104dc0 0x7fad44108e00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 --2- 172.21.15.44:0/3946694030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad44104410 0x7fad441047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 -- 172.21.15.44:0/3946694030 >> 172.21.15.44:0/3946694030 conn(0x7fad440ffcf0 msgr2=0x7fad44102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 -- 172.21.15.44:0/3946694030 shutdown_connections 2024-04-20T11:28:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.022+0000 7fad49600640 1 -- 172.21.15.44:0/3946694030 wait complete. 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.023+0000 7fad49600640 1 Processor -- start 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 -- start start 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104410 0x7fad441a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad441094e0 0x7fad441a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 0x7fad4419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad44114690 con 0x7fad4419aae0 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad44114510 con 0x7fad44104410 2024-04-20T11:28:25.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad49600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad44114810 con 0x7fad441094e0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad43400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 0x7fad4419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad43400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 0x7fad4419af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32914/0 (socket says 172.21.15.44:32914) 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad43400640 1 -- 172.21.15.44:0/211573443 learned_addr learned my addr 172.21.15.44:0/211573443 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad42a00640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104410 0x7fad441a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.024+0000 7fad42000640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad441094e0 0x7fad441a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad43400640 1 -- 172.21.15.44:0/211573443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad441094e0 msgr2=0x7fad441a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad43400640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad441094e0 0x7fad441a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad43400640 1 -- 172.21.15.44:0/211573443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104410 msgr2=0x7fad441a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad43400640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104410 0x7fad441a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad43400640 1 -- 172.21.15.44:0/211573443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4419b600 con 0x7fad4419aae0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad43400640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 0x7fad4419af90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00e970 tx=0x7fad3c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad3b400640 1 -- 172.21.15.44:0/211573443 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00ccc0 con 0x7fad4419aae0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad3b400640 1 -- 172.21.15.44:0/211573443 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad3c00ce60 con 0x7fad4419aae0 2024-04-20T11:28:25.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.025+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4419b8f0 con 0x7fad4419aae0 2024-04-20T11:28:25.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.026+0000 7fad3b400640 1 -- 172.21.15.44:0/211573443 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c010690 con 0x7fad4419aae0 2024-04-20T11:28:25.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.026+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad441a7aa0 con 0x7fad4419aae0 2024-04-20T11:28:25.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.030+0000 7fad4c398740 1 Processor -- start 2024-04-20T11:28:25.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.030+0000 7fad3b400640 1 -- 172.21.15.44:0/211573443 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fad3c0046c0 con 0x7fad4419aae0 2024-04-20T11:28:25.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.030+0000 7fad4c398740 1 -- start start 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad4c398740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a66b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad4c398740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a673f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad4c398740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad4c398740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b3a5d0050 con 0x557b3a66afb0 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad4c398740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b3a576490 con 0x557b3a66b8d0 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad4c398740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b3a5cc010 con 0x557b3a6744b0 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a673f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a66b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a673f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59278/0 (socket says 172.21.15.44:59278) 2024-04-20T11:28:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad42a00640 1 -- 172.21.15.44:0/2216627196 learned_addr learned my addr 172.21.15.44:0/2216627196 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.031+0000 7fad42000640 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42a00640 1 -- 172.21.15.44:0/2216627196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 msgr2=0x557b3a6768a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42a00640 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6768a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42a00640 1 -- 172.21.15.44:0/2216627196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 msgr2=0x557b3a66b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42a00640 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a66b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42a00640 1 -- 172.21.15.44:0/2216627196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3a57e070 con 0x557b3a66b8d0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42000640 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a66b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42a00640 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a673f70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fad34009c90 tx=0x7fad34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e554b670a652e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad3b400640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fad240a3d20 0x7fad240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad3b400640 1 -- 172.21.15.44:0/211573443 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fad3c0d0b80 con 0x7fad4419aae0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad42000640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fad240a3d20 0x7fad240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.032+0000 7fad33400640 1 -- 172.21.15.44:0/2216627196 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34005220 con 0x557b3a66b8d0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.033+0000 7fad33400640 1 -- 172.21.15.44:0/2216627196 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad34004ca0 con 0x557b3a66b8d0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.033+0000 7fad33400640 1 -- 172.21.15.44:0/2216627196 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34019810 con 0x557b3a66b8d0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.033+0000 7fad4c398740 1 -- 172.21.15.44:0/2216627196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 msgr2=0x557b3a673f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.033+0000 7fad4c398740 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a673f70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fad34009c90 tx=0x7fad34011810 comp rx=0 tx=0).stop 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.033+0000 7fad42000640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fad240a3d20 0x7fad240a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad2c02c710 tx=0x7fad2c02e040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.033+0000 7fad4c398740 1 -- 172.21.15.44:0/2216627196 shutdown_connections 2024-04-20T11:28:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a673f70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 --2- 172.21.15.44:0/2216627196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a66b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 -- 172.21.15.44:0/2216627196 >> 172.21.15.44:0/2216627196 conn(0x557b3a6656e0 msgr2=0x557b3a665ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 -- 172.21.15.44:0/2216627196 shutdown_connections 2024-04-20T11:28:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 -- 172.21.15.44:0/2216627196 wait complete. 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 -- start start 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.034+0000 7fad4c398740 1 Processor -- start 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a671510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a671a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a671f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b3a6772a0 con 0x557b3a66afb0 2024-04-20T11:28:25.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b3a589860 con 0x557b3a66b8d0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b3a677420 con 0x557b3a6744b0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad4c398740 10 client.0 _ll_register_callbacks cb 0x557b3a557fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad42a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a671510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad42000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a671a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad42a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a671510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32932/0 (socket says 172.21.15.44:32932) 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad42a00640 1 -- 172.21.15.44:0/661842708 learned_addr learned my addr 172.21.15.44:0/661842708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.035+0000 7fad43400640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a671f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42a00640 1 -- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 msgr2=0x557b3a671f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42a00640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a671f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42a00640 1 -- 172.21.15.44:0/661842708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 msgr2=0x557b3a671a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42a00640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a671a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42a00640 1 -- 172.21.15.44:0/661842708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3a58d950 con 0x557b3a66afb0 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42000640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a671a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad42a00640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a671510 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fad34005080 tx=0x7fad340050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85577901ce371340 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad2be00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad2be00640 1 -- 172.21.15.44:0/661842708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34029070 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad2be00640 1 -- 172.21.15.44:0/661842708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fad34019920 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.036+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad34004ca0 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad2be00640 1 -- 172.21.15.44:0/661842708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fad34035050 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 10 client.16914 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557b3a55b820 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad2be00640 1 -- 172.21.15.44:0/661842708 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad34004ca0 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad2be00640 1 -- 172.21.15.44:0/661842708 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7fad34023930 con 0x557b3a66afb0 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 10 client.16914 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 20 client.16914 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6843c0 0x557b3a6867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 4 client.16914 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:25.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.037+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3a5dda70 con 0x557b3a6843c0 2024-04-20T11:28:25.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.038+0000 7fad42000640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6843c0 0x557b3a6867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.038+0000 7fad42000640 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6843c0 0x557b3a6867b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76500e032a51991e server_cookie=9c912d570c2d0c4a in_seq=0 out_seq=0 2024-04-20T11:28:25.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.038+0000 7fad2be00640 10 client.16914 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.039+0000 7fad2be00640 1 -- 172.21.15.44:0/661842708 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b3a5dda70 con 0x557b3a6843c0 2024-04-20T11:28:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.039+0000 7fad2be00640 10 client.16914 handle_command_reply: tid=0 2024-04-20T11:28:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.039+0000 7fad4c398740 1 client.16914 shutdown 2024-04-20T11:28:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 msgr2=0x557b3a671510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a671510 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fad34005080 tx=0x7fad340050b0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 shutdown_connections 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6843c0 0x557b3a6867b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a671f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66b8d0 0x557b3a671a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 --2- 172.21.15.44:0/661842708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a671510 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 >> 172.21.15.44:0/661842708 conn(0x557b3a6656e0 msgr2=0x557b3a667430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad2be00640 0 client.16914 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 shutdown_connections 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 1 -- 172.21.15.44:0/661842708 wait complete. 2024-04-20T11:28:25.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.040+0000 7fad4c398740 20 client.16914 trim_cache size 0 max 16384 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 Processor -- start 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 -- start start 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a673c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 0x557b3a680160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a6744b0 0x557b3a6704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad441afea0 con 0x557b3a66afb0 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad441b1a70 con 0x557b3a6744b0 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.055+0000 7fad4c398740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad441b0570 con 0x557b3a66b8d0 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad42000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 0x557b3a680160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a6744b0 0x557b3a6704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad42000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 0x557b3a680160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44222/0 (socket says 172.21.15.44:44222) 2024-04-20T11:28:25.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad42000640 1 -- 172.21.15.44:0/3909119031 learned_addr learned my addr 172.21.15.44:0/3909119031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad42a00640 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a673c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 -- 172.21.15.44:0/3909119031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 msgr2=0x557b3a680160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 0x557b3a680160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 -- 172.21.15.44:0/3909119031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 msgr2=0x557b3a673c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a673c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 -- 172.21.15.44:0/3909119031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3a2e23f0 con 0x557b3a6744b0 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad42a00640 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a673c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad42000640 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 0x557b3a680160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.056+0000 7fad43400640 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a6744b0 0x557b3a6704f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0ce2e0 tx=0x7fad3c098e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8b9e3de855d5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad32a00640 1 -- 172.21.15.44:0/3909119031 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c09de60 con 0x557b3a6744b0 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad32a00640 1 -- 172.21.15.44:0/3909119031 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad3c0a57a0 con 0x557b3a6744b0 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad32a00640 1 -- 172.21.15.44:0/3909119031 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c0a5530 con 0x557b3a6744b0 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 -- 172.21.15.44:0/3909119031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a6744b0 msgr2=0x557b3a6704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a6744b0 0x557b3a6704f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0ce2e0 tx=0x7fad3c098e50 comp rx=0 tx=0).stop 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 -- 172.21.15.44:0/3909119031 shutdown_connections 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a6744b0 0x557b3a6704f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a66b8d0 0x557b3a680160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 --2- 172.21.15.44:0/3909119031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66afb0 0x557b3a673c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 -- 172.21.15.44:0/3909119031 >> 172.21.15.44:0/3909119031 conn(0x557b3a6656e0 msgr2=0x557b3a66fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.057+0000 7fad4c398740 1 -- 172.21.15.44:0/3909119031 shutdown_connections 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.058+0000 7fad4c398740 1 -- 172.21.15.44:0/3909119031 wait complete. 2024-04-20T11:28:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.058+0000 7fad4c398740 1 -- start start 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.058+0000 7fad4c398740 1 Processor -- start 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 0x557b3a677630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 0x557b3a677b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad441b1370 con 0x557b3a66b8d0 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad441b0c70 con 0x557b3a66afb0 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad441ade60 con 0x557b3a6744b0 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad4c398740 10 client.0 _ll_register_callbacks cb 0x7fad340049f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 0x557b3a677b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 0x557b3a677630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 0x557b3a677630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59296/0 (socket says 172.21.15.44:59296) 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 -- 172.21.15.44:0/1825849057 learned_addr learned my addr 172.21.15.44:0/1825849057 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 0x557b3a677b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32948/0 (socket says 172.21.15.44:32948) 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad43400640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 -- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 msgr2=0x557b3a6e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6e87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 -- 172.21.15.44:0/1825849057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 msgr2=0x557b3a677b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 0x557b3a677b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42a00640 1 -- 172.21.15.44:0/1825849057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3a3ac7d0 con 0x557b3a66afb0 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad43400640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.059+0000 7fad42000640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 0x557b3a677b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad42a00640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 0x557b3a677630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fad34023d30 tx=0x7fad340096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6fb1e3aa548b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad2be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad2be00640 1 -- 172.21.15.44:0/1825849057 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3403e060 con 0x557b3a66afb0 2024-04-20T11:28:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad3c0a57a0 con 0x557b3a66afb0 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad2be00640 1 -- 172.21.15.44:0/1825849057 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fad34004aa0 con 0x557b3a66afb0 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad2be00640 1 -- 172.21.15.44:0/1825849057 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fad34029040 con 0x557b3a66afb0 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad4c398740 10 client.27050 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.060+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557b3a6e9c70 con 0x557b3a66afb0 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad2be00640 1 -- 172.21.15.44:0/1825849057 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad3c0a57a0 con 0x557b3a66afb0 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad2be00640 1 -- 172.21.15.44:0/1825849057 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7fad34041020 con 0x557b3a66afb0 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad4c398740 10 client.27050 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad4c398740 10 client.27050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad4c398740 20 client.27050 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad4c398740 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6ea390 0x557b3a6ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad4c398740 4 client.27050 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b3a5d78e0 con 0x557b3a6ea390 2024-04-20T11:28:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.061+0000 7fad42000640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6ea390 0x557b3a6ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad42000640 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6ea390 0x557b3a6ec7b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d9756a5df17542c server_cookie=d3874cafe15da457 in_seq=0 out_seq=0 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad2be00640 10 client.27050 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad2be00640 1 -- 172.21.15.44:0/1825849057 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b3a5d78e0 con 0x557b3a6ea390 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad2be00640 10 client.27050 handle_command_reply: tid=0 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 client.27050 shutdown 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 msgr2=0x557b3a677630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 0x557b3a677630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fad34023d30 tx=0x7fad340096b0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 shutdown_connections 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b3a6ea390 0x557b3a6ec7b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b3a6744b0 0x557b3a6e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b3a66b8d0 0x557b3a677b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 --2- 172.21.15.44:0/1825849057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b3a66afb0 0x557b3a677630 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 >> 172.21.15.44:0/1825849057 conn(0x557b3a6656e0 msgr2=0x557b3a670670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.062+0000 7fad2be00640 0 client.27050 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 shutdown_connections 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad4c398740 1 -- 172.21.15.44:0/1825849057 wait complete. 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad4c398740 20 client.27050 trim_cache size 0 max 16384 2024-04-20T11:28:25.067 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fad240a3d20 msgr2=0x7fad240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fad240a3d20 0x7fad240a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad2c02c710 tx=0x7fad2c02e040 comp rx=0 tx=0).stop 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 msgr2=0x7fad4419af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 0x7fad4419af90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00e970 tx=0x7fad3c00ed70 comp rx=0 tx=0).stop 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 shutdown_connections 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fad240a3d20 0x7fad240a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad4419aae0 0x7fad4419af90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad441094e0 0x7fad441a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 --2- 172.21.15.44:0/211573443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad44104410 0x7fad441a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.063+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 >> 172.21.15.44:0/211573443 conn(0x7fad440ffcf0 msgr2=0x7fad44102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.064+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 shutdown_connections 2024-04-20T11:28:25.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.064+0000 7fad49600640 1 -- 172.21.15.44:0/211573443 wait complete. 2024-04-20T11:28:25.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:28:25.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.288+0000 7fa2d6800640 1 -- 172.21.15.44:0/1844537717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076db0 msgr2=0x7fa2d0112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.288+0000 7fa2d6800640 1 --2- 172.21.15.44:0/1844537717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076db0 0x7fa2d0112760 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009900 tx=0x7fa2bc02bc40 comp rx=0 tx=0).stop 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 -- 172.21.15.44:0/1844537717 shutdown_connections 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 --2- 172.21.15.44:0/1844537717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 0x7fa2d0115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 --2- 172.21.15.44:0/1844537717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076db0 0x7fa2d0112760 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 --2- 172.21.15.44:0/1844537717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d0076490 0x7fa2d0076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 -- 172.21.15.44:0/1844537717 >> 172.21.15.44:0/1844537717 conn(0x7fa2d00ffad0 msgr2=0x7fa2d0101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 -- 172.21.15.44:0/1844537717 shutdown_connections 2024-04-20T11:28:25.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.289+0000 7fa2d6800640 1 -- 172.21.15.44:0/1844537717 wait complete. 2024-04-20T11:28:25.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.290+0000 7fa2d6800640 1 Processor -- start 2024-04-20T11:28:25.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.290+0000 7fa2d6800640 1 -- start start 2024-04-20T11:28:25.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 0x7fa2d019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d0076db0 0x7fa2d019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 0x7fa2d01a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01188d0 con 0x7fa2d0076490 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d6800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0118750 con 0x7fa2d0112ca0 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d6800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0118a50 con 0x7fa2d0076db0 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 0x7fa2d019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 0x7fa2d019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32968/0 (socket says 172.21.15.44:32968) 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d5400640 1 -- 172.21.15.44:0/3576502521 learned_addr learned my addr 172.21.15.44:0/3576502521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.291+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 0x7fa2d01a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5e00640 1 -- 172.21.15.44:0/3576502521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d0076db0 msgr2=0x7fa2d019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d0076db0 0x7fa2d019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5e00640 1 -- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 msgr2=0x7fa2d019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 0x7fa2d019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5e00640 1 -- 172.21.15.44:0/3576502521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d01a3de0 con 0x7fa2d0112ca0 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 0x7fa2d019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 0x7fa2d01a36b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005fd0 tx=0x7fa2c800dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2ce000640 1 -- 172.21.15.44:0/3576502521 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800bd40 con 0x7fa2d0112ca0 2024-04-20T11:28:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d01a40d0 con 0x7fa2d0112ca0 2024-04-20T11:28:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2ce000640 1 -- 172.21.15.44:0/3576502521 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa2c8004030 con 0x7fa2d0112ca0 2024-04-20T11:28:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.292+0000 7fa2ce000640 1 -- 172.21.15.44:0/3576502521 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8010790 con 0x7fa2d0112ca0 2024-04-20T11:28:25.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.293+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d01ab960 con 0x7fa2d0112ca0 2024-04-20T11:28:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.296+0000 7fa2daa27740 1 Processor -- start 2024-04-20T11:28:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.296+0000 7fa2ce000640 1 -- 172.21.15.44:0/3576502521 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa2c80041e0 con 0x7fa2d0112ca0 2024-04-20T11:28:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.296+0000 7fa2daa27740 1 -- start start 2024-04-20T11:28:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fa5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5a90 0x5586d0fae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2daa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5586d0f0a050 con 0x5586d0fa5170 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2daa27740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586d0eb0490 con 0x5586d0fae670 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2daa27740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5586d0f06010 con 0x5586d0fa5a90 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2d4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fa5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5a90 0x5586d0fae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2d4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fa5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32970/0 (socket says 172.21.15.44:32970) 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.297+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3477817660 learned_addr learned my addr 172.21.15.44:0/3477817660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3477817660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5a90 msgr2=0x5586d0fae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5a90 0x5586d0fae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3477817660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 msgr2=0x5586d0fb0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fb0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3477817660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d0eb8070 con 0x5586d0fa5170 2024-04-20T11:28:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5a90 0x5586d0fae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fa5550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc02c750 tx=0x7fa2bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f01ed91b3212622 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2ce000640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa2b80a3da0 0x7fa2b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.298+0000 7fa2c2600640 1 -- 172.21.15.44:0/3477817660 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc02cbd0 con 0x5586d0fa5170 2024-04-20T11:28:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.299+0000 7fa2c2600640 1 -- 172.21.15.44:0/3477817660 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa2bc046070 con 0x5586d0fa5170 2024-04-20T11:28:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.299+0000 7fa2c2600640 1 -- 172.21.15.44:0/3477817660 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc003cd0 con 0x5586d0fa5170 2024-04-20T11:28:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.299+0000 7fa2ce000640 1 -- 172.21.15.44:0/3576502521 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa2c80d0ad0 con 0x7fa2d0112ca0 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.299+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa2b80a3da0 0x7fa2b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.299+0000 7fa2daa27740 1 -- 172.21.15.44:0/3477817660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 msgr2=0x5586d0fa5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.299+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fa5550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc02c750 tx=0x7fa2bc0056d0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa2b80a3da0 0x7fa2b80a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c010 tx=0x7fa2c400b480 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 -- 172.21.15.44:0/3477817660 shutdown_connections 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5a90 0x5586d0fae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3477817660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fa5550 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 -- 172.21.15.44:0/3477817660 >> 172.21.15.44:0/3477817660 conn(0x5586d0f9f8a0 msgr2=0x5586d0f9fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 -- 172.21.15.44:0/3477817660 shutdown_connections 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 -- 172.21.15.44:0/3477817660 wait complete. 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 -- start start 2024-04-20T11:28:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.300+0000 7fa2daa27740 1 Processor -- start 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5170 0x5586d0fab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fabc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5586d0fb1400 con 0x5586d0fa5a90 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586d0ec3860 con 0x5586d0fa5170 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5586d0fb1580 con 0x5586d0fae670 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2daa27740 10 client.0 _ll_register_callbacks cb 0x5586d0e91fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2d4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fabc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5170 0x5586d0fab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2d4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fabc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32986/0 (socket says 172.21.15.44:32986) 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2d4a00640 1 -- 172.21.15.44:0/2546479001 learned_addr learned my addr 172.21.15.44:0/2546479001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.301+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2d4a00640 1 -- 172.21.15.44:0/2546479001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 msgr2=0x5586d0fac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2d4a00640 1 -- 172.21.15.44:0/2546479001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5170 msgr2=0x5586d0fab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5170 0x5586d0fab730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2d4a00640 1 -- 172.21.15.44:0/2546479001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d0ec7950 con 0x5586d0fa5a90 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fabc70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0092e0 tx=0x7fa2bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110e9498644150bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2b6a00640 1 -- 172.21.15.44:0/2546479001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc04e070 con 0x5586d0fa5a90 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2b6a00640 1 -- 172.21.15.44:0/2546479001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa2bc02c9d0 con 0x5586d0fa5a90 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.302+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2bc046070 con 0x5586d0fa5a90 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2b6a00640 1 -- 172.21.15.44:0/2546479001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa2bc05a050 con 0x5586d0fa5a90 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 10 client.16932 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5586d0e95820 con 0x5586d0fa5a90 2024-04-20T11:28:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2b6a00640 1 -- 172.21.15.44:0/2546479001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa2bc046070 con 0x5586d0fa5a90 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2b6a00640 1 -- 172.21.15.44:0/2546479001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7fa2bc040040 con 0x5586d0fa5a90 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 10 client.16932 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 20 client.16932 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d0fbe520 0x5586d0fc0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 4 client.16932 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.303+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d0f17a70 con 0x5586d0fbe520 2024-04-20T11:28:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.304+0000 7fa2d5400640 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d0fbe520 0x5586d0fc0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.304+0000 7fa2d5400640 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d0fbe520 0x5586d0fc0910 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b422c3f0dc53bca1 server_cookie=b4c5760c4c7f4bbb in_seq=0 out_seq=0 2024-04-20T11:28:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.304+0000 7fa2b6a00640 10 client.16932 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2b6a00640 1 -- 172.21.15.44:0/2546479001 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586d0f17a70 con 0x5586d0fbe520 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2b6a00640 10 client.16932 handle_command_reply: tid=0 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 client.16932 shutdown 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 msgr2=0x5586d0fabc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fabc70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0092e0 tx=0x7fa2bc000bc0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 shutdown_connections 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d0fbe520 0x5586d0fc0910 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fabc70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2b6a00640 0 client.16932 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 --2- 172.21.15.44:0/2546479001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5170 0x5586d0fab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.305+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 >> 172.21.15.44:0/2546479001 conn(0x5586d0f9f8a0 msgr2=0x5586d0fa15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.306+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 shutdown_connections 2024-04-20T11:28:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.306+0000 7fa2daa27740 1 -- 172.21.15.44:0/2546479001 wait complete. 2024-04-20T11:28:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.306+0000 7fa2daa27740 20 client.16932 trim_cache size 0 max 16384 2024-04-20T11:28:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 Processor -- start 2024-04-20T11:28:25.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 -- start start 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5a90 0x5586d0fa89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fa8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b3f00 con 0x5586d0fa5170 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b5ad0 con 0x5586d0fa5a90 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2daa27740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b45d0 con 0x5586d0fae670 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:32996/0 (socket says 172.21.15.44:32996) 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2d5400640 1 -- 172.21.15.44:0/1603473164 learned_addr learned my addr 172.21.15.44:0/1603473164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.321+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5a90 0x5586d0fa89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fa8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5400640 1 -- 172.21.15.44:0/1603473164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 msgr2=0x5586d0fa8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5400640 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fa8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5400640 1 -- 172.21.15.44:0/1603473164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5a90 msgr2=0x5586d0fa89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5400640 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5a90 0x5586d0fa89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5400640 1 -- 172.21.15.44:0/1603473164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d0c1c3f0 con 0x5586d0fa5170 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5a90 0x5586d0fa89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2d5400640 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fade60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005fd0 tx=0x7fa2c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2136f9cabbb77967 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2c1c00640 1 -- 172.21.15.44:0/1603473164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4010e10 con 0x5586d0fa5170 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.322+0000 7fa2c1c00640 1 -- 172.21.15.44:0/1603473164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa2c402a440 con 0x5586d0fa5170 2024-04-20T11:28:25.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2c1c00640 1 -- 172.21.15.44:0/1603473164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4005020 con 0x5586d0fa5170 2024-04-20T11:28:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 -- 172.21.15.44:0/1603473164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 msgr2=0x5586d0fade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fade60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005fd0 tx=0x7fa2c4019000 comp rx=0 tx=0).stop 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 -- 172.21.15.44:0/1603473164 shutdown_connections 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fae670 0x5586d0fa8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fa5a90 0x5586d0fa89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 --2- 172.21.15.44:0/1603473164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5170 0x5586d0fade60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 -- 172.21.15.44:0/1603473164 >> 172.21.15.44:0/1603473164 conn(0x5586d0f9f8a0 msgr2=0x5586d0fa6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 -- 172.21.15.44:0/1603473164 shutdown_connections 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.323+0000 7fa2daa27740 1 -- 172.21.15.44:0/1603473164 wait complete. 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 -- start start 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 Processor -- start 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 0x5586d0fc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fc1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fc1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b53d0 con 0x5586d0fa5a90 2024-04-20T11:28:25.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b4cd0 con 0x5586d0fae670 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b1ec0 con 0x5586d0fa5170 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.324+0000 7fa2daa27740 10 client.0 _ll_register_callbacks cb 0x7fa2bc02c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fc1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 0x5586d0fc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 0x5586d0fc12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44278/0 (socket says 172.21.15.44:44278) 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d5400640 1 -- 172.21.15.44:0/3014317171 learned_addr learned my addr 172.21.15.44:0/3014317171 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fc1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3014317171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 msgr2=0x5586d0fc12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 0x5586d0fc12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3014317171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 msgr2=0x5586d0fc1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fc1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 -- 172.21.15.44:0/3014317171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d0ce67d0 con 0x5586d0fa5a90 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 0x5586d0fc12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d5e00640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fc1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2d4a00640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fc1810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0095b0 tx=0x7fa2bc005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2d7fc9c18c1178 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.325+0000 7fa2b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2b6a00640 1 -- 172.21.15.44:0/3014317171 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc04e030 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2b6a00640 1 -- 172.21.15.44:0/3014317171 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7fa2bc034d60 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c402a440 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2b6a00640 1 -- 172.21.15.44:0/3014317171 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa2bc0344e0 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2daa27740 10 client.16938 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5586d1024770 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2b6a00640 1 -- 172.21.15.44:0/3014317171 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa2c402a440 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2b6a00640 1 -- 172.21.15.44:0/3014317171 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7fa2bc0054d0 con 0x5586d0fa5a90 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.326+0000 7fa2daa27740 10 client.16938 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.327+0000 7fa2daa27740 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.327+0000 7fa2daa27740 20 client.16938 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.327+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d1024ec0 0x5586d10272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.327+0000 7fa2daa27740 4 client.16938 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.327+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586d0f118e0 con 0x5586d1024ec0 2024-04-20T11:28:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.327+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d1024ec0 0x5586d10272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2b6a00640 10 client.16938 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2d5400640 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d1024ec0 0x5586d10272e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7b53ff4ba215364 server_cookie=87ec5fe33f673795 in_seq=0 out_seq=0 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2b6a00640 1 -- 172.21.15.44:0/3014317171 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586d0f118e0 con 0x5586d1024ec0 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2b6a00640 10 client.16938 handle_command_reply: tid=0 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 client.16938 shutdown 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 msgr2=0x5586d0fc1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fc1810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0095b0 tx=0x7fa2bc005990 comp rx=0 tx=0).stop 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 shutdown_connections 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5586d1024ec0 0x5586d10272e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586d0fae670 0x5586d0fc1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586d0fa5a90 0x5586d0fc1810 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 --2- 172.21.15.44:0/3014317171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586d0fa5170 0x5586d0fc12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2b6a00640 0 client.16938 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 >> 172.21.15.44:0/3014317171 conn(0x5586d0f9f8a0 msgr2=0x5586d0fb97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 shutdown_connections 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.328+0000 7fa2daa27740 1 -- 172.21.15.44:0/3014317171 wait complete. 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2daa27740 20 client.16938 trim_cache size 0 max 16384 2024-04-20T11:28:25.332 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa2b80a3da0 msgr2=0x7fa2b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa2b80a3da0 0x7fa2b80a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c010 tx=0x7fa2c400b480 comp rx=0 tx=0).stop 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 msgr2=0x7fa2d01a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 0x7fa2d01a36b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005fd0 tx=0x7fa2c800dea0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 shutdown_connections 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2d0112ca0 0x7fa2d01a36b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d0076db0 0x7fa2d019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa2b80a3da0 0x7fa2b80a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 --2- 172.21.15.44:0/3576502521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d0076490 0x7fa2d019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 >> 172.21.15.44:0/3576502521 conn(0x7fa2d00ffad0 msgr2=0x7fa2d0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 shutdown_connections 2024-04-20T11:28:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.329+0000 7fa2d6800640 1 -- 172.21.15.44:0/3576502521 wait complete. 2024-04-20T11:28:25.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:28:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 -- 172.21.15.44:0/2672540517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 msgr2=0x7f48b4103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 --2- 172.21.15.44:0/2672540517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b4103410 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009900 tx=0x7f48a402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 -- 172.21.15.44:0/2672540517 shutdown_connections 2024-04-20T11:28:25.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 --2- 172.21.15.44:0/2672540517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b410c450 0x7f48b410e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 --2- 172.21.15.44:0/2672540517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b4103950 0x7f48b410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 --2- 172.21.15.44:0/2672540517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b4103410 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 -- 172.21.15.44:0/2672540517 >> 172.21.15.44:0/2672540517 conn(0x7f48b40fc870 msgr2=0x7f48b40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 -- 172.21.15.44:0/2672540517 shutdown_connections 2024-04-20T11:28:25.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.562+0000 7f48b3e00640 1 -- 172.21.15.44:0/2672540517 wait complete. 2024-04-20T11:28:25.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 Processor -- start 2024-04-20T11:28:25.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 -- start start 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4103950 0x7f48b419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 0x7f48b4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4112370 con 0x7f48b4103030 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41121f0 con 0x7f48b4103950 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.564+0000 7f48b3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41124f0 con 0x7f48b410c450 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 0x7f48b4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 0x7f48b4198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44292/0 (socket says 172.21.15.44:44292) 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 -- 172.21.15.44:0/820817862 learned_addr learned my addr 172.21.15.44:0/820817862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33024/0 (socket says 172.21.15.44:33024) 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b2000640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4103950 0x7f48b419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 -- 172.21.15.44:0/820817862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4103950 msgr2=0x7f48b419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4103950 0x7f48b419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 -- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 msgr2=0x7f48b419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 -- 172.21.15.44:0/820817862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b4198f30 con 0x7f48b410c450 2024-04-20T11:28:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.565+0000 7f48b3400640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 0x7f48b4198670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00efd0 tx=0x7f48ac00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.566+0000 7f48ab400640 1 -- 172.21.15.44:0/820817862 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac003b70 con 0x7f48b410c450 2024-04-20T11:28:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.566+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b4199220 con 0x7f48b410c450 2024-04-20T11:28:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.566+0000 7f48ab400640 1 -- 172.21.15.44:0/820817862 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f48ac003d10 con 0x7f48b410c450 2024-04-20T11:28:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.566+0000 7f48ab400640 1 -- 172.21.15.44:0/820817862 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac010a90 con 0x7f48b410c450 2024-04-20T11:28:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.566+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b41a5530 con 0x7f48b410c450 2024-04-20T11:28:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.569+0000 7f48bc65a740 1 Processor -- start 2024-04-20T11:28:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.569+0000 7f48ab400640 1 -- 172.21.15.44:0/820817862 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f48ac004020 con 0x7f48b410c450 2024-04-20T11:28:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 -- start start 2024-04-20T11:28:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc05837960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc05840540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05842e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fc05798050 con 0x55fc05837580 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fc0573e490 con 0x55fc05840a80 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48bc65a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fc05794010 con 0x55fc05837ea0 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48b2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc05837960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc05840540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc05840540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44302/0 (socket says 172.21.15.44:44302) 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.570+0000 7f48b2a00640 1 -- 172.21.15.44:0/2291480698 learned_addr learned my addr 172.21.15.44:0/2291480698 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2000640 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05842e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2a00640 1 -- 172.21.15.44:0/2291480698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 msgr2=0x55fc05842e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2a00640 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05842e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2a00640 1 -- 172.21.15.44:0/2291480698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 msgr2=0x55fc05837960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2a00640 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc05837960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2a00640 1 -- 172.21.15.44:0/2291480698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc05746070 con 0x55fc05837ea0 2024-04-20T11:28:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2000640 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05842e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48ab400640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f48980a3da0 0x7f48980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2a00640 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc05840540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009900 tx=0x7f48a40095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555f612a7c4c33f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48ab400640 1 -- 172.21.15.44:0/820817862 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f48ac0d0ae0 con 0x7f48b410c450 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.571+0000 7f48b2000640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f48980a3da0 0x7f48980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f489fe00640 1 -- 172.21.15.44:0/2291480698 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a402c7a0 con 0x55fc05837ea0 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f489fe00640 1 -- 172.21.15.44:0/2291480698 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f48a4004040 con 0x55fc05837ea0 2024-04-20T11:28:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f489fe00640 1 -- 172.21.15.44:0/2291480698 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a40046c0 con 0x55fc05837ea0 2024-04-20T11:28:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 -- 172.21.15.44:0/2291480698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 msgr2=0x55fc05840540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48b2000640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f48980a3da0 0x7f48980a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009660 tx=0x7f48a0005850 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc05840540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009900 tx=0x7f48a40095f0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 -- 172.21.15.44:0/2291480698 shutdown_connections 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05842e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc05840540 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2291480698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc05837960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.572+0000 7f48bc65a740 1 -- 172.21.15.44:0/2291480698 >> 172.21.15.44:0/2291480698 conn(0x55fc05831cb0 msgr2=0x55fc05832090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.573+0000 7f48bc65a740 1 -- 172.21.15.44:0/2291480698 shutdown_connections 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.573+0000 7f48bc65a740 1 -- 172.21.15.44:0/2291480698 wait complete. 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.573+0000 7f48bc65a740 1 -- start start 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.573+0000 7f48bc65a740 1 Processor -- start 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc0583dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc0583e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fc05843810 con 0x55fc05837580 2024-04-20T11:28:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fc05751860 con 0x55fc05840a80 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fc05843990 con 0x55fc05837ea0 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48bc65a740 10 client.0 _ll_register_callbacks cb 0x55fc0571ffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc0583e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc0583e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59372/0 (socket says 172.21.15.44:59372) 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33042/0 (socket says 172.21.15.44:33042) 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b3400640 1 -- 172.21.15.44:0/3994675680 learned_addr learned my addr 172.21.15.44:0/3994675680 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b2000640 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc0583dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.574+0000 7f48b3400640 1 -- 172.21.15.44:0/3994675680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 msgr2=0x55fc0583dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48b3400640 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc0583dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48b3400640 1 -- 172.21.15.44:0/3994675680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 msgr2=0x55fc0583da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48b3400640 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48b3400640 1 -- 172.21.15.44:0/3994675680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc05755950 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48b3400640 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc0583e4f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0994c0 tx=0x7f48ac09c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb547932e09fcf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f489cc00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f489cc00640 1 -- 172.21.15.44:0/3994675680 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac09d1f0 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f489cc00640 1 -- 172.21.15.44:0/3994675680 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f48ac004310 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48a4004040 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f489cc00640 1 -- 172.21.15.44:0/3994675680 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f48ac09caf0 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48bc65a740 10 client.27077 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.575+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fc05723820 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f489cc00640 1 -- 172.21.15.44:0/3994675680 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f48a4004040 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f489cc00640 1 -- 172.21.15.44:0/3994675680 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f48ac01f020 con 0x55fc05840a80 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48bc65a740 10 client.27077 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48bc65a740 10 client.27077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48bc65a740 20 client.27077 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48bc65a740 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058509c0 0x55fc05852e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48bc65a740 4 client.27077 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc057a5a70 con 0x55fc058509c0 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.576+0000 7f48b2a00640 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058509c0 0x55fc05852e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.577+0000 7f48b2a00640 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058509c0 0x55fc05852e30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3003f653e85b695 server_cookie=ff5f361c8a3a5378 in_seq=0 out_seq=0 2024-04-20T11:28:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.577+0000 7f489cc00640 10 client.27077 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f489cc00640 1 -- 172.21.15.44:0/3994675680 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc057a5a70 con 0x55fc058509c0 2024-04-20T11:28:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f489cc00640 10 client.27077 handle_command_reply: tid=0 2024-04-20T11:28:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 client.27077 shutdown 2024-04-20T11:28:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 msgr2=0x55fc0583e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc0583e4f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0994c0 tx=0x7f48ac09c450 comp rx=0 tx=0).stop 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 shutdown_connections 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058509c0 0x55fc05852e30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc0583e4f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837ea0 0x55fc0583dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 --2- 172.21.15.44:0/3994675680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f489cc00640 0 client.27077 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 >> 172.21.15.44:0/3994675680 conn(0x55fc05831cb0 msgr2=0x55fc05833a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 shutdown_connections 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.578+0000 7f48bc65a740 1 -- 172.21.15.44:0/3994675680 wait complete. 2024-04-20T11:28:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.579+0000 7f48bc65a740 20 client.27077 trim_cache size 0 max 16384 2024-04-20T11:28:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 Processor -- start 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 -- start start 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 0x55fc058401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837ea0 0x55fc0584c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05858700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41ab730 con 0x55fc05837ea0 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41ad300 con 0x55fc05840a80 2024-04-20T11:28:25.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.596+0000 7f48bc65a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41abe00 con 0x55fc05837580 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 0x55fc058401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837ea0 0x55fc0584c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 0x55fc058401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44318/0 (socket says 172.21.15.44:44318) 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05858700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 -- 172.21.15.44:0/2542190252 learned_addr learned my addr 172.21.15.44:0/2542190252 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 -- 172.21.15.44:0/2542190252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 msgr2=0x55fc05858700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05858700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 -- 172.21.15.44:0/2542190252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837ea0 msgr2=0x55fc0584c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837ea0 0x55fc0584c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 -- 172.21.15.44:0/2542190252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc05837cd0 con 0x55fc05837580 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.597+0000 7f48b2a00640 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 0x55fc058401a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009900 tx=0x7f48a4005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a57bc8222d4b8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.598+0000 7f489f400640 1 -- 172.21.15.44:0/2542190252 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4005450 con 0x55fc05837580 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.598+0000 7f489f400640 1 -- 172.21.15.44:0/2542190252 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f48a40055f0 con 0x55fc05837580 2024-04-20T11:28:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.598+0000 7f489f400640 1 -- 172.21.15.44:0/2542190252 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4004920 con 0x55fc05837580 2024-04-20T11:28:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 -- 172.21.15.44:0/2542190252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 msgr2=0x55fc058401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 0x55fc058401a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009900 tx=0x7f48a4005320 comp rx=0 tx=0).stop 2024-04-20T11:28:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 -- 172.21.15.44:0/2542190252 shutdown_connections 2024-04-20T11:28:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05840a80 0x55fc05858700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837ea0 0x55fc0584c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 --2- 172.21.15.44:0/2542190252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05837580 0x55fc058401a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 -- 172.21.15.44:0/2542190252 >> 172.21.15.44:0/2542190252 conn(0x55fc05831cb0 msgr2=0x55fc0582c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 -- 172.21.15.44:0/2542190252 shutdown_connections 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 -- 172.21.15.44:0/2542190252 wait complete. 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 -- start start 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.599+0000 7f48bc65a740 1 Processor -- start 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05837ea0 0x55fc0583b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05840a80 0x55fc058b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41acc00 con 0x55fc05837580 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41ac500 con 0x55fc05837ea0 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41a96f0 con 0x55fc05840a80 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48bc65a740 10 client.0 _ll_register_callbacks cb 0x7f48ac09c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33056/0 (socket says 172.21.15.44:33056) 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48b2a00640 1 -- 172.21.15.44:0/4101480694 learned_addr learned my addr 172.21.15.44:0/4101480694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:25.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48b2000640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05837ea0 0x55fc0583b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.600+0000 7f48b3400640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05840a80 0x55fc058b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2a00640 1 -- 172.21.15.44:0/4101480694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05840a80 msgr2=0x55fc058b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2a00640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05840a80 0x55fc058b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2a00640 1 -- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05837ea0 msgr2=0x55fc0583b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2a00640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05837ea0 0x55fc0583b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2a00640 1 -- 172.21.15.44:0/4101480694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc054aa3f0 con 0x55fc05837580 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2000640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05837ea0 0x55fc0583b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48b2a00640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583ae50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009620 tx=0x7f48a4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6656fdec1948dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f489cc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f489cc00640 1 -- 172.21.15.44:0/4101480694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a402ca60 con 0x55fc05837580 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f489cc00640 1 -- 172.21.15.44:0/4101480694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f48a402cc00 con 0x55fc05837580 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.601+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48a40055f0 con 0x55fc05837580 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f489cc00640 1 -- 172.21.15.44:0/4101480694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f48a4061050 con 0x55fc05837580 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 10 client.16956 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fc055747d0 con 0x55fc05837580 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f489cc00640 1 -- 172.21.15.44:0/4101480694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f48a40055f0 con 0x55fc05837580 2024-04-20T11:28:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f489cc00640 1 -- 172.21.15.44:0/4101480694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f48a4003710 con 0x55fc05837580 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 10 client.16956 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 20 client.16956 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058b76e0 0x55fc058b9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 4 client.16956 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.602+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc0579f8e0 con 0x55fc058b76e0 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.603+0000 7f48b2000640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058b76e0 0x55fc058b9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.603+0000 7f48b2000640 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058b76e0 0x55fc058b9ad0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4eb561439172cca server_cookie=13b36cd1828cfeeb in_seq=0 out_seq=0 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.603+0000 7f489cc00640 10 client.16956 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f489cc00640 1 -- 172.21.15.44:0/4101480694 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc0579f8e0 con 0x55fc058b76e0 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f489cc00640 10 client.16956 handle_command_reply: tid=0 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 client.16956 shutdown 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 msgr2=0x55fc0583ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583ae50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009620 tx=0x7f48a4009650 comp rx=0 tx=0).stop 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 shutdown_connections 2024-04-20T11:28:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55fc058b76e0 0x55fc058b9ad0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc05840a80 0x55fc058b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fc05837ea0 0x55fc0583b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 --2- 172.21.15.44:0/4101480694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc05837580 0x55fc0583ae50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f489cc00640 0 client.16956 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 >> 172.21.15.44:0/4101480694 conn(0x55fc05831cb0 msgr2=0x55fc0583da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 shutdown_connections 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 1 -- 172.21.15.44:0/4101480694 wait complete. 2024-04-20T11:28:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.604+0000 7f48bc65a740 20 client.16956 trim_cache size 0 max 16384 2024-04-20T11:28:25.613 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f48980a3da0 msgr2=0x7f48980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f48980a3da0 0x7f48980a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009660 tx=0x7f48a0005850 comp rx=0 tx=0).stop 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 msgr2=0x7f48b4198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 0x7f48b4198670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00efd0 tx=0x7f48ac00c6a0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 shutdown_connections 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48b410c450 0x7f48b4198670 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48b4103950 0x7f48b419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f48980a3da0 0x7f48980a6260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 --2- 172.21.15.44:0/820817862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48b4103030 0x7f48b419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 >> 172.21.15.44:0/820817862 conn(0x7f48b40fc870 msgr2=0x7f48b40fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 shutdown_connections 2024-04-20T11:28:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:25.605+0000 7f48b3e00640 1 -- 172.21.15.44:0/820817862 wait complete. 2024-04-20T11:28:25.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:25 smithi044 ceph-mon[16855]: pgmap v177: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:25 smithi078 ceph-mon[21464]: pgmap v177: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:25 smithi165 ceph-mon[21975]: pgmap v177: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:26.624 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:28:26.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:28:26.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 -- 172.21.15.44:0/1475259317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104dc0 msgr2=0x7f8b70108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 --2- 172.21.15.44:0/1475259317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104dc0 0x7f8b70108e00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009900 tx=0x7f8b5802bc40 comp rx=0 tx=0).stop 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 -- 172.21.15.44:0/1475259317 shutdown_connections 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 --2- 172.21.15.44:0/1475259317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b701094e0 0x7f8b701110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 --2- 172.21.15.44:0/1475259317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104dc0 0x7f8b70108e00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 --2- 172.21.15.44:0/1475259317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b70104410 0x7f8b701047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 -- 172.21.15.44:0/1475259317 >> 172.21.15.44:0/1475259317 conn(0x7f8b700ffcf0 msgr2=0x7f8b70102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 -- 172.21.15.44:0/1475259317 shutdown_connections 2024-04-20T11:28:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.860+0000 7f8b76800640 1 -- 172.21.15.44:0/1475259317 wait complete. 2024-04-20T11:28:26.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.861+0000 7f8b76800640 1 Processor -- start 2024-04-20T11:28:26.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.861+0000 7f8b76800640 1 -- start start 2024-04-20T11:28:26.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b76800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 0x7f8b701a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b76800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b701094e0 0x7f8b701a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b76800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b7019a9f0 0x7f8b7019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b76800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701146b0 con 0x7f8b70104410 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b76800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70114530 con 0x7f8b7019a9f0 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b76800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70114830 con 0x7f8b701094e0 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 0x7f8b701a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 0x7f8b701a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33080/0 (socket says 172.21.15.44:33080) 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b6f400640 1 -- 172.21.15.44:0/1262828174 learned_addr learned my addr 172.21.15.44:0/1262828174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.862+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b7019a9f0 0x7f8b7019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b701094e0 0x7f8b701a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6f400640 1 -- 172.21.15.44:0/1262828174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b701094e0 msgr2=0x7f8b701a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6f400640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b701094e0 0x7f8b701a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6f400640 1 -- 172.21.15.44:0/1262828174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b7019a9f0 msgr2=0x7f8b7019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6f400640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b7019a9f0 0x7f8b7019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6f400640 1 -- 172.21.15.44:0/1262828174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7019b510 con 0x7f8b70104410 2024-04-20T11:28:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b6f400640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 0x7f8b701a0840 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d950 tx=0x7f8b6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b67e00640 1 -- 172.21.15.44:0/1262828174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60017070 con 0x7f8b70104410 2024-04-20T11:28:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b67e00640 1 -- 172.21.15.44:0/1262828174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b60009c30 con 0x7f8b70104410 2024-04-20T11:28:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b67e00640 1 -- 172.21.15.44:0/1262828174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000f740 con 0x7f8b70104410 2024-04-20T11:28:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.863+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7019b800 con 0x7f8b70104410 2024-04-20T11:28:26.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.864+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b701a7aa0 con 0x7f8b70104410 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.867+0000 7f8b7944a740 1 Processor -- start 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.867+0000 7f8b7944a740 1 -- start start 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.867+0000 7f8b67e00640 1 -- 172.21.15.44:0/1262828174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8b60010a80 con 0x7f8b70104410 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.867+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 0x561f2de9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9b830 0x561f2dea3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b7944a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561f2ddffd20 con 0x561f2dea4410 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b7944a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561f2dda63e0 con 0x561f2de9b830 2024-04-20T11:28:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b7944a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f2ddfbce0 con 0x561f2de9af10 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 0x561f2de9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9b830 0x561f2dea3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 0x561f2de9b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44358/0 (socket says 172.21.15.44:44358) 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 -- 172.21.15.44:0/954229311 learned_addr learned my addr 172.21.15.44:0/954229311 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 -- 172.21.15.44:0/954229311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9b830 msgr2=0x561f2dea3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9b830 0x561f2dea3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 -- 172.21.15.44:0/954229311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 msgr2=0x561f2dea6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.868+0000 7f8b6fe00640 1 -- 172.21.15.44:0/954229311 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2ddc5980 con 0x561f2de9af10 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 0x561f2de9b2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009f70 tx=0x7f8b68010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c200ca067d5866 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b67e00640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b540a3da0 0x7f8b540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b64c00640 1 -- 172.21.15.44:0/954229311 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68016070 con 0x561f2de9af10 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b64c00640 1 -- 172.21.15.44:0/954229311 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b68004d20 con 0x561f2de9af10 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b64c00640 1 -- 172.21.15.44:0/954229311 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b680129b0 con 0x561f2de9af10 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b67e00640 1 -- 172.21.15.44:0/1262828174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8b600d0350 con 0x7f8b70104410 2024-04-20T11:28:26.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.869+0000 7f8b6f400640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b540a3da0 0x7f8b540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 -- 172.21.15.44:0/954229311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 msgr2=0x561f2de9b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b6f400640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b540a3da0 0x7f8b540a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8b600cdbc0 tx=0x7f8b600d3ea0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 0x561f2de9b2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009f70 tx=0x7f8b68010800 comp rx=0 tx=0).stop 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 -- 172.21.15.44:0/954229311 shutdown_connections 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9b830 0x561f2dea3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 --2- 172.21.15.44:0/954229311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9af10 0x561f2de9b2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 -- 172.21.15.44:0/954229311 >> 172.21.15.44:0/954229311 conn(0x561f2de95640 msgr2=0x561f2de95a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:26.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 -- 172.21.15.44:0/954229311 shutdown_connections 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.870+0000 7f8b7944a740 1 -- 172.21.15.44:0/954229311 wait complete. 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.871+0000 7f8b7944a740 1 -- start start 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.871+0000 7f8b7944a740 1 Processor -- start 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.871+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2dea13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.871+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2dea18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b7944a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561f2dea7200 con 0x561f2dea4410 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b7944a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561f2ddb97a0 con 0x561f2de9af10 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b7944a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f2dea7380 con 0x561f2de9b830 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b7944a740 10 client.0 _ll_register_callbacks cb 0x561f2ddbbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2dea18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2dea18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44364/0 (socket says 172.21.15.44:44364) 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2dea13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 -- 172.21.15.44:0/3595834724 learned_addr learned my addr 172.21.15.44:0/3595834724 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 -- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 msgr2=0x561f2dea13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2dea13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 -- 172.21.15.44:0/3595834724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 msgr2=0x561f2dea1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6ea00640 1 -- 172.21.15.44:0/3595834724 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2dbda7e0 con 0x561f2de9b830 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.872+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2dea13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2dea18e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b58035000 tx=0x7f8b5802ddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c947823dc66de07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b5d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b5d600640 1 -- 172.21.15.44:0/3595834724 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58043070 con 0x561f2de9b830 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b68004d20 con 0x561f2de9b830 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b5d600640 1 -- 172.21.15.44:0/3595834724 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8b58005030 con 0x561f2de9b830 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b5d600640 1 -- 172.21.15.44:0/3595834724 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8b5804f050 con 0x561f2de9b830 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b7944a740 10 client.27094 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.873+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561f2dc3af70 con 0x561f2de9b830 2024-04-20T11:28:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b5d600640 1 -- 172.21.15.44:0/3595834724 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b68004d20 con 0x561f2de9b830 2024-04-20T11:28:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b5d600640 1 -- 172.21.15.44:0/3595834724 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f8b580035f0 con 0x561f2de9b830 2024-04-20T11:28:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b7944a740 10 client.27094 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b7944a740 10 client.27094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b7944a740 20 client.27094 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b7944a740 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2deb4710 0x561f2deb6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b7944a740 4 client.27094 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.874+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f2de0d740 con 0x561f2deb4710 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.875+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2deb4710 0x561f2deb6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.875+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2deb4710 0x561f2deb6b30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1596cf4069d1da3 server_cookie=19352cbeff4507b8 in_seq=0 out_seq=0 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.875+0000 7f8b5d600640 10 client.27094 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b5d600640 1 -- 172.21.15.44:0/3595834724 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f2de0d740 con 0x561f2deb4710 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b5d600640 10 client.27094 handle_command_reply: tid=0 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 client.27094 shutdown 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 msgr2=0x561f2dea18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2dea18e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b58035000 tx=0x7f8b5802ddf0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 shutdown_connections 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2deb4710 0x561f2deb6b30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2dea4410 0x561f2dea1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2dea18e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 --2- 172.21.15.44:0/3595834724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2dea13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b5d600640 0 client.27094 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 >> 172.21.15.44:0/3595834724 conn(0x561f2de95640 msgr2=0x561f2de97390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 shutdown_connections 2024-04-20T11:28:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.876+0000 7f8b7944a740 1 -- 172.21.15.44:0/3595834724 wait complete. 2024-04-20T11:28:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.877+0000 7f8b7944a740 20 client.27094 trim_cache size 0 max 16384 2024-04-20T11:28:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 Processor -- start 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 -- start start 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 0x561f2dea3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2deb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2dea4410 0x561f2debc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701aff20 con 0x561f2de9af10 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b1af0 con 0x561f2dea4410 2024-04-20T11:28:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.896+0000 7f8b7944a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b05f0 con 0x561f2de9b830 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.897+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 0x561f2dea3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.897+0000 7f8b6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 0x561f2dea3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33104/0 (socket says 172.21.15.44:33104) 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.897+0000 7f8b6fe00640 1 -- 172.21.15.44:0/2358845912 learned_addr learned my addr 172.21.15.44:0/2358845912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.897+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2deb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.897+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2dea4410 0x561f2debc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6ea00640 1 -- 172.21.15.44:0/2358845912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2dea4410 msgr2=0x561f2debc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2dea4410 0x561f2debc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6ea00640 1 -- 172.21.15.44:0/2358845912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 msgr2=0x561f2dea3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 0x561f2dea3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6ea00640 1 -- 172.21.15.44:0/2358845912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2dad2170 con 0x561f2de9b830 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 0x561f2dea3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2deb00f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006010 tx=0x7f8b58005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2fe7217d22cb0d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b5fe00640 1 -- 172.21.15.44:0/2358845912 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58058070 con 0x561f2de9b830 2024-04-20T11:28:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b5fe00640 1 -- 172.21.15.44:0/2358845912 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b5804c030 con 0x561f2de9b830 2024-04-20T11:28:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.898+0000 7f8b5fe00640 1 -- 172.21.15.44:0/2358845912 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58043040 con 0x561f2de9b830 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 -- 172.21.15.44:0/2358845912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 msgr2=0x561f2deb00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2deb00f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006010 tx=0x7f8b58005e00 comp rx=0 tx=0).stop 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 -- 172.21.15.44:0/2358845912 shutdown_connections 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2dea4410 0x561f2debc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2de9b830 0x561f2deb00f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 --2- 172.21.15.44:0/2358845912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9af10 0x561f2dea3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 -- 172.21.15.44:0/2358845912 >> 172.21.15.44:0/2358845912 conn(0x561f2de95640 msgr2=0x561f2de94650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 -- 172.21.15.44:0/2358845912 shutdown_connections 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 -- 172.21.15.44:0/2358845912 wait complete. 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.899+0000 7f8b7944a740 1 -- start start 2024-04-20T11:28:26.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 Processor -- start 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2de9e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 0x561f2de9e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2dea4410 0x561f2de9ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b13f0 con 0x561f2de9b830 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701b0cf0 con 0x561f2de9af10 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b701adee0 con 0x561f2dea4410 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b7944a740 10 client.0 _ll_register_callbacks cb 0x7f8b58005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 0x561f2de9e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 0x561f2de9e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33112/0 (socket says 172.21.15.44:33112) 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.900+0000 7f8b6ea00640 1 -- 172.21.15.44:0/1790972055 learned_addr learned my addr 172.21.15.44:0/1790972055 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2de9e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2dea4410 0x561f2de9ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6ea00640 1 -- 172.21.15.44:0/1790972055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2dea4410 msgr2=0x561f2de9ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2dea4410 0x561f2de9ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6ea00640 1 -- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 msgr2=0x561f2de9e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2de9e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6ea00640 1 -- 172.21.15.44:0/1790972055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2df19ec0 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2dea4410 0x561f2de9ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b6ea00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 0x561f2de9e810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b5802c980 tx=0x7f8b58004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f19bc8b48ec9670 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b5d600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.901+0000 7f8b5d600640 1 -- 172.21.15.44:0/1790972055 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58058070 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b5d600640 1 -- 172.21.15.44:0/1790972055 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f8b58005240 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5804c030 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b5d600640 1 -- 172.21.15.44:0/1790972055 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8b58043040 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 10 client.16974 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561f2df1b200 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b5d600640 1 -- 172.21.15.44:0/1790972055 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b5804c030 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b5d600640 1 -- 172.21.15.44:0/1790972055 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f8b580344e0 con 0x561f2de9b830 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 10 client.16974 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 20 client.16974 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:26.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2df1b7b0 0x561f2df1dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:26.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 4 client.16974 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:28:26.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.902+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561f2de075b0 con 0x561f2df1b7b0 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.903+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2df1b7b0 0x561f2df1dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.903+0000 7f8b6fe00640 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2df1b7b0 0x561f2df1dba0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4a135bbf53c76b server_cookie=f81425ec053e8ad2 in_seq=0 out_seq=0 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.903+0000 7f8b5d600640 10 client.16974 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.904+0000 7f8b5d600640 1 -- 172.21.15.44:0/1790972055 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561f2de075b0 con 0x561f2df1b7b0 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.904+0000 7f8b5d600640 10 client.16974 handle_command_reply: tid=0 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.904+0000 7f8b7944a740 1 client.16974 shutdown 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 msgr2=0x561f2de9e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 0x561f2de9e810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b5802c980 tx=0x7f8b58004540 comp rx=0 tx=0).stop 2024-04-20T11:28:26.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 shutdown_connections 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561f2df1b7b0 0x561f2df1dba0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f2dea4410 0x561f2de9ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f2de9b830 0x561f2de9e810 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b5d600640 0 client.16974 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 --2- 172.21.15.44:0/1790972055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561f2de9af10 0x561f2de9e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 >> 172.21.15.44:0/1790972055 conn(0x561f2de95640 msgr2=0x561f2dea1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 shutdown_connections 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 1 -- 172.21.15.44:0/1790972055 wait complete. 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.905+0000 7f8b7944a740 20 client.16974 trim_cache size 0 max 16384 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:28:26.907 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:28:26.908 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "cf8b3cc3-5370-424a-8023-39d407e7aaae", 2024-04-20T11:28:26.908 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:28:26.908 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:28:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b540a3da0 msgr2=0x7f8b540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b540a3da0 0x7f8b540a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8b600cdbc0 tx=0x7f8b600d3ea0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 msgr2=0x7f8b701a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 0x7f8b701a0840 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d950 tx=0x7f8b6000de10 comp rx=0 tx=0).stop 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 shutdown_connections 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b7019a9f0 0x7f8b7019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b701094e0 0x7f8b701a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8b540a3da0 0x7f8b540a6260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 --2- 172.21.15.44:0/1262828174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b70104410 0x7f8b701a0840 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 >> 172.21.15.44:0/1262828174 conn(0x7f8b700ffcf0 msgr2=0x7f8b701010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 shutdown_connections 2024-04-20T11:28:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:26.906+0000 7f8b76800640 1 -- 172.21.15.44:0/1262828174 wait complete. 2024-04-20T11:28:26.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:28:26.924 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 -- 172.21.15.44:0/1739537845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 msgr2=0x7f17f010e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 --2- 172.21.15.44:0/1739537845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f010e9e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009900 tx=0x7f17e402bc30 comp rx=0 tx=0).stop 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 -- 172.21.15.44:0/1739537845 shutdown_connections 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 --2- 172.21.15.44:0/1739537845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108f50 0x7f17f01153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 --2- 172.21.15.44:0/1739537845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108590 0x7f17f0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 --2- 172.21.15.44:0/1739537845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f010e9e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 -- 172.21.15.44:0/1739537845 >> 172.21.15.44:0/1739537845 conn(0x7f17f00764a0 msgr2=0x7f17f00788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 -- 172.21.15.44:0/1739537845 shutdown_connections 2024-04-20T11:28:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.137+0000 7f17f8000640 1 -- 172.21.15.44:0/1739537845 wait complete. 2024-04-20T11:28:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.139+0000 7f17f8000640 1 Processor -- start 2024-04-20T11:28:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.139+0000 7f17f8000640 1 -- start start 2024-04-20T11:28:27.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.139+0000 7f17f8000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 0x7f17f019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.139+0000 7f17f8000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108f50 0x7f17f019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f8000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f01a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f8000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0118a20 con 0x7f17f010e600 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f8000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01188a0 con 0x7f17f0108590 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f8000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0118ba0 con 0x7f17f0108f50 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 0x7f17f019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f01a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 0x7f17f019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59454/0 (socket says 172.21.15.44:59454) 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 -- 172.21.15.44:0/3333842781 learned_addr learned my addr 172.21.15.44:0/3333842781 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:27.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f4c00640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108f50 0x7f17f019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 -- 172.21.15.44:0/3333842781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108f50 msgr2=0x7f17f019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108f50 0x7f17f019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 -- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 msgr2=0x7f17f01a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f01a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f5600640 1 -- 172.21.15.44:0/3333842781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f01a3e90 con 0x7f17f0108590 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f6000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f01a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.140+0000 7f17f4c00640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108f50 0x7f17f019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.141+0000 7f17f5600640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 0x7f17f019eea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f17e4005940 tx=0x7f17e402dc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.141+0000 7f17ee000640 1 -- 172.21.15.44:0/3333842781 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4043020 con 0x7f17f0108590 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.141+0000 7f17ee000640 1 -- 172.21.15.44:0/3333842781 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17e4004250 con 0x7f17f0108590 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.141+0000 7f17ee000640 1 -- 172.21.15.44:0/3333842781 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4003950 con 0x7f17f0108590 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.141+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f01a4120 con 0x7f17f0108590 2024-04-20T11:28:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.141+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01aba10 con 0x7f17f0108590 2024-04-20T11:28:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.144+0000 7f17fae14740 1 Processor -- start 2024-04-20T11:28:27.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 -- start start 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17ee000640 1 -- 172.21.15.44:0/3333842781 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f17e4046020 con 0x7f17f0108590 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd4885580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd4885ac0 0x556dd488e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd4890a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556dd47eb150 con 0x556dd488e6a0 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556dd4791370 con 0x556dd4885ac0 2024-04-20T11:28:27.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17fae14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556dd47e7110 con 0x556dd48851a0 2024-04-20T11:28:27.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.145+0000 7f17f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd4890a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd4885580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd4885ac0 0x556dd488e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd4890a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33142/0 (socket says 172.21.15.44:33142) 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 -- 172.21.15.44:0/2485644181 learned_addr learned my addr 172.21.15.44:0/2485644181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 -- 172.21.15.44:0/2485644181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 msgr2=0x556dd4885580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd4885580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 -- 172.21.15.44:0/2485644181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd4885ac0 msgr2=0x556dd488e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd4885ac0 0x556dd488e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 -- 172.21.15.44:0/2485644181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd45c77d0 con 0x556dd488e6a0 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f6000640 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd4885580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.146+0000 7f17f4c00640 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd4890a90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0114f0 tx=0x7f17dc0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebdac56b20726fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17e2600640 1 -- 172.21.15.44:0/2485644181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc004020 con 0x556dd488e6a0 2024-04-20T11:28:27.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17e2600640 1 -- 172.21.15.44:0/2485644181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17dc0041c0 con 0x556dd488e6a0 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17e2600640 1 -- 172.21.15.44:0/2485644181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00f9c0 con 0x556dd488e6a0 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17ee000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17d80a3d20 0x7f17d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17ee000640 1 -- 172.21.15.44:0/3333842781 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17e40f26c0 con 0x7f17f0108590 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17f6000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17d80a3d20 0x7f17d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17fae14740 1 -- 172.21.15.44:0/2485644181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 msgr2=0x556dd4890a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17fae14740 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd4890a90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0114f0 tx=0x7f17dc0119b0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.147+0000 7f17fae14740 1 -- 172.21.15.44:0/2485644181 shutdown_connections 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd4890a90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd4885ac0 0x556dd488e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 --2- 172.21.15.44:0/2485644181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd4885580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 -- 172.21.15.44:0/2485644181 >> 172.21.15.44:0/2485644181 conn(0x556dd487f8d0 msgr2=0x556dd487fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 -- 172.21.15.44:0/2485644181 shutdown_connections 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 -- 172.21.15.44:0/2485644181 wait complete. 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17f6000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17d80a3d20 0x7f17d80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f17e8005fd0 tx=0x7f17e8009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 -- start start 2024-04-20T11:28:27.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.148+0000 7f17fae14740 1 Processor -- start 2024-04-20T11:28:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd488b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd488bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd488e6a0 0x556dd488c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556dd4891490 con 0x556dd4885ac0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556dd47a4740 con 0x556dd488e6a0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556dd4891610 con 0x556dd48851a0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17fae14740 10 client.0 _ll_register_callbacks cb 0x556dd479fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd488bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17f6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd488e6a0 0x556dd488c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd488bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33148/0 (socket says 172.21.15.44:33148) 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17f4c00640 1 -- 172.21.15.44:0/3082647174 learned_addr learned my addr 172.21.15.44:0/3082647174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.149+0000 7f17f5600640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd488b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f4c00640 1 -- 172.21.15.44:0/3082647174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 msgr2=0x556dd488b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f4c00640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd488b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f4c00640 1 -- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd488e6a0 msgr2=0x556dd488c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f4c00640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd488e6a0 0x556dd488c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f4c00640 1 -- 172.21.15.44:0/3082647174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd44fd3f0 con 0x556dd4885ac0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f6000640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd488e6a0 0x556dd488c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17f4c00640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd488bbf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f17dc011eb0 tx=0x7f17dc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d9d9b41960bf70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.150+0000 7f17d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17d6a00640 1 -- 172.21.15.44:0/3082647174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc028020 con 0x556dd4885ac0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc0041c0 con 0x556dd4885ac0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17d6a00640 1 -- 172.21.15.44:0/3082647174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17dc016040 con 0x556dd4885ac0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17d6a00640 1 -- 172.21.15.44:0/3082647174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f17dc033050 con 0x556dd4885ac0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17fae14740 10 client.16986 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556dd44bd0f0 con 0x556dd4885ac0 2024-04-20T11:28:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.151+0000 7f17d6a00640 1 -- 172.21.15.44:0/3082647174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17dc0041c0 con 0x556dd4885ac0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17d6a00640 1 -- 172.21.15.44:0/3082647174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f17dc004b30 con 0x556dd4885ac0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17fae14740 10 client.16986 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17fae14740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17fae14740 20 client.16986 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17fae14740 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd489e9a0 0x556dd48a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17fae14740 4 client.16986 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dd47f8b70 con 0x556dd489e9a0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17f5600640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd489e9a0 0x556dd48a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17f5600640 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd489e9a0 0x556dd48a0d90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622072d1f824539a server_cookie=3da803f84c9a11e0 in_seq=0 out_seq=0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.152+0000 7f17d6a00640 10 client.16986 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.153+0000 7f17d6a00640 1 -- 172.21.15.44:0/3082647174 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556dd47f8b70 con 0x556dd489e9a0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.153+0000 7f17d6a00640 10 client.16986 handle_command_reply: tid=0 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.153+0000 7f17fae14740 1 client.16986 shutdown 2024-04-20T11:28:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.153+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 msgr2=0x556dd488bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.153+0000 7f17fae14740 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd488bbf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f17dc011eb0 tx=0x7f17dc005720 comp rx=0 tx=0).stop 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 shutdown_connections 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd489e9a0 0x556dd48a0d90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd488e6a0 0x556dd488c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd488bbf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 --2- 172.21.15.44:0/3082647174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd48851a0 0x556dd488b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17d6a00640 0 client.16986 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 >> 172.21.15.44:0/3082647174 conn(0x556dd487f8d0 msgr2=0x556dd4881620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 shutdown_connections 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 1 -- 172.21.15.44:0/3082647174 wait complete. 2024-04-20T11:28:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.154+0000 7f17fae14740 20 client.16986 trim_cache size 0 max 16384 2024-04-20T11:28:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 Processor -- start 2024-04-20T11:28:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 -- start start 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd488dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd489a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 0x556dd487e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b4180 con 0x556dd4885ac0 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b5d50 con 0x556dd48851a0 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.174+0000 7f17fae14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b4850 con 0x556dd488e6a0 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd489a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 0x556dd487e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd488dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 0x556dd487e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44404/0 (socket says 172.21.15.44:44404) 2024-04-20T11:28:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd489a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33164/0 (socket says 172.21.15.44:33164) 2024-04-20T11:28:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 -- 172.21.15.44:0/1744229420 learned_addr learned my addr 172.21.15.44:0/1744229420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 -- 172.21.15.44:0/1744229420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 msgr2=0x556dd488dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd488dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 -- 172.21.15.44:0/1744229420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 msgr2=0x556dd489a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.175+0000 7f17f6000640 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd489a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.176+0000 7f17f6000640 1 -- 172.21.15.44:0/1744229420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd4879d10 con 0x556dd488e6a0 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.176+0000 7f17f6000640 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 0x556dd487e740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f17e800fc60 tx=0x7f17e801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e729756a0914ede4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.176+0000 7f17e1c00640 1 -- 172.21.15.44:0/1744229420 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e800fd50 con 0x556dd488e6a0 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.176+0000 7f17e1c00640 1 -- 172.21.15.44:0/1744229420 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17e8004020 con 0x556dd488e6a0 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.176+0000 7f17e1c00640 1 -- 172.21.15.44:0/1744229420 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8029680 con 0x556dd488e6a0 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.177+0000 7f17fae14740 1 -- 172.21.15.44:0/1744229420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 msgr2=0x556dd487e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.177+0000 7f17fae14740 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 0x556dd487e740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f17e800fc60 tx=0x7f17e801a000 comp rx=0 tx=0).stop 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 -- 172.21.15.44:0/1744229420 shutdown_connections 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd488e6a0 0x556dd487e740 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd4885ac0 0x556dd489a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 --2- 172.21.15.44:0/1744229420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd488dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 -- 172.21.15.44:0/1744229420 >> 172.21.15.44:0/1744229420 conn(0x556dd487f8d0 msgr2=0x556dd487e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 -- 172.21.15.44:0/1744229420 shutdown_connections 2024-04-20T11:28:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 -- 172.21.15.44:0/1744229420 wait complete. 2024-04-20T11:28:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 -- start start 2024-04-20T11:28:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.178+0000 7f17fae14740 1 Processor -- start 2024-04-20T11:28:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd4896b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd4885ac0 0x556dd4887f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd48884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b5650 con 0x556dd488e6a0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b4f50 con 0x556dd48851a0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01b2140 con 0x556dd4885ac0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17fae14740 10 client.0 _ll_register_callbacks cb 0x7f17e40f8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17f5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd4896b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17f6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd48884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17f5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd4896b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59490/0 (socket says 172.21.15.44:59490) 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17f5600640 1 -- 172.21.15.44:0/1174836979 learned_addr learned my addr 172.21.15.44:0/1174836979 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.179+0000 7f17f4c00640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd4885ac0 0x556dd4887f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f6000640 1 -- 172.21.15.44:0/1174836979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd4885ac0 msgr2=0x556dd4887f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f6000640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd4885ac0 0x556dd4887f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f6000640 1 -- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 msgr2=0x556dd4896b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f6000640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd4896b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f6000640 1 -- 172.21.15.44:0/1174836979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd4888b60 con 0x556dd488e6a0 2024-04-20T11:28:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f4c00640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd4885ac0 0x556dd4887f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f5600640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd4896b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17f6000640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd48884a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f17e800f790 tx=0x7f17e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9f24d7cbdc40f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17d6a00640 1 -- 172.21.15.44:0/1174836979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80043d0 con 0x556dd488e6a0 2024-04-20T11:28:27.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17d6a00640 1 -- 172.21.15.44:0/1174836979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f17e8004570 con 0x556dd488e6a0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.180+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e8004020 con 0x556dd488e6a0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17d6a00640 1 -- 172.21.15.44:0/1174836979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f17e8003970 con 0x556dd488e6a0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 10 client.16992 fetch_fsmap learned FSMap version 50 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556dd4904d30 con 0x556dd488e6a0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17d6a00640 1 -- 172.21.15.44:0/1174836979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17e8004020 con 0x556dd488e6a0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17d6a00640 1 -- 172.21.15.44:0/1174836979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 50) ==== 8375+0+0 (secure 0 0 0) 0x7f17e8032c90 con 0x556dd488e6a0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 10 client.16992 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 20 client.16992 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd49053c0 0x556dd49077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 4 client.16992 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.181+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556dd47f29e0 con 0x556dd49053c0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.182+0000 7f17f5600640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd49053c0 0x556dd49077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.182+0000 7f17f5600640 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd49053c0 0x556dd49077e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f0cc15ce0c56b9 server_cookie=7c07fced70be6c9f in_seq=0 out_seq=0 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.182+0000 7f17d6a00640 10 client.16992 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.182+0000 7f17d6a00640 1 -- 172.21.15.44:0/1174836979 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556dd47f29e0 con 0x556dd49053c0 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.182+0000 7f17d6a00640 10 client.16992 handle_command_reply: tid=0 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.182+0000 7f17fae14740 1 client.16992 shutdown 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 msgr2=0x556dd48884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd48884a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f17e800f790 tx=0x7f17e8004ab0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 shutdown_connections 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556dd49053c0 0x556dd49077e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556dd488e6a0 0x556dd48884a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd4885ac0 0x556dd4887f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 --2- 172.21.15.44:0/1174836979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556dd48851a0 0x556dd4896b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17d6a00640 0 client.16992 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 >> 172.21.15.44:0/1174836979 conn(0x556dd487f8d0 msgr2=0x556dd488bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 shutdown_connections 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 1 -- 172.21.15.44:0/1174836979 wait complete. 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.183+0000 7f17fae14740 20 client.16992 trim_cache size 0 max 16384 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stdout: "cf8b3cc3-5370-424a-8023-39d407e7aaae": { 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:28:27.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "cf8b3cc3-5370-424a-8023-39d407e7aaae", 2024-04-20T11:28:27.187 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:28:27.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:28:27.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:28:27.187 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:28:27.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17d80a3d20 msgr2=0x7f17d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17d80a3d20 0x7f17d80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f17e8005fd0 tx=0x7f17e8009500 comp rx=0 tx=0).stop 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 msgr2=0x7f17f019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 0x7f17f019eea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f17e4005940 tx=0x7f17e402dc80 comp rx=0 tx=0).stop 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 shutdown_connections 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f17d80a3d20 0x7f17d80a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17f010e600 0x7f17f01a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f0108f50 0x7f17f019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 --2- 172.21.15.44:0/3333842781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0108590 0x7f17f019eea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 >> 172.21.15.44:0/3333842781 conn(0x7f17f00764a0 msgr2=0x7f17f0077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 shutdown_connections 2024-04-20T11:28:27.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:27.184+0000 7f17f8000640 1 -- 172.21.15.44:0/3333842781 wait complete. 2024-04-20T11:28:27.202 INFO:tasks.cephfs.filesystem:scrub status for tag:cf8b3cc3-5370-424a-8023-39d407e7aaae - {'path': '/', 'tag': 'cf8b3cc3-5370-424a-8023-39d407e7aaae', 'options': 'recursive,force'} 2024-04-20T11:28:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:27 smithi078 ceph-mon[21464]: pgmap v178: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:27 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:28:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:27 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:28:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:27 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:28:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:27 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:28:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:27 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:28:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:27 smithi044 ceph-mon[16855]: pgmap v178: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:27 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:28:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:27 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:28:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:27 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:28:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:27 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:28:27.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:27 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:28:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:27 smithi165 ceph-mon[21975]: pgmap v178: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:27 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:28:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:27 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:28:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:27 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:28:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:27 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:28:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:27 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:28:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:29 smithi078 ceph-mon[21464]: pgmap v179: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:29 smithi044 ceph-mon[16855]: pgmap v179: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:29 smithi165 ceph-mon[21975]: pgmap v179: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:30.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:30 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:30.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:30 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:30.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:30 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:31.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:31 smithi078 ceph-mon[21464]: pgmap v180: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:31.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:31 smithi044 ceph-mon[16855]: pgmap v180: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:31 smithi165 ceph-mon[21975]: pgmap v180: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:33.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:33 smithi078 ceph-mon[21464]: pgmap v181: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:33.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:33 smithi044 ceph-mon[16855]: pgmap v181: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:33 smithi165 ceph-mon[21975]: pgmap v181: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:35.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:35 smithi078 ceph-mon[21464]: pgmap v182: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:35.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:35 smithi044 ceph-mon[16855]: pgmap v182: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:35.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:35 smithi165 ceph-mon[21975]: pgmap v182: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:37.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:37 smithi078 ceph-mon[21464]: pgmap v183: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:37.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:37 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:37.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:37 smithi044 ceph-mon[16855]: pgmap v183: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:37.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:37 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:37 smithi165 ceph-mon[21975]: pgmap v183: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:28:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:37 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:39.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:39 smithi078 ceph-mon[21464]: pgmap v184: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:39 smithi044 ceph-mon[16855]: pgmap v184: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:39 smithi165 ceph-mon[21975]: pgmap v184: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:28:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:41 smithi078 ceph-mon[21464]: pgmap v185: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:41 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:41 smithi044 ceph-mon[16855]: pgmap v185: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:41 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:41 smithi165 ceph-mon[21975]: pgmap v185: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:28:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:41 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:43.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:43 smithi078 ceph-mon[21464]: pgmap v186: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:28:43.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:43 smithi044 ceph-mon[16855]: pgmap v186: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:28:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:43 smithi165 ceph-mon[21975]: pgmap v186: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:28:45.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:45 smithi078 ceph-mon[21464]: pgmap v187: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:45.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:45 smithi044 ceph-mon[16855]: pgmap v187: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:45 smithi165 ceph-mon[21975]: pgmap v187: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:47.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:47 smithi078 ceph-mon[21464]: pgmap v188: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:47.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:47 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:47 smithi044 ceph-mon[16855]: pgmap v188: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:47 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:47 smithi165 ceph-mon[21975]: pgmap v188: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:47 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:28:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:49 smithi078 ceph-mon[21464]: pgmap v189: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-20T11:28:49.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:49 smithi044 ceph-mon[16855]: pgmap v189: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-20T11:28:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:49 smithi165 ceph-mon[21975]: pgmap v189: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-20T11:28:51.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:50 smithi078 ceph-mon[21464]: pgmap v190: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-20T11:28:51.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:50 smithi044 ceph-mon[16855]: pgmap v190: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-20T11:28:51.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:50 smithi165 ceph-mon[21975]: pgmap v190: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-20T11:28:53.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:53 smithi078 ceph-mon[21464]: pgmap v191: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-20T11:28:53.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:53 smithi044 ceph-mon[16855]: pgmap v191: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-20T11:28:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:53 smithi165 ceph-mon[21975]: pgmap v191: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-20T11:28:54.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:54 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:28:54.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:54 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:28:54.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:54 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:28:55.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:55 smithi078 ceph-mon[21464]: pgmap v192: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:55.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:55 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:28:55.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:55 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:28:55.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:55 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:28:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:55 smithi044 ceph-mon[16855]: pgmap v192: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:55 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:28:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:55 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:28:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:55 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:28:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:55 smithi165 ceph-mon[21975]: pgmap v192: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:55 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:28:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:55 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:28:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:55 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:28:57.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:28:57.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:28:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 -- 172.21.15.44:0/1830484527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc8104410 msgr2=0x7febc81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 --2- 172.21.15.44:0/1830484527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc8104410 0x7febc81047f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7febb8009900 tx=0x7febb802bc30 comp rx=0 tx=0).stop 2024-04-20T11:28:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 -- 172.21.15.44:0/1830484527 shutdown_connections 2024-04-20T11:28:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 --2- 172.21.15.44:0/1830484527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febc81094e0 0x7febc81110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 --2- 172.21.15.44:0/1830484527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febc8104dc0 0x7febc8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 --2- 172.21.15.44:0/1830484527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc8104410 0x7febc81047f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 -- 172.21.15.44:0/1830484527 >> 172.21.15.44:0/1830484527 conn(0x7febc80ffcf0 msgr2=0x7febc8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 -- 172.21.15.44:0/1830484527 shutdown_connections 2024-04-20T11:28:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.427+0000 7febcd200640 1 -- 172.21.15.44:0/1830484527 wait complete. 2024-04-20T11:28:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.428+0000 7febcd200640 1 Processor -- start 2024-04-20T11:28:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 -- start start 2024-04-20T11:28:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febc8104410 0x7febc81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febc81094e0 0x7febc81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 0x7febc819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febc8114660 con 0x7febc819aa40 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febc81144e0 con 0x7febc81094e0 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febcd200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febc81147e0 con 0x7febc8104410 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febc6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 0x7febc819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febc6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 0x7febc819aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53766/0 (socket says 172.21.15.44:53766) 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febc6800640 1 -- 172.21.15.44:0/517124330 learned_addr learned my addr 172.21.15.44:0/517124330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.429+0000 7febc5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febc81094e0 0x7febc81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc5e00640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febc8104410 0x7febc81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc6800640 1 -- 172.21.15.44:0/517124330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febc8104410 msgr2=0x7febc81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc6800640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febc8104410 0x7febc81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc6800640 1 -- 172.21.15.44:0/517124330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febc81094e0 msgr2=0x7febc81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc6800640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febc81094e0 0x7febc81a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc6800640 1 -- 172.21.15.44:0/517124330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc819b560 con 0x7febc819aa40 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.430+0000 7febc6800640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 0x7febc819aef0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7febc000eaa0 tx=0x7febc000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.431+0000 7febbea00640 1 -- 172.21.15.44:0/517124330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000cdb0 con 0x7febc819aa40 2024-04-20T11:28:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.431+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc819b850 con 0x7febc819aa40 2024-04-20T11:28:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.431+0000 7febbea00640 1 -- 172.21.15.44:0/517124330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febc0004b20 con 0x7febc819aa40 2024-04-20T11:28:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.431+0000 7febbea00640 1 -- 172.21.15.44:0/517124330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0011760 con 0x7febc819aa40 2024-04-20T11:28:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.431+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc81a7aa0 con 0x7febc819aa40 2024-04-20T11:28:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.435+0000 7febcffb6740 1 Processor -- start 2024-04-20T11:28:57.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febbea00640 1 -- 172.21.15.44:0/517124330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7febc0011900 con 0x7febc819aa40 2024-04-20T11:28:57.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 -- start start 2024-04-20T11:28:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c308f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3098160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c309aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566c2ff5150 con 0x5566c308f1a0 2024-04-20T11:28:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566c2f9b370 con 0x5566c30986a0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febcffb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566c2ff1110 con 0x5566c308fac0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.436+0000 7febc5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c308f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c308f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53770/0 (socket says 172.21.15.44:53770) 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5400640 1 -- 172.21.15.44:0/299823861 learned_addr learned my addr 172.21.15.44:0/299823861 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3098160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5400640 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c309aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 -- 172.21.15.44:0/299823861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 msgr2=0x5566c309aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c309aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 -- 172.21.15.44:0/299823861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 msgr2=0x5566c308f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c308f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 -- 172.21.15.44:0/299823861 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c2dd17d0 con 0x5566c308fac0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5400640 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c308f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febbea00640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febac0a3da0 0x7febac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.437+0000 7febc5e00640 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3098160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7febb8000c00 tx=0x7febb802c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f74b5d43101092 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.438+0000 7febbea00640 1 -- 172.21.15.44:0/517124330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7febc00d0580 con 0x7febc819aa40 2024-04-20T11:28:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.438+0000 7febc5400640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febac0a3da0 0x7febac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.438+0000 7febb7400640 1 -- 172.21.15.44:0/299823861 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb802c800 con 0x5566c308fac0 2024-04-20T11:28:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.438+0000 7febb7400640 1 -- 172.21.15.44:0/299823861 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febb8004040 con 0x5566c308fac0 2024-04-20T11:28:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.438+0000 7febb7400640 1 -- 172.21.15.44:0/299823861 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb80046c0 con 0x5566c308fac0 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 -- 172.21.15.44:0/299823861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 msgr2=0x5566c3098160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3098160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7febb8000c00 tx=0x7febb802c610 comp rx=0 tx=0).stop 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febc5400640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febac0a3da0 0x7febac0a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5566c308aa60 tx=0x7febb0009570 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 -- 172.21.15.44:0/299823861 shutdown_connections 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c309aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3098160 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 --2- 172.21.15.44:0/299823861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c308f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 -- 172.21.15.44:0/299823861 >> 172.21.15.44:0/299823861 conn(0x5566c30898d0 msgr2=0x5566c3089cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 -- 172.21.15.44:0/299823861 shutdown_connections 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 -- 172.21.15.44:0/299823861 wait complete. 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.439+0000 7febcffb6740 1 -- start start 2024-04-20T11:28:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 Processor -- start 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3095700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308fac0 0x5566c3095c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c30986a0 0x5566c3096180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566c309b430 con 0x5566c308f1a0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566c2fae740 con 0x5566c308fac0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566c309b5b0 con 0x5566c30986a0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febcffb6740 10 client.0 _ll_register_callbacks cb 0x5566c2fa9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.440+0000 7febc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3095700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3095700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53772/0 (socket says 172.21.15.44:53772) 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 -- 172.21.15.44:0/1200511921 learned_addr learned my addr 172.21.15.44:0/1200511921 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5400640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308fac0 0x5566c3095c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc6800640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c30986a0 0x5566c3096180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 -- 172.21.15.44:0/1200511921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c30986a0 msgr2=0x5566c3096180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c30986a0 0x5566c3096180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 -- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308fac0 msgr2=0x5566c3095c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308fac0 0x5566c3095c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 -- 172.21.15.44:0/1200511921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c2d073f0 con 0x5566c308f1a0 2024-04-20T11:28:57.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febc5e00640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3095700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7febb8005360 tx=0x7febb8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506a243c8f3ac812 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.441+0000 7febabe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:57.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febabe00640 1 -- 172.21.15.44:0/1200511921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb804f070 con 0x5566c308f1a0 2024-04-20T11:28:57.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febabe00640 1 -- 172.21.15.44:0/1200511921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7febb802cbd0 con 0x5566c308f1a0 2024-04-20T11:28:57.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb8004040 con 0x5566c308f1a0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febabe00640 1 -- 172.21.15.44:0/1200511921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7febb805b050 con 0x5566c308f1a0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febcffb6740 10 client.17004 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5566c2cc70f0 con 0x5566c308f1a0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.442+0000 7febabe00640 1 -- 172.21.15.44:0/1200511921 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febb8004040 con 0x5566c308f1a0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febabe00640 1 -- 172.21.15.44:0/1200511921 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7febb8004990 con 0x5566c308f1a0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febcffb6740 10 client.17004 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febcffb6740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febcffb6740 20 client.17004 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febcffb6740 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c30a8880 0x5566c30aac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febcffb6740 4 client.17004 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c3002b70 con 0x5566c30a8880 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.443+0000 7febc5400640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c30a8880 0x5566c30aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.444+0000 7febc5400640 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c30a8880 0x5566c30aac70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19316eff944e93f server_cookie=6dfea00a178119f6 in_seq=0 out_seq=0 2024-04-20T11:28:57.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.444+0000 7febabe00640 10 client.17004 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febabe00640 1 -- 172.21.15.44:0/1200511921 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566c3002b70 con 0x5566c30a8880 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febabe00640 10 client.17004 handle_command_reply: tid=0 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 client.17004 shutdown 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 msgr2=0x5566c3095700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3095700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7febb8005360 tx=0x7febb8005de0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 shutdown_connections 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c30a8880 0x5566c30aac70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c30986a0 0x5566c3096180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308fac0 0x5566c3095c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febabe00640 0 client.17004 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 --2- 172.21.15.44:0/1200511921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3095700 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 >> 172.21.15.44:0/1200511921 conn(0x5566c30898d0 msgr2=0x5566c308b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 shutdown_connections 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 1 -- 172.21.15.44:0/1200511921 wait complete. 2024-04-20T11:28:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.445+0000 7febcffb6740 20 client.17004 trim_cache size 0 max 16384 2024-04-20T11:28:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.464+0000 7febcffb6740 1 Processor -- start 2024-04-20T11:28:57.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 -- start start 2024-04-20T11:28:57.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3097e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3094660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c30a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febc81afd20 con 0x5566c308f1a0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febc81b1890 con 0x5566c30986a0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febcffb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febc81b03f0 con 0x5566c308fac0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3097e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febc5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3097e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53778/0 (socket says 172.21.15.44:53778) 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febc5e00640 1 -- 172.21.15.44:0/2861632228 learned_addr learned my addr 172.21.15.44:0/2861632228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febc5400640 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3094660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.465+0000 7febc6800640 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c30a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc5e00640 1 -- 172.21.15.44:0/2861632228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 msgr2=0x5566c3094660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc5e00640 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3094660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc5e00640 1 -- 172.21.15.44:0/2861632228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 msgr2=0x5566c30a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc5e00640 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c30a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc5e00640 1 -- 172.21.15.44:0/2861632228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c308f8f0 con 0x5566c308f1a0 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc6800640 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c30a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:57.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febc5e00640 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3097e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7febb802c170 tx=0x7febb8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e615dd02326da6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febb6a00640 1 -- 172.21.15.44:0/2861632228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb804f070 con 0x5566c308f1a0 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febb6a00640 1 -- 172.21.15.44:0/2861632228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febb8005180 con 0x5566c308f1a0 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.466+0000 7febb6a00640 1 -- 172.21.15.44:0/2861632228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8034740 con 0x5566c308f1a0 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 -- 172.21.15.44:0/2861632228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 msgr2=0x5566c3097e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3097e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7febb802c170 tx=0x7febb8005fd0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 -- 172.21.15.44:0/2861632228 shutdown_connections 2024-04-20T11:28:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c30986a0 0x5566c30a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3094660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 --2- 172.21.15.44:0/2861632228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c308f1a0 0x5566c3097e30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 -- 172.21.15.44:0/2861632228 >> 172.21.15.44:0/2861632228 conn(0x5566c30898d0 msgr2=0x5566c308b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 -- 172.21.15.44:0/2861632228 shutdown_connections 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.467+0000 7febcffb6740 1 -- 172.21.15.44:0/2861632228 wait complete. 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 -- start start 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 Processor -- start 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 0x5566c3092ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3093020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c30986a0 0x5566c310db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febc81b1190 con 0x5566c30986a0 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febc81b0ac0 con 0x5566c308f1a0 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febc81add10 con 0x5566c308fac0 2024-04-20T11:28:57.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.468+0000 7febcffb6740 10 client.0 _ll_register_callbacks cb 0x7febb0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 0x5566c3092ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c30986a0 0x5566c310db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3093020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 0x5566c3092ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:57706/0 (socket says 172.21.15.44:57706) 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 -- 172.21.15.44:0/183776421 learned_addr learned my addr 172.21.15.44:0/183776421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 -- 172.21.15.44:0/183776421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 msgr2=0x5566c3093020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3093020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 -- 172.21.15.44:0/183776421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c30986a0 msgr2=0x5566c310db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c30986a0 0x5566c310db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 -- 172.21.15.44:0/183776421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c310e0d0 con 0x5566c308f1a0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5400640 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3093020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febc5e00640 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 0x5566c3092ae0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7febb8009690 tx=0x7febb80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275e7685b7f53ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febabe00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.469+0000 7febabe00640 1 -- 172.21.15.44:0/183776421 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8058040 con 0x5566c308f1a0 2024-04-20T11:28:57.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb8005180 con 0x5566c308f1a0 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febabe00640 1 -- 172.21.15.44:0/183776421 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7febb802c9d0 con 0x5566c308f1a0 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febabe00640 1 -- 172.21.15.44:0/183776421 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7febb806d050 con 0x5566c308f1a0 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 10 client.27137 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5566c310f390 con 0x5566c308f1a0 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febabe00640 1 -- 172.21.15.44:0/183776421 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febb8005180 con 0x5566c308f1a0 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febabe00640 1 -- 172.21.15.44:0/183776421 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7febb8054440 con 0x5566c308f1a0 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 10 client.27137 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 10 client.27137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 20 client.27137 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c310fa20 0x5566c3111e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 4 client.27137 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.470+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566c2ffc9e0 con 0x5566c310fa20 2024-04-20T11:28:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.471+0000 7febc5400640 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c310fa20 0x5566c3111e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.471+0000 7febc5400640 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c310fa20 0x5566c3111e40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc80610caec989e6 server_cookie=bc6a5f2e65771d66 in_seq=0 out_seq=0 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.471+0000 7febabe00640 10 client.27137 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.471+0000 7febabe00640 1 -- 172.21.15.44:0/183776421 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5566c2ffc9e0 con 0x5566c310fa20 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.471+0000 7febabe00640 10 client.27137 handle_command_reply: tid=0 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.471+0000 7febcffb6740 1 client.27137 shutdown 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 msgr2=0x5566c3092ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 0x5566c3092ae0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7febb8009690 tx=0x7febb80057f0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 shutdown_connections 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5566c310fa20 0x5566c3111e40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566c30986a0 0x5566c310db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566c308fac0 0x5566c3093020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febabe00640 0 client.27137 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 --2- 172.21.15.44:0/183776421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566c308f1a0 0x5566c3092ae0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 >> 172.21.15.44:0/183776421 conn(0x5566c30898d0 msgr2=0x5566c3088030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 shutdown_connections 2024-04-20T11:28:57.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 1 -- 172.21.15.44:0/183776421 wait complete. 2024-04-20T11:28:57.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.472+0000 7febcffb6740 20 client.27137 trim_cache size 0 max 16384 2024-04-20T11:28:57.477 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:28:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:28:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:28:57.477 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febac0a3da0 msgr2=0x7febac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febac0a3da0 0x7febac0a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5566c308aa60 tx=0x7febb0009570 comp rx=0 tx=0).stop 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 msgr2=0x7febc819aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 0x7febc819aef0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7febc000eaa0 tx=0x7febc000ef60 comp rx=0 tx=0).stop 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 shutdown_connections 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febac0a3da0 0x7febac0a6260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc819aa40 0x7febc819aef0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febc81094e0 0x7febc81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 --2- 172.21.15.44:0/517124330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febc8104410 0x7febc81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 >> 172.21.15.44:0/517124330 conn(0x7febc80ffcf0 msgr2=0x7febc81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 shutdown_connections 2024-04-20T11:28:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.473+0000 7febcd200640 1 -- 172.21.15.44:0/517124330 wait complete. 2024-04-20T11:28:57.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:28:57.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:28:57.559 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:57 smithi044 ceph-mon[16855]: pgmap v193: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:57.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:57 smithi078 ceph-mon[21464]: pgmap v193: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 -- 172.21.15.44:0/1310852242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b8109290 msgr2=0x7f76b8110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 --2- 172.21.15.44:0/1310852242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b8109290 0x7f76b8110e60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009ca0 tx=0x7f76b002c0f0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 -- 172.21.15.44:0/1310852242 shutdown_connections 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 --2- 172.21.15.44:0/1310852242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b8109290 0x7f76b8110e60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 --2- 172.21.15.44:0/1310852242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8104b70 0x7f76b8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 --2- 172.21.15.44:0/1310852242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b81041c0 0x7f76b81045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 -- 172.21.15.44:0/1310852242 >> 172.21.15.44:0/1310852242 conn(0x7f76b80ffa00 msgr2=0x7f76b8101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 -- 172.21.15.44:0/1310852242 shutdown_connections 2024-04-20T11:28:57.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.695+0000 7f76be400640 1 -- 172.21.15.44:0/1310852242 wait complete. 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.696+0000 7f76be400640 1 Processor -- start 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.696+0000 7f76be400640 1 -- start start 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76be400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b81041c0 0x7f76b8076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76be400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8109290 0x7f76b80769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76be400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b8076ef0 0x7f76b81aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76be400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8114460 con 0x7f76b81041c0 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76be400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81142e0 con 0x7f76b8076ef0 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76be400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81145e0 con 0x7f76b8109290 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76bd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b81041c0 0x7f76b8076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76bd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b81041c0 0x7f76b8076470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53820/0 (socket says 172.21.15.44:53820) 2024-04-20T11:28:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76bd000640 1 -- 172.21.15.44:0/858297045 learned_addr learned my addr 172.21.15.44:0/858297045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76bda00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b8076ef0 0x7f76b81aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.697+0000 7f76b7e00640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8109290 0x7f76b80769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76bda00640 1 -- 172.21.15.44:0/858297045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8109290 msgr2=0x7f76b80769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76bda00640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8109290 0x7f76b80769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76bda00640 1 -- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b81041c0 msgr2=0x7f76b8076470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76bda00640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b81041c0 0x7f76b8076470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76bda00640 1 -- 172.21.15.44:0/858297045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81ab430 con 0x7f76b8076ef0 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76b7e00640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8109290 0x7f76b80769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.698+0000 7f76bda00640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b8076ef0 0x7f76b81aaef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009bf0 tx=0x7f76b0032570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.699+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0039040 con 0x7f76b8076ef0 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.699+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f76b0044070 con 0x7f76b8076ef0 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.699+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0004330 con 0x7f76b8076ef0 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.699+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81ab6c0 con 0x7f76b8076ef0 2024-04-20T11:28:57.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.699+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81abb30 con 0x7f76b8076ef0 2024-04-20T11:28:57.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.700+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b8105690 con 0x7f76b8076ef0 2024-04-20T11:28:57.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.703+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f76b0003970 con 0x7f76b8076ef0 2024-04-20T11:28:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.704+0000 7f76b5600640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f76a00a3da0 0x7f76a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.704+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f76b00f3120 con 0x7f76b8076ef0 2024-04-20T11:28:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.704+0000 7f76bd000640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f76a00a3da0 0x7f76a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:57.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.705+0000 7f76bd000640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f76a00a3da0 0x7f76a00a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0095f0 tx=0x7f76ac005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:57.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.706+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76b00bf970 con 0x7f76b8076ef0 2024-04-20T11:28:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:57 smithi165 ceph-mon[21975]: pgmap v193: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-20T11:28:57.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.954+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76b810c550 con 0x7f76b8076ef0 2024-04-20T11:28:57.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.956+0000 7f76b5600640 1 -- 172.21.15.44:0/858297045 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11942 (secure 0 0 0) 0x7f76b00bf270 con 0x7f76b8076ef0 2024-04-20T11:28:57.957 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:28:57.957 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:28:45.784953+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24550,24494,24596,24506,24562]},"id":1}]} 2024-04-20T11:28:57.957 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 54 2024-04-20T11:28:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f76a00a3da0 msgr2=0x7f76a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f76a00a3da0 0x7f76a00a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0095f0 tx=0x7f76ac005f90 comp rx=0 tx=0).stop 2024-04-20T11:28:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b8076ef0 msgr2=0x7f76b81aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b8076ef0 0x7f76b81aaef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009bf0 tx=0x7f76b0032570 comp rx=0 tx=0).stop 2024-04-20T11:28:57.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 shutdown_connections 2024-04-20T11:28:57.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76b8076ef0 0x7f76b81aaef0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76b8109290 0x7f76b80769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f76a00a3da0 0x7f76a00a6260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 --2- 172.21.15.44:0/858297045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b81041c0 0x7f76b8076470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 >> 172.21.15.44:0/858297045 conn(0x7f76b80ffa00 msgr2=0x7f76b8100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 shutdown_connections 2024-04-20T11:28:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:57.959+0000 7f76be400640 1 -- 172.21.15.44:0/858297045 wait complete. 2024-04-20T11:28:57.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 -- 172.21.15.44:0/3875602855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa4108ea0 msgr2=0x7f3aa41152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/3875602855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa4108ea0 0x7f3aa41152c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009ca0 tx=0x7f3a9c02c180 comp rx=0 tx=0).stop 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 -- 172.21.15.44:0/3875602855 shutdown_connections 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/3875602855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa4108ea0 0x7f3aa41152c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/3875602855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108500 0x7f3aa4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/3875602855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa410d3d0 0x7f3aa410d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 -- 172.21.15.44:0/3875602855 >> 172.21.15.44:0/3875602855 conn(0x7f3aa4077d80 msgr2=0x7f3aa4106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 -- 172.21.15.44:0/3875602855 shutdown_connections 2024-04-20T11:28:58.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.200+0000 7f3aa3e00640 1 -- 172.21.15.44:0/3875602855 wait complete. 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.201+0000 7f3aa3e00640 1 Processor -- start 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.201+0000 7f3aa3e00640 1 -- start start 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa4108500 0x7f3aa419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 0x7f3aa419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa410d3d0 0x7f3aa41a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4118950 con 0x7f3aa4108ea0 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41187d0 con 0x7f3aa4108500 2024-04-20T11:28:58.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4118ad0 con 0x7f3aa410d3d0 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3a9a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 0x7f3aa419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa4108500 0x7f3aa419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3a9a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 0x7f3aa419f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53838/0 (socket says 172.21.15.44:53838) 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3a9a000640 1 -- 172.21.15.44:0/4234534947 learned_addr learned my addr 172.21.15.44:0/4234534947 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.202+0000 7f3aa3400640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa410d3d0 0x7f3aa41a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa2a00640 1 -- 172.21.15.44:0/4234534947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa410d3d0 msgr2=0x7f3aa41a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa2a00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa410d3d0 0x7f3aa41a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa2a00640 1 -- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 msgr2=0x7f3aa419f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa2a00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 0x7f3aa419f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa2a00640 1 -- 172.21.15.44:0/4234534947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41a3d80 con 0x7f3aa4108500 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3a9a000640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 0x7f3aa419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa2a00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa4108500 0x7f3aa419ed90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400d6d0 tx=0x7f3a9400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3a9be00640 1 -- 172.21.15.44:0/4234534947 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94017070 con 0x7f3aa4108500 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3a9be00640 1 -- 172.21.15.44:0/4234534947 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a94009820 con 0x7f3aa4108500 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41a4070 con 0x7f3aa4108500 2024-04-20T11:28:58.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.203+0000 7f3a9be00640 1 -- 172.21.15.44:0/4234534947 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94004e00 con 0x7f3aa4108500 2024-04-20T11:28:58.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.204+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41ab900 con 0x7f3aa4108500 2024-04-20T11:28:58.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.208+0000 7f3aac99d740 1 Processor -- start 2024-04-20T11:28:58.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.208+0000 7f3aac99d740 1 -- start start 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3a9be00640 1 -- 172.21.15.44:0/4234534947 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3a940099d0 con 0x7f3aa4108500 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce0f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce18130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce1aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aac99d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c4cd74050 con 0x559c4ce0fa90 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aac99d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c4cd1a490 con 0x559c4ce0f170 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aac99d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c4cd70010 con 0x559c4ce18670 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aa2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce18130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce0f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce0f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:57730/0 (socket says 172.21.15.44:57730) 2024-04-20T11:28:58.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3aa3400640 1 -- 172.21.15.44:0/4036648972 learned_addr learned my addr 172.21.15.44:0/4036648972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.209+0000 7f3a9a000640 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce1aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa3400640 1 -- 172.21.15.44:0/4036648972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 msgr2=0x559c4ce1aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa3400640 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce1aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa3400640 1 -- 172.21.15.44:0/4036648972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 msgr2=0x559c4ce18130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa3400640 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce18130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa3400640 1 -- 172.21.15.44:0/4036648972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4cd22070 con 0x559c4ce0f170 2024-04-20T11:28:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa2a00640 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce18130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3aa3400640 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce0f550 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009ca0 tx=0x7f3a9c009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29003bd0f8e341b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.210+0000 7f3a93e00640 1 -- 172.21.15.44:0/4036648972 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c043070 con 0x559c4ce0f170 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3a9be00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3a880a3da0 0x7f3a880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3a93e00640 1 -- 172.21.15.44:0/4036648972 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a9c032720 con 0x559c4ce0f170 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3a93e00640 1 -- 172.21.15.44:0/4036648972 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c039040 con 0x559c4ce0f170 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3a9be00640 1 -- 172.21.15.44:0/4234534947 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3a940d0120 con 0x7f3aa4108500 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3aa2a00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3a880a3da0 0x7f3a880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3aac99d740 1 -- 172.21.15.44:0/4036648972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 msgr2=0x559c4ce0f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3aac99d740 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce0f550 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009ca0 tx=0x7f3a9c009900 comp rx=0 tx=0).stop 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.211+0000 7f3aac99d740 1 -- 172.21.15.44:0/4036648972 shutdown_connections 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce1aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce18130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 --2- 172.21.15.44:0/4036648972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce0f550 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 -- 172.21.15.44:0/4036648972 >> 172.21.15.44:0/4036648972 conn(0x559c4ce098a0 msgr2=0x559c4ce09c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aa2a00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3a880a3da0 0x7f3a880a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559c4ce0ec40 tx=0x7f3a940d3bd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 -- 172.21.15.44:0/4036648972 shutdown_connections 2024-04-20T11:28:58.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 -- 172.21.15.44:0/4036648972 wait complete. 2024-04-20T11:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 -- start start 2024-04-20T11:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.212+0000 7f3aac99d740 1 Processor -- start 2024-04-20T11:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce15710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce15c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce16190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c4ce1b400 con 0x559c4ce0fa90 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c4cd2d860 con 0x559c4ce0f170 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c4ce1b580 con 0x559c4ce18670 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aac99d740 10 client.0 _ll_register_callbacks cb 0x559c4ccfbfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3a9a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce15c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce15710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce15710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:57746/0 (socket says 172.21.15.44:57746) 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3a9a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce15c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53866/0 (socket says 172.21.15.44:53866) 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aa3400640 1 -- 172.21.15.44:0/375360802 learned_addr learned my addr 172.21.15.44:0/375360802 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.213+0000 7f3aa3400640 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce16190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a9a000640 1 -- 172.21.15.44:0/375360802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 msgr2=0x559c4ce16190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a9a000640 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce16190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a9a000640 1 -- 172.21.15.44:0/375360802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 msgr2=0x559c4ce15710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a9a000640 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce15710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a9a000640 1 -- 172.21.15.44:0/375360802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4cd31950 con 0x559c4ce0fa90 2024-04-20T11:28:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a9a000640 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce15c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009620 tx=0x7f3a8c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61393867e268f063 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a90c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a90c00640 1 -- 172.21.15.44:0/375360802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c012110 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3a90c00640 1 -- 172.21.15.44:0/375360802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3a8c004050 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.214+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9c032720 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3a90c00640 1 -- 172.21.15.44:0/375360802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3a8c0122b0 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 10 client.17028 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559c4ccff820 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3a90c00640 1 -- 172.21.15.44:0/375360802 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a9c032720 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3a90c00640 1 -- 172.21.15.44:0/375360802 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f3a8c024020 con 0x559c4ce0fa90 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 10 client.17028 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 10 client.17028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 20 client.17028 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce28640 0x559c4ce2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 4 client.17028 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.215+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4cd81a70 con 0x559c4ce28640 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.216+0000 7f3aa3400640 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce28640 0x559c4ce2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.216+0000 7f3aa3400640 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce28640 0x559c4ce2aa70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7077dc6e5873ff0d server_cookie=b6736cbb4e04290a in_seq=0 out_seq=0 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.216+0000 7f3a90c00640 10 client.17028 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3a90c00640 1 -- 172.21.15.44:0/375360802 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c4cd81a70 con 0x559c4ce28640 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3a90c00640 10 client.17028 handle_command_reply: tid=0 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 client.17028 shutdown 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 msgr2=0x559c4ce15c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce15c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009620 tx=0x7f3a8c00f790 comp rx=0 tx=0).stop 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 shutdown_connections 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce16190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0fa90 0x559c4ce15c50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce28640 0x559c4ce2aa70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 --2- 172.21.15.44:0/375360802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0f170 0x559c4ce15710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 >> 172.21.15.44:0/375360802 conn(0x559c4ce098a0 msgr2=0x559c4ce0b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3a90c00640 0 client.17028 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 shutdown_connections 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 1 -- 172.21.15.44:0/375360802 wait complete. 2024-04-20T11:28:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.217+0000 7f3aac99d740 20 client.17028 trim_cache size 0 max 16384 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.232+0000 7f3aac99d740 1 Processor -- start 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.232+0000 7f3aac99d740 1 -- start start 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0fa90 0x559c4ce0b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce15de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aac99d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b3fa0 con 0x559c4ce0f170 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aac99d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b5b10 con 0x559c4ce0fa90 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aac99d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b4670 con 0x559c4ce18670 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53876/0 (socket says 172.21.15.44:53876) 2024-04-20T11:28:58.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aa3400640 1 -- 172.21.15.44:0/2530901510 learned_addr learned my addr 172.21.15.44:0/2530901510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3a9a000640 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0fa90 0x559c4ce0b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3aa3400640 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce15de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3a9a000640 1 -- 172.21.15.44:0/2530901510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 msgr2=0x559c4ce15de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3a9a000640 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce15de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3a9a000640 1 -- 172.21.15.44:0/2530901510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 msgr2=0x559c4ce242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3a9a000640 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.233+0000 7f3a9a000640 1 -- 172.21.15.44:0/2530901510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4ca863f0 con 0x559c4ce0fa90 2024-04-20T11:28:58.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.234+0000 7f3a9a000640 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0fa90 0x559c4ce0b560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000c00 tx=0x7f3a8c026460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee840ffc065e59a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.234+0000 7f3a93400640 1 -- 172.21.15.44:0/2530901510 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c026810 con 0x559c4ce0fa90 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.234+0000 7f3a93400640 1 -- 172.21.15.44:0/2530901510 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a8c0269b0 con 0x559c4ce0fa90 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.234+0000 7f3a93400640 1 -- 172.21.15.44:0/2530901510 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c003900 con 0x559c4ce0fa90 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 -- 172.21.15.44:0/2530901510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0fa90 msgr2=0x559c4ce0b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0fa90 0x559c4ce0b560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000c00 tx=0x7f3a8c026460 comp rx=0 tx=0).stop 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 -- 172.21.15.44:0/2530901510 shutdown_connections 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce18670 0x559c4ce15de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce0fa90 0x559c4ce0b560 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 --2- 172.21.15.44:0/2530901510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 -- 172.21.15.44:0/2530901510 >> 172.21.15.44:0/2530901510 conn(0x559c4ce098a0 msgr2=0x559c4ce156d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 -- 172.21.15.44:0/2530901510 shutdown_connections 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 -- 172.21.15.44:0/2530901510 wait complete. 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 -- start start 2024-04-20T11:28:58.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.235+0000 7f3aac99d740 1 Processor -- start 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 0x559c4ce1d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce18670 0x559c4ce1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b5410 con 0x559c4ce0f170 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b4d40 con 0x559c4ce18670 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41b1f70 con 0x559c4ce0fa90 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aac99d740 10 client.0 _ll_register_callbacks cb 0x7f3a8c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3a9a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 0x559c4ce1d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce20a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53884/0 (socket says 172.21.15.44:53884) 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3aa3400640 1 -- 172.21.15.44:0/1279444055 learned_addr learned my addr 172.21.15.44:0/1279444055 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.236+0000 7f3a9a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 0x559c4ce1d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45136/0 (socket says 172.21.15.44:45136) 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3aa3400640 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce18670 0x559c4ce1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a9a000640 1 -- 172.21.15.44:0/1279444055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce18670 msgr2=0x559c4ce1dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a9a000640 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce18670 0x559c4ce1dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a9a000640 1 -- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 msgr2=0x559c4ce20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a9a000640 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a9a000640 1 -- 172.21.15.44:0/1279444055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4cb507d0 con 0x559c4ce0fa90 2024-04-20T11:28:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a9a000640 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 0x559c4ce1d7d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x559c4ce27430 tx=0x7f3a8c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfdaf49d0079de5e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a90c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:58.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.237+0000 7f3a90c00640 1 -- 172.21.15.44:0/1279444055 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c026810 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8c0269b0 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3a90c00640 1 -- 172.21.15.44:0/1279444055 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f3a8c02c070 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3a90c00640 1 -- 172.21.15.44:0/1279444055 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3a8c037050 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 10 client.27160 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559c4ce8f870 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3a90c00640 1 -- 172.21.15.44:0/1279444055 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a8c0269b0 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3a90c00640 1 -- 172.21.15.44:0/1279444055 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f3a8c024030 con 0x559c4ce0fa90 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 10 client.27160 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 10 client.27160 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 20 client.27160 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.238+0000 7f3aac99d740 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce8fdd0 0x559c4ce921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.239+0000 7f3aac99d740 4 client.27160 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.239+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c4cd7b8e0 con 0x559c4ce8fdd0 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.239+0000 7f3aa3400640 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce8fdd0 0x559c4ce921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.239+0000 7f3aa3400640 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce8fdd0 0x559c4ce921c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=afe5b823e5a46806 server_cookie=8ba8b945b5967963 in_seq=0 out_seq=0 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.239+0000 7f3a90c00640 10 client.27160 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3a90c00640 1 -- 172.21.15.44:0/1279444055 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c4cd7b8e0 con 0x559c4ce8fdd0 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3a90c00640 10 client.27160 handle_command_reply: tid=0 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 client.27160 shutdown 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 msgr2=0x559c4ce1d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 0x559c4ce1d7d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x559c4ce27430 tx=0x7f3a8c005270 comp rx=0 tx=0).stop 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 shutdown_connections 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559c4ce8fdd0 0x559c4ce921c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c4ce18670 0x559c4ce1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c4ce0fa90 0x559c4ce1d7d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 --2- 172.21.15.44:0/1279444055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c4ce0f170 0x559c4ce20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3a90c00640 0 client.27160 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:28:58.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 >> 172.21.15.44:0/1279444055 conn(0x559c4ce098a0 msgr2=0x559c4ce0b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 shutdown_connections 2024-04-20T11:28:58.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 1 -- 172.21.15.44:0/1279444055 wait complete. 2024-04-20T11:28:58.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.240+0000 7f3aac99d740 20 client.27160 trim_cache size 0 max 16384 2024-04-20T11:28:58.246 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3a880a3da0 msgr2=0x7f3a880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3a880a3da0 0x7f3a880a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559c4ce0ec40 tx=0x7f3a940d3bd0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa4108500 msgr2=0x7f3aa419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa4108500 0x7f3aa419ed90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400d6d0 tx=0x7f3a9400db90 comp rx=0 tx=0).stop 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 shutdown_connections 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3aa410d3d0 0x7f3aa41a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3a880a3da0 0x7f3a880a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3aa4108ea0 0x7f3aa419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 --2- 172.21.15.44:0/4234534947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aa4108500 0x7f3aa419ed90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 >> 172.21.15.44:0/4234534947 conn(0x7f3aa4077d80 msgr2=0x7f3aa4078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 shutdown_connections 2024-04-20T11:28:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.241+0000 7f3aa3e00640 1 -- 172.21.15.44:0/4234534947 wait complete. 2024-04-20T11:28:58.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:28:58.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 -- 172.21.15.44:0/2593155882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed4102fc0 msgr2=0x7efed41033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 --2- 172.21.15.44:0/2593155882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed4102fc0 0x7efed41033a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efec4009ca0 tx=0x7efec402c190 comp rx=0 tx=0).stop 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 -- 172.21.15.44:0/2593155882 shutdown_connections 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 --2- 172.21.15.44:0/2593155882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efed410c3e0 0x7efed410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 --2- 172.21.15.44:0/2593155882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed41038e0 0x7efed410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 --2- 172.21.15.44:0/2593155882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed4102fc0 0x7efed41033a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 -- 172.21.15.44:0/2593155882 >> 172.21.15.44:0/2593155882 conn(0x7efed40fc7e0 msgr2=0x7efed40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 -- 172.21.15.44:0/2593155882 shutdown_connections 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.469+0000 7efed3200640 1 -- 172.21.15.44:0/2593155882 wait complete. 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.470+0000 7efed3200640 1 Processor -- start 2024-04-20T11:28:58.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.470+0000 7efed3200640 1 -- start start 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.470+0000 7efed3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed4102fc0 0x7efed41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.470+0000 7efed3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 0x7efed41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efed410c3e0 0x7efed419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efed4112300 con 0x7efed41038e0 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efed4112180 con 0x7efed4102fc0 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efed4112480 con 0x7efed410c3e0 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 0x7efed41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed4102fc0 0x7efed41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 0x7efed41a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42644/0 (socket says 172.21.15.44:42644) 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1400640 1 -- 172.21.15.44:0/974398369 learned_addr learned my addr 172.21.15.44:0/974398369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed2800640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efed410c3e0 0x7efed419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1e00640 1 -- 172.21.15.44:0/974398369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efed410c3e0 msgr2=0x7efed419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1e00640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efed410c3e0 0x7efed419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1e00640 1 -- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 msgr2=0x7efed41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1e00640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 0x7efed41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1e00640 1 -- 172.21.15.44:0/974398369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed419afd0 con 0x7efed4102fc0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.471+0000 7efed1400640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 0x7efed41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.472+0000 7efed1e00640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed4102fc0 0x7efed41a0690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efec402c6d0 tx=0x7efec4034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.472+0000 7efecaa00640 1 -- 172.21.15.44:0/974398369 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4039040 con 0x7efed4102fc0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.472+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed419b260 con 0x7efed4102fc0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.472+0000 7efecaa00640 1 -- 172.21.15.44:0/974398369 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efec4044070 con 0x7efed4102fc0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.472+0000 7efecaa00640 1 -- 172.21.15.44:0/974398369 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec40042a0 con 0x7efed4102fc0 2024-04-20T11:28:58.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.472+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41a74e0 con 0x7efed4102fc0 2024-04-20T11:28:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.475+0000 7efedb863740 1 Processor -- start 2024-04-20T11:28:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.475+0000 7efecaa00640 1 -- 172.21.15.44:0/974398369 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efec4032d20 con 0x7efed4102fc0 2024-04-20T11:28:58.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.475+0000 7efedb863740 1 -- start start 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efedb863740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x559594197550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efedb863740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x5595941a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efedb863740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5595941a0670 0x5595941a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efedb863740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595940fc050 con 0x559594197a90 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efedb863740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595940a2490 con 0x559594197170 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efedb863740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5595940f8010 con 0x5595941a0670 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efed1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x5595941a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efed2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x559594197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efed2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x559594197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37034/0 (socket says 172.21.15.44:37034) 2024-04-20T11:28:58.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efed2800640 1 -- 172.21.15.44:0/2691631814 learned_addr learned my addr 172.21.15.44:0/2691631814 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.476+0000 7efed1400640 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5595941a0670 0x5595941a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed2800640 1 -- 172.21.15.44:0/2691631814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5595941a0670 msgr2=0x5595941a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed2800640 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5595941a0670 0x5595941a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed2800640 1 -- 172.21.15.44:0/2691631814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 msgr2=0x5595941a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed2800640 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x5595941a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed2800640 1 -- 172.21.15.44:0/2691631814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595940aa070 con 0x559594197170 2024-04-20T11:28:58.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed2800640 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x559594197550 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efecc009c10 tx=0x7efecc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff6261d32aa8ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efecaa00640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efeb80a3d20 0x7efeb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efebf000640 1 -- 172.21.15.44:0/2691631814 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc004230 con 0x559594197170 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efebf000640 1 -- 172.21.15.44:0/2691631814 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efecc0043d0 con 0x559594197170 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efebf000640 1 -- 172.21.15.44:0/2691631814 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc019820 con 0x559594197170 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efecaa00640 1 -- 172.21.15.44:0/974398369 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efec40f29b0 con 0x7efed4102fc0 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.477+0000 7efed1e00640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efeb80a3d20 0x7efeb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efed1e00640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efeb80a3d20 0x7efeb80a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x559594196c40 tx=0x7efec40f5f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 -- 172.21.15.44:0/2691631814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 msgr2=0x559594197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x559594197550 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efecc009c10 tx=0x7efecc011810 comp rx=0 tx=0).stop 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 -- 172.21.15.44:0/2691631814 shutdown_connections 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5595941a0670 0x5595941a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x5595941a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 --2- 172.21.15.44:0/2691631814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x559594197550 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 -- 172.21.15.44:0/2691631814 >> 172.21.15.44:0/2691631814 conn(0x5595941918a0 msgr2=0x559594191c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 -- 172.21.15.44:0/2691631814 shutdown_connections 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.478+0000 7efedb863740 1 -- 172.21.15.44:0/2691631814 wait complete. 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 -- start start 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 Processor -- start 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197170 0x55959419d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x55959419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x55959419e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595941a3460 con 0x559594197a90 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595940b5860 con 0x5595941a0670 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5595941a35e0 con 0x559594197170 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.479+0000 7efedb863740 10 client.0 _ll_register_callbacks cb 0x559594083fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x55959419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x55959419dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42668/0 (socket says 172.21.15.44:42668) 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 -- 172.21.15.44:0/3251805249 learned_addr learned my addr 172.21.15.44:0/3251805249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed2800640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197170 0x55959419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed2800640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x55959419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 -- 172.21.15.44:0/3251805249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197170 msgr2=0x55959419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197170 0x55959419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 -- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 msgr2=0x55959419e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x55959419e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 -- 172.21.15.44:0/3251805249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595940b9950 con 0x559594197a90 2024-04-20T11:28:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed2800640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197170 0x55959419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efed1400640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x55959419dc40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efec0012a10 tx=0x7efec0012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ed2fe478137345 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.480+0000 7efeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efeb3400640 1 -- 172.21.15.44:0/3251805249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec000fa40 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efecc0043d0 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efeb3400640 1 -- 172.21.15.44:0/3251805249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efec000fbe0 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efeb3400640 1 -- 172.21.15.44:0/3251805249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7efec000fde0 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efedb863740 10 client.17046 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559594087820 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.481+0000 7efeb3400640 1 -- 172.21.15.44:0/3251805249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efecc0043d0 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efeb3400640 1 -- 172.21.15.44:0/3251805249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7efec0024020 con 0x559594197a90 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efedb863740 10 client.17046 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efedb863740 10 client.17046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:28:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efedb863740 20 client.17046 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efedb863740 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595941b0710 0x5595941b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efedb863740 4 client.17046 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559594109a70 con 0x5595941b0710 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.482+0000 7efed2800640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595941b0710 0x5595941b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.483+0000 7efed2800640 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595941b0710 0x5595941b2b50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93b9a47a32d69dfc server_cookie=3255e0e67ee9ad0f in_seq=0 out_seq=0 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.483+0000 7efeb3400640 10 client.17046 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.483+0000 7efeb3400640 1 -- 172.21.15.44:0/3251805249 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559594109a70 con 0x5595941b0710 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.483+0000 7efeb3400640 10 client.17046 handle_command_reply: tid=0 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 client.17046 shutdown 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 msgr2=0x55959419dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x55959419dc40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efec0012a10 tx=0x7efec0012ed0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 shutdown_connections 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x55959419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595941b0710 0x5595941b2b50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197a90 0x55959419dc40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 --2- 172.21.15.44:0/3251805249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197170 0x55959419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efeb3400640 0 client.17046 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 >> 172.21.15.44:0/3251805249 conn(0x5595941918a0 msgr2=0x5595941935f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 shutdown_connections 2024-04-20T11:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 1 -- 172.21.15.44:0/3251805249 wait complete. 2024-04-20T11:28:58.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.484+0000 7efedb863740 20 client.17046 trim_cache size 0 max 16384 2024-04-20T11:28:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.499+0000 7efedb863740 1 Processor -- start 2024-04-20T11:28:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.499+0000 7efedb863740 1 -- start start 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efedb863740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x55959419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efedb863740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941ac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efedb863740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595941a0670 0x5595941907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efedb863740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efed41afc50 con 0x5595941a0670 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efedb863740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efed41b17c0 con 0x559594197170 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efedb863740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efed41b0320 con 0x559594197a90 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941ac320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41138/0 (socket says 172.21.15.44:41138) 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed1400640 1 -- 172.21.15.44:0/3381184464 learned_addr learned my addr 172.21.15.44:0/3381184464 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed2800640 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x55959419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed1400640 1 -- 172.21.15.44:0/3381184464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 msgr2=0x55959419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed1400640 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x55959419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed1400640 1 -- 172.21.15.44:0/3381184464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595941a0670 msgr2=0x5595941907e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.500+0000 7efed2800640 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595941a0670 0x5595941907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efed1400640 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595941a0670 0x5595941907e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efed1400640 1 -- 172.21.15.44:0/3381184464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559593e0e3f0 con 0x559594197a90 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efed2800640 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595941a0670 0x5595941907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efed1400640 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941ac320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efec0012420 tx=0x7efec000fc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d2b5125d90ca907 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efebe600640 1 -- 172.21.15.44:0/3381184464 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00264e0 con 0x559594197a90 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efebe600640 1 -- 172.21.15.44:0/3381184464 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efec002c030 con 0x559594197a90 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.501+0000 7efebe600640 1 -- 172.21.15.44:0/3381184464 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00035f0 con 0x559594197a90 2024-04-20T11:28:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 -- 172.21.15.44:0/3381184464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 msgr2=0x5595941ac320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941ac320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efec0012420 tx=0x7efec000fc80 comp rx=0 tx=0).stop 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 -- 172.21.15.44:0/3381184464 shutdown_connections 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595941a0670 0x5595941907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941ac320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 --2- 172.21.15.44:0/3381184464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559594197170 0x55959419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 -- 172.21.15.44:0/3381184464 >> 172.21.15.44:0/3381184464 conn(0x5595941918a0 msgr2=0x5595941900d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 -- 172.21.15.44:0/3381184464 shutdown_connections 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.502+0000 7efedb863740 1 -- 172.21.15.44:0/3381184464 wait complete. 2024-04-20T11:28:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 -- start start 2024-04-20T11:28:58.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 Processor -- start 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197170 0x5595941a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x5595941a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efed41b10c0 con 0x559594197170 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efed41b09f0 con 0x5595941a0670 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efed41adc20 con 0x559594197a90 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.503+0000 7efedb863740 10 client.0 _ll_register_callbacks cb 0x7efec0012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197170 0x5595941a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197170 0x5595941a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42692/0 (socket says 172.21.15.44:42692) 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed2800640 1 -- 172.21.15.44:0/2168748430 learned_addr learned my addr 172.21.15.44:0/2168748430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed2800640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x5595941a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 -- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 msgr2=0x5595941a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x5595941a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 -- 172.21.15.44:0/2168748430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197170 msgr2=0x5595941a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197170 0x5595941a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 -- 172.21.15.44:0/2168748430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559593ed87d0 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed2800640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x5595941a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efed1400640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941a4890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x559594191540 tx=0x7efec000fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923e72793aa3a52d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.504+0000 7efeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efeb3400640 1 -- 172.21.15.44:0/2168748430 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0003800 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec002c030 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efeb3400640 1 -- 172.21.15.44:0/2168748430 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7efec00039a0 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efeb3400640 1 -- 172.21.15.44:0/2168748430 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7efec003e050 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efedb863740 10 client.27178 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559594216e70 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.505+0000 7efeb3400640 1 -- 172.21.15.44:0/2168748430 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efec002c030 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efeb3400640 1 -- 172.21.15.44:0/2168748430 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7efec0024040 con 0x559594197a90 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efedb863740 10 client.27178 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efedb863740 10 client.27178 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efedb863740 20 client.27178 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efedb863740 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595942174a0 0x5595942198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efedb863740 4 client.27178 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595941038e0 con 0x5595942174a0 2024-04-20T11:28:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.506+0000 7efed2800640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595942174a0 0x5595942198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efed2800640 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595942174a0 0x5595942198c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75f8b32e5d965832 server_cookie=596c51a6c7197199 in_seq=0 out_seq=0 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efeb3400640 10 client.27178 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efeb3400640 1 -- 172.21.15.44:0/2168748430 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595941038e0 con 0x5595942174a0 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efeb3400640 10 client.27178 handle_command_reply: tid=0 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 client.27178 shutdown 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 msgr2=0x5595941a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941a4890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x559594191540 tx=0x7efec000fa40 comp rx=0 tx=0).stop 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 shutdown_connections 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595941a0670 0x5595941a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559594197a90 0x5595941a4890 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x5595942174a0 0x5595942198c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 --2- 172.21.15.44:0/2168748430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559594197170 0x5595941a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 >> 172.21.15.44:0/2168748430 conn(0x5595941918a0 msgr2=0x55959419df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efeb3400640 0 client.27178 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 shutdown_connections 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 1 -- 172.21.15.44:0/2168748430 wait complete. 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.507+0000 7efedb863740 20 client.27178 trim_cache size 0 max 16384 2024-04-20T11:28:58.511 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efeb80a3d20 msgr2=0x7efeb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efeb80a3d20 0x7efeb80a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x559594196c40 tx=0x7efec40f5f90 comp rx=0 tx=0).stop 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed4102fc0 msgr2=0x7efed41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed4102fc0 0x7efed41a0690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efec402c6d0 tx=0x7efec4034670 comp rx=0 tx=0).stop 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 shutdown_connections 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efed410c3e0 0x7efed419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efeb80a3d20 0x7efeb80a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed41038e0 0x7efed41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 --2- 172.21.15.44:0/974398369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed4102fc0 0x7efed41a0690 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 >> 172.21.15.44:0/974398369 conn(0x7efed40fc7e0 msgr2=0x7efed4100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 shutdown_connections 2024-04-20T11:28:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.508+0000 7efed3200640 1 -- 172.21.15.44:0/974398369 wait complete. 2024-04-20T11:28:58.525 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/858297045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:28:58.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:28:58.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:58 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/858297045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:28:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:58 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/858297045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:28:58.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 -- 172.21.15.44:0/1906635115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104dc0 msgr2=0x7f788c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 --2- 172.21.15.44:0/1906635115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104dc0 0x7f788c108e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402bc30 comp rx=0 tx=0).stop 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 -- 172.21.15.44:0/1906635115 shutdown_connections 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 --2- 172.21.15.44:0/1906635115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788c1094e0 0x7f788c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 --2- 172.21.15.44:0/1906635115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104dc0 0x7f788c108e00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 --2- 172.21.15.44:0/1906635115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f788c104410 0x7f788c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 -- 172.21.15.44:0/1906635115 >> 172.21.15.44:0/1906635115 conn(0x7f788c0ffcf0 msgr2=0x7f788c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 -- 172.21.15.44:0/1906635115 shutdown_connections 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.734+0000 7f7891200640 1 -- 172.21.15.44:0/1906635115 wait complete. 2024-04-20T11:28:58.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.735+0000 7f7891200640 1 Processor -- start 2024-04-20T11:28:58.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.735+0000 7f7891200640 1 -- start start 2024-04-20T11:28:58.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7891200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 0x7f788c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7891200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f788c1094e0 0x7f788c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7891200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788c19a9c0 0x7f788c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7891200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f788c114660 con 0x7f788c19a9c0 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7891200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1144e0 con 0x7f788c1094e0 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7891200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1147e0 con 0x7f788c104410 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7889e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 0x7f788c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7889e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 0x7f788c1a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41160/0 (socket says 172.21.15.44:41160) 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7889e00640 1 -- 172.21.15.44:0/94187350 learned_addr learned my addr 172.21.15.44:0/94187350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f7881400640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f788c1094e0 0x7f788c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.736+0000 7f788a800640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788c19a9c0 0x7f788c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7889e00640 1 -- 172.21.15.44:0/94187350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f788c1094e0 msgr2=0x7f788c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7889e00640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f788c1094e0 0x7f788c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7889e00640 1 -- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788c19a9c0 msgr2=0x7f788c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7889e00640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788c19a9c0 0x7f788c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7889e00640 1 -- 172.21.15.44:0/94187350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c19b4e0 con 0x7f788c104410 2024-04-20T11:28:58.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7889e00640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 0x7f788c1a0810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f787c00db80 tx=0x7f787c00b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7883400640 1 -- 172.21.15.44:0/94187350 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c017070 con 0x7f788c104410 2024-04-20T11:28:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7883400640 1 -- 172.21.15.44:0/94187350 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f787c003b70 con 0x7f788c104410 2024-04-20T11:28:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7883400640 1 -- 172.21.15.44:0/94187350 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c0106c0 con 0x7f788c104410 2024-04-20T11:28:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c19b7d0 con 0x7f788c104410 2024-04-20T11:28:58.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.737+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c1a79d0 con 0x7f788c104410 2024-04-20T11:28:58.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.741+0000 7f7893f7a740 1 Processor -- start 2024-04-20T11:28:58.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.741+0000 7f7893f7a740 1 -- start start 2024-04-20T11:28:58.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.741+0000 7f7883400640 1 -- 172.21.15.44:0/94187350 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f787c004020 con 0x7f788c104410 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522e38d0 0x5570522ebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522ec4b0 0x5570522ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7893f7a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557052248050 con 0x5570522e38d0 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7893f7a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570521ee490 con 0x5570522ec4b0 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7893f7a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557052244010 con 0x5570522e2fb0 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f788a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7889e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522e38d0 0x5570522ebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f7881400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522ec4b0 0x5570522ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f788a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41174/0 (socket says 172.21.15.44:41174) 2024-04-20T11:28:58.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.742+0000 7f788a800640 1 -- 172.21.15.44:0/3712298572 learned_addr learned my addr 172.21.15.44:0/3712298572 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f788a800640 1 -- 172.21.15.44:0/3712298572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522ec4b0 msgr2=0x5570522ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f788a800640 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522ec4b0 0x5570522ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f788a800640 1 -- 172.21.15.44:0/3712298572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522e38d0 msgr2=0x5570522ebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f788a800640 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522e38d0 0x5570522ebf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f788a800640 1 -- 172.21.15.44:0/3712298572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570521f6070 con 0x5570522e2fb0 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f788a800640 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e3390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7884009c20 tx=0x7f7884011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de29716aed07fe4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f787b400640 1 -- 172.21.15.44:0/3712298572 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788401e070 con 0x5570522e2fb0 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f787b400640 1 -- 172.21.15.44:0/3712298572 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7884019b30 con 0x5570522e2fb0 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.743+0000 7f787b400640 1 -- 172.21.15.44:0/3712298572 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788400eae0 con 0x5570522e2fb0 2024-04-20T11:28:58.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7883400640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78700a3da0 0x7f78700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7883400640 1 -- 172.21.15.44:0/94187350 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f787c0d0390 con 0x7f788c104410 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7889e00640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78700a3da0 0x7f78700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 -- 172.21.15.44:0/3712298572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 msgr2=0x5570522e3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e3390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7884009c20 tx=0x7f7884011810 comp rx=0 tx=0).stop 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 -- 172.21.15.44:0/3712298572 shutdown_connections 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522ec4b0 0x5570522ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522e38d0 0x5570522ebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3712298572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e3390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 -- 172.21.15.44:0/3712298572 >> 172.21.15.44:0/3712298572 conn(0x5570522dd6e0 msgr2=0x5570522ddac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.744+0000 7f7893f7a740 1 -- 172.21.15.44:0/3712298572 shutdown_connections 2024-04-20T11:28:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.745+0000 7f7893f7a740 1 -- 172.21.15.44:0/3712298572 wait complete. 2024-04-20T11:28:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.745+0000 7f7889e00640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78700a3da0 0x7f78700a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f787c0cdc00 tx=0x7f787c098040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.745+0000 7f7893f7a740 1 -- start start 2024-04-20T11:28:58.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.745+0000 7f7893f7a740 1 Processor -- start 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5570522ef2a0 con 0x5570522ec4b0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557052201860 con 0x5570522e2fb0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5570522ef420 con 0x5570522e38d0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7893f7a740 10 client.0 _ll_register_callbacks cb 0x5570521cffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7881400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7881400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522e9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41178/0 (socket says 172.21.15.44:41178) 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f7881400640 1 -- 172.21.15.44:0/3249052330 learned_addr learned my addr 172.21.15.44:0/3249052330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f788a800640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.746+0000 7f788a800640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7881400640 1 -- 172.21.15.44:0/3249052330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 msgr2=0x5570522e9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7881400640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522e9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7881400640 1 -- 172.21.15.44:0/3249052330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 msgr2=0x5570522e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7881400640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7881400640 1 -- 172.21.15.44:0/3249052330 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557052205950 con 0x5570522e38d0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f788a800640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7881400640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522e9ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402de70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac040c7bcbc9319 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f786fe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f786fe00640 1 -- 172.21.15.44:0/3249052330 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874044070 con 0x5570522e38d0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f786fe00640 1 -- 172.21.15.44:0/3249052330 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7874004310 con 0x5570522e38d0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.747+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7884019b30 con 0x5570522e38d0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f786fe00640 1 -- 172.21.15.44:0/3249052330 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7874050050 con 0x5570522e38d0 2024-04-20T11:28:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f7893f7a740 10 client.27190 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5570521d3820 con 0x5570522e38d0 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f786fe00640 1 -- 172.21.15.44:0/3249052330 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7884019b30 con 0x5570522e38d0 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f786fe00640 1 -- 172.21.15.44:0/3249052330 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f78740035f0 con 0x5570522e38d0 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f7893f7a740 10 client.27190 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.748+0000 7f7893f7a740 10 client.27190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.749+0000 7f7893f7a740 20 client.27190 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.749+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5570522fc4d0 0x5570522fe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.749+0000 7f7893f7a740 4 client.27190 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.749+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557052255a70 con 0x5570522fc4d0 2024-04-20T11:28:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.749+0000 7f788a800640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5570522fc4d0 0x5570522fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.750+0000 7f788a800640 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5570522fc4d0 0x5570522fe910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9deef97df0c205f7 server_cookie=4a71a407abca93a0 in_seq=0 out_seq=0 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.750+0000 7f786fe00640 10 client.27190 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.750+0000 7f786fe00640 1 -- 172.21.15.44:0/3249052330 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557052255a70 con 0x5570522fc4d0 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.750+0000 7f786fe00640 10 client.27190 handle_command_reply: tid=0 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.750+0000 7f7893f7a740 1 client.27190 shutdown 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 msgr2=0x5570522e9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522e9ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402de70 comp rx=0 tx=0).stop 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 shutdown_connections 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x5570522fc4d0 0x5570522fe910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522e9ab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 --2- 172.21.15.44:0/3249052330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522e9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f786fe00640 0 client.27190 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:58.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 >> 172.21.15.44:0/3249052330 conn(0x5570522dd6e0 msgr2=0x5570522df430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 shutdown_connections 2024-04-20T11:28:58.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 1 -- 172.21.15.44:0/3249052330 wait complete. 2024-04-20T11:28:58.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.751+0000 7f7893f7a740 20 client.27190 trim_cache size 0 max 16384 2024-04-20T11:28:58.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 Processor -- start 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 -- start start 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1afeb0 con 0x5570522ec4b0 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b1a20 con 0x5570522e2fb0 2024-04-20T11:28:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.767+0000 7f7893f7a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b0580 con 0x5570522e38d0 2024-04-20T11:28:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f788a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41182/0 (socket says 172.21.15.44:41182) 2024-04-20T11:28:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f788a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522ebca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37116/0 (socket says 172.21.15.44:37116) 2024-04-20T11:28:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 -- 172.21.15.44:0/2371984791 learned_addr learned my addr 172.21.15.44:0/2371984791 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f788a800640 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522dc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 -- 172.21.15.44:0/2371984791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 msgr2=0x5570522ebca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522ebca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 -- 172.21.15.44:0/2371984791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 msgr2=0x5570522dc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522dc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.768+0000 7f7881400640 1 -- 172.21.15.44:0/2371984791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557051f5a3f0 con 0x5570522e38d0 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.769+0000 7f7881400640 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522f8160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f787402bc70 tx=0x7f7874005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c5ba3571729eda server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.769+0000 7f787aa00640 1 -- 172.21.15.44:0/2371984791 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787404d030 con 0x5570522e38d0 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.769+0000 7f787aa00640 1 -- 172.21.15.44:0/2371984791 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f78740035f0 con 0x5570522e38d0 2024-04-20T11:28:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.769+0000 7f787aa00640 1 -- 172.21.15.44:0/2371984791 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874044040 con 0x5570522e38d0 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.769+0000 7f7893f7a740 1 -- 172.21.15.44:0/2371984791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 msgr2=0x5570522f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.769+0000 7f7893f7a740 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522f8160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f787402bc70 tx=0x7f7874005c00 comp rx=0 tx=0).stop 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 -- 172.21.15.44:0/2371984791 shutdown_connections 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522dc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e38d0 0x5570522f8160 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 --2- 172.21.15.44:0/2371984791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e2fb0 0x5570522ebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 -- 172.21.15.44:0/2371984791 >> 172.21.15.44:0/2371984791 conn(0x5570522dd6e0 msgr2=0x5570522dbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 -- 172.21.15.44:0/2371984791 shutdown_connections 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 -- 172.21.15.44:0/2371984791 wait complete. 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 -- start start 2024-04-20T11:28:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.770+0000 7f7893f7a740 1 Processor -- start 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e38d0 0x5570522e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b1320 con 0x5570522ec4b0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b0c50 con 0x5570522e38d0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1adea0 con 0x5570522e2fb0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7893f7a740 10 client.0 _ll_register_callbacks cb 0x7f787402db00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f788a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f788a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42742/0 (socket says 172.21.15.44:42742) 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f788a800640 1 -- 172.21.15.44:0/430422238 learned_addr learned my addr 172.21.15.44:0/430422238 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f7881400640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e38d0 0x5570522e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.771+0000 7f788a800640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f788a800640 1 -- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 msgr2=0x5570522e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f788a800640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f788a800640 1 -- 172.21.15.44:0/430422238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e38d0 msgr2=0x5570522e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f788a800640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e38d0 0x5570522e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f788a800640 1 -- 172.21.15.44:0/430422238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570520247d0 con 0x5570522ec4b0 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f7881400640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e38d0 0x5570522e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f788a800640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e6f60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7884009a30 tx=0x7f7884014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece17960da4d7e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f786fe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f786fe00640 1 -- 172.21.15.44:0/430422238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788401e040 con 0x5570522ec4b0 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f786fe00640 1 -- 172.21.15.44:0/430422238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7884005330 con 0x5570522ec4b0 2024-04-20T11:28:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.772+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78740035f0 con 0x5570522ec4b0 2024-04-20T11:28:58.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f786fe00640 1 -- 172.21.15.44:0/430422238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7884005530 con 0x5570522ec4b0 2024-04-20T11:28:58.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 10 client.17067 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:58.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557052362dd0 con 0x5570522ec4b0 2024-04-20T11:28:58.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f786fe00640 1 -- 172.21.15.44:0/430422238 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f78740035f0 con 0x5570522ec4b0 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f786fe00640 1 -- 172.21.15.44:0/430422238 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f7884024040 con 0x5570522ec4b0 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 10 client.17067 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 20 client.17067 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557052363300 0x5570523656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 4 client.17067 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.773+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55705224f8e0 con 0x557052363300 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.774+0000 7f7881400640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557052363300 0x5570523656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.774+0000 7f7881400640 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557052363300 0x5570523656f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47c23b782294e8e4 server_cookie=7f94fc109e0f5b07 in_seq=0 out_seq=0 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.774+0000 7f786fe00640 10 client.17067 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f786fe00640 1 -- 172.21.15.44:0/430422238 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55705224f8e0 con 0x557052363300 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f786fe00640 10 client.17067 handle_command_reply: tid=0 2024-04-20T11:28:58.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 client.17067 shutdown 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 msgr2=0x5570522e6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e6f60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7884009a30 tx=0x7f7884014040 comp rx=0 tx=0).stop 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 shutdown_connections 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557052363300 0x5570523656f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570522ec4b0 0x5570522e6f60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5570522e38d0 0x5570522e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 --2- 172.21.15.44:0/430422238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570522e2fb0 0x5570522e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f786fe00640 0 client.17067 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 >> 172.21.15.44:0/430422238 conn(0x5570522dd6e0 msgr2=0x5570522e9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 shutdown_connections 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 1 -- 172.21.15.44:0/430422238 wait complete. 2024-04-20T11:28:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.775+0000 7f7893f7a740 20 client.17067 trim_cache size 0 max 16384 2024-04-20T11:28:58.780 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78700a3da0 msgr2=0x7f78700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78700a3da0 0x7f78700a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f787c0cdc00 tx=0x7f787c098040 comp rx=0 tx=0).stop 2024-04-20T11:28:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 msgr2=0x7f788c1a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 0x7f788c1a0810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f787c00db80 tx=0x7f787c00b630 comp rx=0 tx=0).stop 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 shutdown_connections 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78700a3da0 0x7f78700a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788c19a9c0 0x7f788c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f788c1094e0 0x7f788c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 --2- 172.21.15.44:0/94187350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104410 0x7f788c1a0810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 >> 172.21.15.44:0/94187350 conn(0x7f788c0ffcf0 msgr2=0x7f788c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 shutdown_connections 2024-04-20T11:28:58.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:58.776+0000 7f7891200640 1 -- 172.21.15.44:0/94187350 wait complete. 2024-04-20T11:28:58.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 -- 172.21.15.44:0/2846593181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 msgr2=0x7eff3810d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 --2- 172.21.15.44:0/2846593181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 0x7eff3810d7b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7eff28009860 tx=0x7eff2802bc10 comp rx=0 tx=0).stop 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 -- 172.21.15.44:0/2846593181 shutdown_connections 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 --2- 172.21.15.44:0/2846593181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff38108ea0 0x7eff381152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 --2- 172.21.15.44:0/2846593181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff38108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 --2- 172.21.15.44:0/2846593181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 0x7eff3810d7b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 -- 172.21.15.44:0/2846593181 >> 172.21.15.44:0/2846593181 conn(0x7eff38077d80 msgr2=0x7eff38106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 -- 172.21.15.44:0/2846593181 shutdown_connections 2024-04-20T11:28:59.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.002+0000 7eff3da00640 1 -- 172.21.15.44:0/2846593181 wait complete. 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.003+0000 7eff3da00640 1 Processor -- start 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.003+0000 7eff3da00640 1 -- start start 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff3819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff38108ea0 0x7eff3819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 0x7eff381a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff38118950 con 0x7eff38108500 2024-04-20T11:28:59.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3da00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff381187d0 con 0x7eff38108ea0 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff38118ad0 con 0x7eff3810d3d0 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff3819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff3819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42774/0 (socket says 172.21.15.44:42774) 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 -- 172.21.15.44:0/3758132080 learned_addr learned my addr 172.21.15.44:0/3758132080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37400640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff38108ea0 0x7eff3819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff3d000640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 0x7eff381a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 -- 172.21.15.44:0/3758132080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 msgr2=0x7eff381a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 0x7eff381a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 -- 172.21.15.44:0/3758132080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff38108ea0 msgr2=0x7eff3819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff38108ea0 0x7eff3819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.004+0000 7eff37e00640 1 -- 172.21.15.44:0/3758132080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff381a3dc0 con 0x7eff38108500 2024-04-20T11:28:59.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.005+0000 7eff37e00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff3819edd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7eff280057c0 tx=0x7eff2802c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.005+0000 7eff34c00640 1 -- 172.21.15.44:0/3758132080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28041020 con 0x7eff38108500 2024-04-20T11:28:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.005+0000 7eff34c00640 1 -- 172.21.15.44:0/3758132080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff28034940 con 0x7eff38108500 2024-04-20T11:28:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.005+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff381a4050 con 0x7eff38108500 2024-04-20T11:28:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.005+0000 7eff34c00640 1 -- 172.21.15.44:0/3758132080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28039040 con 0x7eff38108500 2024-04-20T11:28:59.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.005+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381ab940 con 0x7eff38108500 2024-04-20T11:28:59.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 Processor -- start 2024-04-20T11:28:59.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 -- start start 2024-04-20T11:28:59.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff34c00640 1 -- 172.21.15.44:0/3758132080 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7eff28044020 con 0x7eff38108500 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 0x55d00c0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0be130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0be670 0x55d00c0c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d00c01a050 con 0x55d00c0b5a90 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d00bfc0490 con 0x55d00c0be670 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.009+0000 7eff41da9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d00c016010 con 0x55d00c0b5170 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 0x55d00c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0be130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42780/0 (socket says 172.21.15.44:42780) 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff37e00640 1 -- 172.21.15.44:0/1981553120 learned_addr learned my addr 172.21.15.44:0/1981553120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 0x55d00c0b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41218/0 (socket says 172.21.15.44:41218) 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff37400640 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0be670 0x55d00c0c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 -- 172.21.15.44:0/1981553120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0be670 msgr2=0x55d00c0c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0be670 0x55d00c0c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 -- 172.21.15.44:0/1981553120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 msgr2=0x55d00c0be130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0be130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.010+0000 7eff3d000640 1 -- 172.21.15.44:0/1981553120 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00bfc8070 con 0x55d00c0b5170 2024-04-20T11:28:59.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff3d000640 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 0x55d00c0b5550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff30009b70 tx=0x7eff30011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a1d39e5e11fc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff2d600640 1 -- 172.21.15.44:0/1981553120 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30004050 con 0x55d00c0b5170 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff34c00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7eff1c0a3d20 0x7eff1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff2d600640 1 -- 172.21.15.44:0/1981553120 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff30005520 con 0x55d00c0b5170 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff2d600640 1 -- 172.21.15.44:0/1981553120 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30004d20 con 0x55d00c0b5170 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff37e00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7eff1c0a3d20 0x7eff1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.011+0000 7eff34c00640 1 -- 172.21.15.44:0/3758132080 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7eff280f1c90 con 0x7eff38108500 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 -- 172.21.15.44:0/1981553120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 msgr2=0x55d00c0b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 0x55d00c0b5550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff30009b70 tx=0x7eff30011810 comp rx=0 tx=0).stop 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff37e00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7eff1c0a3d20 0x7eff1c0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff2802c220 tx=0x7eff280de040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 -- 172.21.15.44:0/1981553120 shutdown_connections 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0be670 0x55d00c0c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 --2- 172.21.15.44:0/1981553120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0b5170 0x55d00c0b5550 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 -- 172.21.15.44:0/1981553120 >> 172.21.15.44:0/1981553120 conn(0x55d00c0af8a0 msgr2=0x55d00c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 -- 172.21.15.44:0/1981553120 shutdown_connections 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.012+0000 7eff41da9740 1 -- 172.21.15.44:0/1981553120 wait complete. 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 -- start start 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 Processor -- start 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5170 0x55d00c0bb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 0x55d00c0bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0bc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d00c0c1460 con 0x55d00c0b5170 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d00bfd3860 con 0x55d00c0b5a90 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d00c0c15e0 con 0x55d00c0be670 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.013+0000 7eff41da9740 10 client.0 _ll_register_callbacks cb 0x55d00bfa1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff3d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5170 0x55d00c0bb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 0x55d00c0bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 0x55d00c0bbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37160/0 (socket says 172.21.15.44:37160) 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 -- 172.21.15.44:0/2577393281 learned_addr learned my addr 172.21.15.44:0/2577393281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff3d000640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0bc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 -- 172.21.15.44:0/2577393281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 msgr2=0x55d00c0bc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0bc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 -- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5170 msgr2=0x55d00c0bb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5170 0x55d00c0bb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 -- 172.21.15.44:0/2577393281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00bfd7950 con 0x55d00c0b5a90 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff3d000640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5170 0x55d00c0bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff37400640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 0x55d00c0bbb80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff2000eef0 tx=0x7eff2000d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfe3ace63959f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.014+0000 7eff26000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff26000640 1 -- 172.21.15.44:0/2577393281 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2001c030 con 0x55d00c0b5a90 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff30005520 con 0x55d00c0b5a90 2024-04-20T11:28:59.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff26000640 1 -- 172.21.15.44:0/2577393281 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7eff20004b20 con 0x55d00c0b5a90 2024-04-20T11:28:59.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff26000640 1 -- 172.21.15.44:0/2577393281 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7eff20026050 con 0x55d00c0b5a90 2024-04-20T11:28:59.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff41da9740 10 client.27206 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:59.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d00bfa5820 con 0x55d00c0b5a90 2024-04-20T11:28:59.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.015+0000 7eff26000640 1 -- 172.21.15.44:0/2577393281 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff30005520 con 0x55d00c0b5a90 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff26000640 1 -- 172.21.15.44:0/2577393281 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7eff2000b040 con 0x55d00c0b5a90 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff41da9740 10 client.27206 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff41da9740 10 client.27206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff41da9740 20 client.27206 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff41da9740 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c0ce630 0x55d00c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff41da9740 4 client.27206 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d00c027a70 con 0x55d00c0ce630 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.016+0000 7eff3d000640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c0ce630 0x55d00c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.017+0000 7eff3d000640 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c0ce630 0x55d00c0d0a90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf062896de8fd1c8 server_cookie=ec45fbc61c14ddba in_seq=0 out_seq=0 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.017+0000 7eff26000640 10 client.27206 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.017+0000 7eff26000640 1 -- 172.21.15.44:0/2577393281 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d00c027a70 con 0x55d00c0ce630 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.017+0000 7eff26000640 10 client.27206 handle_command_reply: tid=0 2024-04-20T11:28:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.017+0000 7eff41da9740 1 client.27206 shutdown 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 msgr2=0x55d00c0bbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 0x55d00c0bbb80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff2000eef0 tx=0x7eff2000d7b0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 shutdown_connections 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c0ce630 0x55d00c0d0a90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0bc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5a90 0x55d00c0bbb80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 --2- 172.21.15.44:0/2577393281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5170 0x55d00c0bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 >> 172.21.15.44:0/2577393281 conn(0x55d00c0af8a0 msgr2=0x55d00c0b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff26000640 0 client.27206 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 shutdown_connections 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 1 -- 172.21.15.44:0/2577393281 wait complete. 2024-04-20T11:28:59.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.018+0000 7eff41da9740 20 client.27206 trim_cache size 0 max 16384 2024-04-20T11:28:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 Processor -- start 2024-04-20T11:28:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 -- start start 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b4050 con 0x55d00c0b5a90 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b5bc0 con 0x55d00c0b5170 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.033+0000 7eff41da9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b4720 con 0x55d00c0be670 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42808/0 (socket says 172.21.15.44:42808) 2024-04-20T11:28:59.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 -- 172.21.15.44:0/2103861556 learned_addr learned my addr 172.21.15.44:0/2103861556 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff3d000640 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff3d000640 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 -- 172.21.15.44:0/2103861556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 msgr2=0x55d00c0b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 -- 172.21.15.44:0/2103861556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 msgr2=0x55d00c0bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 -- 172.21.15.44:0/2103861556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00c0ba5f0 con 0x55d00c0b5a90 2024-04-20T11:28:59.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.034+0000 7eff37400640 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0ca350 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7eff2000e870 tx=0x7eff20009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297a525d158213ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.035+0000 7eff2cc00640 1 -- 172.21.15.44:0/2103861556 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2001c040 con 0x55d00c0b5a90 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.035+0000 7eff2cc00640 1 -- 172.21.15.44:0/2103861556 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff20021400 con 0x55d00c0b5a90 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.035+0000 7eff2cc00640 1 -- 172.21.15.44:0/2103861556 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20018a90 con 0x55d00c0b5a90 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 -- 172.21.15.44:0/2103861556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 msgr2=0x55d00c0ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0ca350 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7eff2000e870 tx=0x7eff20009300 comp rx=0 tx=0).stop 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 -- 172.21.15.44:0/2103861556 shutdown_connections 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c0b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0ca350 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 --2- 172.21.15.44:0/2103861556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 -- 172.21.15.44:0/2103861556 >> 172.21.15.44:0/2103861556 conn(0x55d00c0af8a0 msgr2=0x55d00c0ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 -- 172.21.15.44:0/2103861556 shutdown_connections 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 -- 172.21.15.44:0/2103861556 wait complete. 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.036+0000 7eff41da9740 1 -- start start 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 Processor -- start 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b54c0 con 0x55d00c0b5a90 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b4df0 con 0x55d00c0b5170 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b2020 con 0x55d00c0be670 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff41da9740 10 client.0 _ll_register_callbacks cb 0x7eff2000ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff37400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff37400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42818/0 (socket says 172.21.15.44:42818) 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff37400640 1 -- 172.21.15.44:0/2880832163 learned_addr learned my addr 172.21.15.44:0/2880832163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.037+0000 7eff3d000640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff3d000640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff37400640 1 -- 172.21.15.44:0/2880832163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 msgr2=0x55d00c133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff37400640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c133bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff37400640 1 -- 172.21.15.44:0/2880832163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 msgr2=0x55d00c0c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff37400640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff37400640 1 -- 172.21.15.44:0/2880832163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00bd2c3f0 con 0x55d00c0b5a90 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff3d000640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff37400640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0c2e20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff2000d7f0 tx=0x7eff20009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fd60131168d82a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff26000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff26000640 1 -- 172.21.15.44:0/2880832163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2001c040 con 0x55d00c0b5a90 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff20021400 con 0x55d00c0b5a90 2024-04-20T11:28:59.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.038+0000 7eff26000640 1 -- 172.21.15.44:0/2880832163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7eff20003970 con 0x55d00c0b5a90 2024-04-20T11:28:59.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff26000640 1 -- 172.21.15.44:0/2880832163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7eff20003b70 con 0x55d00c0b5a90 2024-04-20T11:28:59.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 10 client.17085 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:59.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d00bdf67d0 con 0x55d00c0b5a90 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff26000640 1 -- 172.21.15.44:0/2880832163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff20021400 con 0x55d00c0b5a90 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff26000640 1 -- 172.21.15.44:0/2880832163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7eff2000b040 con 0x55d00c0b5a90 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 10 client.17085 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 20 client.17085 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c135660 0x55d00c137a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 4 client.17085 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.039+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d00c0218e0 con 0x55d00c135660 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.040+0000 7eff3d000640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c135660 0x55d00c137a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.040+0000 7eff3d000640 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c135660 0x55d00c137a80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65deaffe666f8a28 server_cookie=8d7f31717d13bbfe in_seq=0 out_seq=0 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.040+0000 7eff26000640 10 client.17085 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.040+0000 7eff26000640 1 -- 172.21.15.44:0/2880832163 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d00c0218e0 con 0x55d00c135660 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.040+0000 7eff26000640 10 client.17085 handle_command_reply: tid=0 2024-04-20T11:28:59.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.040+0000 7eff41da9740 1 client.17085 shutdown 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 msgr2=0x55d00c0c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0c2e20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff2000d7f0 tx=0x7eff20009b60 comp rx=0 tx=0).stop 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 shutdown_connections 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55d00c135660 0x55d00c137a80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d00c0be670 0x55d00c133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00c0b5a90 0x55d00c0c2e20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 --2- 172.21.15.44:0/2880832163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d00c0b5170 0x55d00c0c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 >> 172.21.15.44:0/2880832163 conn(0x55d00c0af8a0 msgr2=0x55d00c0b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff26000640 0 client.17085 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 shutdown_connections 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 1 -- 172.21.15.44:0/2880832163 wait complete. 2024-04-20T11:28:59.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.041+0000 7eff41da9740 20 client.17085 trim_cache size 0 max 16384 2024-04-20T11:28:59.046 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:59.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7eff1c0a3d20 msgr2=0x7eff1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7eff1c0a3d20 0x7eff1c0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff2802c220 tx=0x7eff280de040 comp rx=0 tx=0).stop 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 msgr2=0x7eff3819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff3819edd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7eff280057c0 tx=0x7eff2802c630 comp rx=0 tx=0).stop 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 shutdown_connections 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff3810d3d0 0x7eff381a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff38108ea0 0x7eff3819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7eff1c0a3d20 0x7eff1c0a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 --2- 172.21.15.44:0/3758132080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff38108500 0x7eff3819edd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 >> 172.21.15.44:0/3758132080 conn(0x7eff38077d80 msgr2=0x7eff380783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 shutdown_connections 2024-04-20T11:28:59.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.042+0000 7eff3da00640 1 -- 172.21.15.44:0/3758132080 wait complete. 2024-04-20T11:28:59.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 -- 172.21.15.44:0/2607171691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7398108500 msgr2=0x7f7398108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 --2- 172.21.15.44:0/2607171691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7398108500 0x7f7398108960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7384009900 tx=0x7f738402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 -- 172.21.15.44:0/2607171691 shutdown_connections 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 --2- 172.21.15.44:0/2607171691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398108ea0 0x7f73981152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 --2- 172.21.15.44:0/2607171691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7398108500 0x7f7398108960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 --2- 172.21.15.44:0/2607171691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f739810d3d0 0x7f739810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 -- 172.21.15.44:0/2607171691 >> 172.21.15.44:0/2607171691 conn(0x7f7398077d80 msgr2=0x7f7398106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 -- 172.21.15.44:0/2607171691 shutdown_connections 2024-04-20T11:28:59.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.268+0000 7f739de00640 1 -- 172.21.15.44:0/2607171691 wait complete. 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 Processor -- start 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 -- start start 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398108500 0x7f739819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7398108ea0 0x7f739819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 0x7f73981a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7398118950 con 0x7f739810d3d0 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73981187d0 con 0x7f7398108ea0 2024-04-20T11:28:59.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.269+0000 7f739de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7398118ad0 con 0x7f7398108500 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 0x7f73981a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 0x7f73981a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42842/0 (socket says 172.21.15.44:42842) 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 -- 172.21.15.44:0/181707111 learned_addr learned my addr 172.21.15.44:0/181707111 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739ca00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398108500 0x7f739819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f7397e00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7398108ea0 0x7f739819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 -- 172.21.15.44:0/181707111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398108500 msgr2=0x7f739819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398108500 0x7f739819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 -- 172.21.15.44:0/181707111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7398108ea0 msgr2=0x7f739819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7398108ea0 0x7f739819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.270+0000 7f739d400640 1 -- 172.21.15.44:0/181707111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73981a3e10 con 0x7f739810d3d0 2024-04-20T11:28:59.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.271+0000 7f739d400640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 0x7f73981a36e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f739000ec70 tx=0x7f739000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.271+0000 7f7395600640 1 -- 172.21.15.44:0/181707111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390003b70 con 0x7f739810d3d0 2024-04-20T11:28:59.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.271+0000 7f7395600640 1 -- 172.21.15.44:0/181707111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f739000ee80 con 0x7f739810d3d0 2024-04-20T11:28:59.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.271+0000 7f7395600640 1 -- 172.21.15.44:0/181707111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390010940 con 0x7f739810d3d0 2024-04-20T11:28:59.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.271+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73981a4100 con 0x7f739810d3d0 2024-04-20T11:28:59.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.271+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981ab990 con 0x7f739810d3d0 2024-04-20T11:28:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f7395600640 1 -- 172.21.15.44:0/181707111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f73900045b0 con 0x7f739810d3d0 2024-04-20T11:28:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 Processor -- start 2024-04-20T11:28:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 -- start start 2024-04-20T11:28:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da91550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91a90 0x55738da9a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738da9ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55738d9f6050 con 0x55738da91170 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55738d99c490 con 0x55738da9a670 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.275+0000 7f73a21bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55738d9f2010 con 0x55738da91a90 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f7397e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da91550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f7397e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da91550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42850/0 (socket says 172.21.15.44:42850) 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f7397e00640 1 -- 172.21.15.44:0/3575835419 learned_addr learned my addr 172.21.15.44:0/3575835419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f739ca00640 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91a90 0x55738da9a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f7397e00640 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738da9ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f739ca00640 1 -- 172.21.15.44:0/3575835419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 msgr2=0x55738da9ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f739ca00640 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738da9ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f739ca00640 1 -- 172.21.15.44:0/3575835419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 msgr2=0x55738da91550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f739ca00640 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da91550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f739ca00640 1 -- 172.21.15.44:0/3575835419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738d9a4070 con 0x55738da91a90 2024-04-20T11:28:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.276+0000 7f7397e00640 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da91550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.277+0000 7f739ca00640 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91a90 0x55738da9a130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f738c009c20 tx=0x7f738c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542b2b686bb262a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.277+0000 7f7389c00640 1 -- 172.21.15.44:0/3575835419 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c004050 con 0x55738da91a90 2024-04-20T11:28:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.277+0000 7f7389c00640 1 -- 172.21.15.44:0/3575835419 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f738c005520 con 0x55738da91a90 2024-04-20T11:28:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.277+0000 7f7389c00640 1 -- 172.21.15.44:0/3575835419 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c004e00 con 0x55738da91a90 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.277+0000 7f7395600640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73800a3da0 0x7f73800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f7395600640 1 -- 172.21.15.44:0/181707111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f73900d0a90 con 0x7f739810d3d0 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f7397e00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73800a3da0 0x7f73800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 -- 172.21.15.44:0/3575835419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91a90 msgr2=0x55738da9a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91a90 0x55738da9a130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f738c009c20 tx=0x7f738c011810 comp rx=0 tx=0).stop 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 -- 172.21.15.44:0/3575835419 shutdown_connections 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738da9ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91a90 0x55738da9a130 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3575835419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da91550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 -- 172.21.15.44:0/3575835419 >> 172.21.15.44:0/3575835419 conn(0x55738da8b8a0 msgr2=0x55738da8bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 -- 172.21.15.44:0/3575835419 shutdown_connections 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f73a21bc740 1 -- 172.21.15.44:0/3575835419 wait complete. 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.278+0000 7f7397e00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73800a3da0 0x7f73800a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55738da8ca30 tx=0x7f7384005b50 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 -- start start 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 Processor -- start 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738da97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da9a670 0x55738da98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55738da9d460 con 0x55738da9a670 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55738d9af860 con 0x55738da91a90 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.279+0000 7f73a21bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55738da9d5e0 con 0x55738da91170 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f73a21bc740 10 client.0 _ll_register_callbacks cb 0x55738d97dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f739ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738da97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738da97c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37202/0 (socket says 172.21.15.44:37202) 2024-04-20T11:28:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 -- 172.21.15.44:0/1193628984 learned_addr learned my addr 172.21.15.44:0/1193628984 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f739ca00640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41280/0 (socket says 172.21.15.44:41280) 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f739d400640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da9a670 0x55738da98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 -- 172.21.15.44:0/1193628984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 msgr2=0x55738da976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 -- 172.21.15.44:0/1193628984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da9a670 msgr2=0x55738da98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da9a670 0x55738da98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.280+0000 7f7397e00640 1 -- 172.21.15.44:0/1193628984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738d9b3950 con 0x55738da91a90 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f7397e00640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738da97c00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7384035000 tx=0x7f738402bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d609d27ee6f118 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f737e000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f737e000640 1 -- 172.21.15.44:0/1193628984 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384049040 con 0x55738da91a90 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f737e000640 1 -- 172.21.15.44:0/1193628984 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f73840035f0 con 0x55738da91a90 2024-04-20T11:28:59.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738c005520 con 0x55738da91a90 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f737e000640 1 -- 172.21.15.44:0/1193628984 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f738405b050 con 0x55738da91a90 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f73a21bc740 10 client.27224 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.281+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55738d981820 con 0x55738da91a90 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f737e000640 1 -- 172.21.15.44:0/1193628984 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f738c005520 con 0x55738da91a90 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f737e000640 1 -- 172.21.15.44:0/1193628984 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f7384040040 con 0x55738da91a90 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f73a21bc740 10 client.27224 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f73a21bc740 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f73a21bc740 20 client.27224 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f73a21bc740 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738daaa720 0x55738daacb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f73a21bc740 4 client.27224 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738da03a70 con 0x55738daaa720 2024-04-20T11:28:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.282+0000 7f739ca00640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738daaa720 0x55738daacb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.283+0000 7f739ca00640 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738daaa720 0x55738daacb80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819f8f921e9986b5 server_cookie=799845b491b148a5 in_seq=0 out_seq=0 2024-04-20T11:28:59.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.283+0000 7f737e000640 10 client.27224 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:59.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f737e000640 1 -- 172.21.15.44:0/1193628984 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55738da03a70 con 0x55738daaa720 2024-04-20T11:28:59.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f737e000640 10 client.27224 handle_command_reply: tid=0 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 client.27224 shutdown 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 msgr2=0x55738da97c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738da97c00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7384035000 tx=0x7f738402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 shutdown_connections 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738daaa720 0x55738daacb80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da9a670 0x55738da98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738da97c00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f737e000640 0 client.27224 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 --2- 172.21.15.44:0/1193628984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 >> 172.21.15.44:0/1193628984 conn(0x55738da8b8a0 msgr2=0x55738da8d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 shutdown_connections 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 1 -- 172.21.15.44:0/1193628984 wait complete. 2024-04-20T11:28:59.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.284+0000 7f73a21bc740 20 client.27224 trim_cache size 0 max 16384 2024-04-20T11:28:59.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 Processor -- start 2024-04-20T11:28:59.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 -- start start 2024-04-20T11:28:59.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da99df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738daa6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da9a670 0x55738dab2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b3f50 con 0x55738da91170 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b5ac0 con 0x55738da91a90 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f73a21bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b4620 con 0x55738da9a670 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f739ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da99df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f739ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da99df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42870/0 (socket says 172.21.15.44:42870) 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f739d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da9a670 0x55738dab2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f739ca00640 1 -- 172.21.15.44:0/3227707071 learned_addr learned my addr 172.21.15.44:0/3227707071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.300+0000 7f7397e00640 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738daa6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739ca00640 1 -- 172.21.15.44:0/3227707071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da9a670 msgr2=0x55738dab2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739ca00640 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da9a670 0x55738dab2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739ca00640 1 -- 172.21.15.44:0/3227707071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 msgr2=0x55738daa6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739ca00640 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738daa6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739ca00640 1 -- 172.21.15.44:0/3227707071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738d7083f0 con 0x55738da91170 2024-04-20T11:28:59.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739d400640 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da9a670 0x55738dab2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:28:59.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f739ca00640 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da99df0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f738c004820 tx=0x7f738c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d0543a87540c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f7389200640 1 -- 172.21.15.44:0/3227707071 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c02edd0 con 0x55738da91170 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f7389200640 1 -- 172.21.15.44:0/3227707071 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f738c02e900 con 0x55738da91170 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.301+0000 7f7389200640 1 -- 172.21.15.44:0/3227707071 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c0035f0 con 0x55738da91170 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 -- 172.21.15.44:0/3227707071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 msgr2=0x55738da99df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da99df0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f738c004820 tx=0x7f738c004a80 comp rx=0 tx=0).stop 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 -- 172.21.15.44:0/3227707071 shutdown_connections 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da9a670 0x55738dab2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da91a90 0x55738daa6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 --2- 172.21.15.44:0/3227707071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91170 0x55738da99df0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 -- 172.21.15.44:0/3227707071 >> 172.21.15.44:0/3227707071 conn(0x55738da8b8a0 msgr2=0x55738da97f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 -- 172.21.15.44:0/3227707071 shutdown_connections 2024-04-20T11:28:59.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.302+0000 7f73a21bc740 1 -- 172.21.15.44:0/3227707071 wait complete. 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.303+0000 7f73a21bc740 1 -- start start 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.303+0000 7f73a21bc740 1 Processor -- start 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.303+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.303+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91a90 0x55738da94cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.303+0000 7f73a21bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738db0fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f73a21bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b53c0 con 0x55738da91a90 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f73a21bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b4cf0 con 0x55738da9a670 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f73a21bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73981b1f60 con 0x55738da91170 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f73a21bc740 10 client.0 _ll_register_callbacks cb 0x7f73840056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da947b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41294/0 (socket says 172.21.15.44:41294) 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 -- 172.21.15.44:0/420457792 learned_addr learned my addr 172.21.15.44:0/420457792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739d400640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738db0fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f7397e00640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91a90 0x55738da94cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 -- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 msgr2=0x55738db0fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738db0fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 -- 172.21.15.44:0/420457792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91a90 msgr2=0x55738da94cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91a90 0x55738da94cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.304+0000 7f739ca00640 1 -- 172.21.15.44:0/420457792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738d7d27d0 con 0x55738da91170 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f739ca00640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da947b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f738c004a50 tx=0x7f738c009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c3ee041d6effe4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f737e000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f737e000640 1 -- 172.21.15.44:0/420457792 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c03c060 con 0x55738da91170 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f737e000640 1 -- 172.21.15.44:0/420457792 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f738c02ebd0 con 0x55738da91170 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738c02e900 con 0x55738da91170 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f737e000640 1 -- 172.21.15.44:0/420457792 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f738c011d10 con 0x55738da91170 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f73a21bc740 10 client.27232 fetch_fsmap learned FSMap version 54 2024-04-20T11:28:59.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.305+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55738db111f0 con 0x55738da91170 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f737e000640 1 -- 172.21.15.44:0/420457792 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f738c02e900 con 0x55738da91170 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f737e000640 1 -- 172.21.15.44:0/420457792 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f738c03f030 con 0x55738da91170 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f73a21bc740 10 client.27232 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f73a21bc740 10 client.27232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f73a21bc740 20 client.27232 populate_metadata read hostname 'smithi044' 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f73a21bc740 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738db117d0 0x55738db13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f73a21bc740 4 client.27232 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55738d9fd8e0 con 0x55738db117d0 2024-04-20T11:28:59.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.306+0000 7f7397e00640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738db117d0 0x55738db13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f7397e00640 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738db117d0 0x55738db13bf0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803d44bbebe41fc8 server_cookie=4df857931e9ae011 in_seq=0 out_seq=0 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f737e000640 10 client.27232 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f737e000640 1 -- 172.21.15.44:0/420457792 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55738d9fd8e0 con 0x55738db117d0 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f737e000640 10 client.27232 handle_command_reply: tid=0 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f73a21bc740 1 client.27232 shutdown 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 msgr2=0x55738da947b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.307+0000 7f73a21bc740 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da947b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f738c004a50 tx=0x7f738c009bb0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 shutdown_connections 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55738db117d0 0x55738db13bf0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55738da9a670 0x55738db0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55738da91a90 0x55738da94cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 --2- 172.21.15.44:0/420457792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55738da91170 0x55738da947b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 >> 172.21.15.44:0/420457792 conn(0x55738da8b8a0 msgr2=0x55738da8cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f737e000640 0 client.27232 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 shutdown_connections 2024-04-20T11:28:59.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 1 -- 172.21.15.44:0/420457792 wait complete. 2024-04-20T11:28:59.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.308+0000 7f73a21bc740 20 client.27232 trim_cache size 0 max 16384 2024-04-20T11:28:59.312 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73800a3da0 msgr2=0x7f73800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73800a3da0 0x7f73800a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55738da8ca30 tx=0x7f7384005b50 comp rx=0 tx=0).stop 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 msgr2=0x7f73981a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 0x7f73981a36e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f739000ec70 tx=0x7f739000c6a0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 shutdown_connections 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f73800a3da0 0x7f73800a6260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739810d3d0 0x7f73981a36e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7398108ea0 0x7f739819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 --2- 172.21.15.44:0/181707111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398108500 0x7f739819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 >> 172.21.15.44:0/181707111 conn(0x7f7398077d80 msgr2=0x7f73980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 shutdown_connections 2024-04-20T11:28:59.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:28:59.309+0000 7f739de00640 1 -- 172.21.15.44:0/181707111 wait complete. 2024-04-20T11:28:59.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:28:59 smithi078 ceph-mon[21464]: pgmap v194: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:28:59.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:28:59 smithi044 ceph-mon[16855]: pgmap v194: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:28:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:28:59 smithi165 ceph-mon[21975]: pgmap v194: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:00.328 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:29:00.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:00.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:29:00.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.549+0000 7f7e2aa00640 1 -- 172.21.15.44:0/2173399046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 msgr2=0x7f7e24110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.549+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/2173399046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e24110f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009ca0 tx=0x7f7e1c02c100 comp rx=0 tx=0).stop 2024-04-20T11:29:00.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 -- 172.21.15.44:0/2173399046 shutdown_connections 2024-04-20T11:29:00.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/2173399046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e24110f20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/2173399046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e24104c30 0x7f7e24108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/2173399046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 0x7f7e24104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 -- 172.21.15.44:0/2173399046 >> 172.21.15.44:0/2173399046 conn(0x7f7e240ffaa0 msgr2=0x7f7e24101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 -- 172.21.15.44:0/2173399046 shutdown_connections 2024-04-20T11:29:00.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.550+0000 7f7e2aa00640 1 -- 172.21.15.44:0/2173399046 wait complete. 2024-04-20T11:29:00.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 Processor -- start 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 -- start start 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 0x7f7e241a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e241a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e2419a820 0x7f7e2419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24114520 con 0x7f7e24109350 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241143a0 con 0x7f7e2419a820 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.551+0000 7f7e2aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241146a0 con 0x7f7e24104280 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e28c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e241a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 0x7f7e241a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e28c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e241a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42884/0 (socket says 172.21.15.44:42884) 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e28c00640 1 -- 172.21.15.44:0/1884579927 learned_addr learned my addr 172.21.15.44:0/1884579927 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e2419a820 0x7f7e2419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 -- 172.21.15.44:0/1884579927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e2419a820 msgr2=0x7f7e2419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e2419a820 0x7f7e2419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 -- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 msgr2=0x7f7e241a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e241a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 -- 172.21.15.44:0/1884579927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2419b3b0 con 0x7f7e24104280 2024-04-20T11:29:00.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.552+0000 7f7e29600640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 0x7f7e241a06b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800e9f0 tx=0x7f7e1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.553+0000 7f7e22000640 1 -- 172.21.15.44:0/1884579927 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800ccc0 con 0x7f7e24104280 2024-04-20T11:29:00.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.553+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2419b6a0 con 0x7f7e24104280 2024-04-20T11:29:00.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.553+0000 7f7e22000640 1 -- 172.21.15.44:0/1884579927 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7e18010930 con 0x7f7e24104280 2024-04-20T11:29:00.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.553+0000 7f7e22000640 1 -- 172.21.15.44:0/1884579927 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e180189f0 con 0x7f7e24104280 2024-04-20T11:29:00.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.553+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a7880 con 0x7f7e24104280 2024-04-20T11:29:00.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.556+0000 7f7e2ec31740 1 Processor -- start 2024-04-20T11:29:00.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.556+0000 7f7e22000640 1 -- 172.21.15.44:0/1884579927 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7e18010ae0 con 0x7f7e24104280 2024-04-20T11:29:00.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.556+0000 7f7e2ec31740 1 -- start start 2024-04-20T11:29:00.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.556+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.556+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e8a10 0x560ace0f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0f15f0 0x560ace0f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560ace04cd20 con 0x560ace0e8a10 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560acdff33e0 con 0x560ace0e80f0 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560ace048ce0 con 0x560ace0f15f0 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e29600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e8a10 0x560ace0f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e29600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e8a10 0x560ace0f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42894/0 (socket says 172.21.15.44:42894) 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e29600640 1 -- 172.21.15.44:0/3506390908 learned_addr learned my addr 172.21.15.44:0/3506390908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e28c00640 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0f15f0 0x560ace0f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2a000640 1 -- 172.21.15.44:0/3506390908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0f15f0 msgr2=0x560ace0f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2a000640 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0f15f0 0x560ace0f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2a000640 1 -- 172.21.15.44:0/3506390908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e8a10 msgr2=0x560ace0f10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2a000640 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e8a10 0x560ace0f10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.557+0000 7f7e2a000640 1 -- 172.21.15.44:0/3506390908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ace012980 con 0x560ace0e80f0 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e2a000640 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0e84d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009bf0 tx=0x7f7e1c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e53cde826ccf4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e22000640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7e0c0a3d20 0x7f7e0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e22000640 1 -- 172.21.15.44:0/1884579927 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7e180d0ab0 con 0x7f7e24104280 2024-04-20T11:29:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e29600640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7e0c0a3d20 0x7f7e0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e16600640 1 -- 172.21.15.44:0/3506390908 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c043070 con 0x560ace0e80f0 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e16600640 1 -- 172.21.15.44:0/3506390908 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7e1c032d40 con 0x560ace0e80f0 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.558+0000 7f7e16600640 1 -- 172.21.15.44:0/3506390908 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c039040 con 0x560ace0e80f0 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 -- 172.21.15.44:0/3506390908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 msgr2=0x560ace0e84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0e84d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009bf0 tx=0x7f7e1c005f80 comp rx=0 tx=0).stop 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e29600640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7e0c0a3d20 0x7f7e0c0a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e1809caa0 tx=0x7f7e180d4560 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 -- 172.21.15.44:0/3506390908 shutdown_connections 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0f15f0 0x560ace0f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e8a10 0x560ace0f10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/3506390908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0e84d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 -- 172.21.15.44:0/3506390908 >> 172.21.15.44:0/3506390908 conn(0x560ace0e2820 msgr2=0x560ace0e2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 -- 172.21.15.44:0/3506390908 shutdown_connections 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.559+0000 7f7e2ec31740 1 -- 172.21.15.44:0/3506390908 wait complete. 2024-04-20T11:29:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 -- start start 2024-04-20T11:29:00.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 Processor -- start 2024-04-20T11:29:00.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560ace0f4380 con 0x560ace0e80f0 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560ace0067a0 con 0x560ace0f15f0 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560ace0f4500 con 0x560ace0e8a10 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.560+0000 7f7e2ec31740 10 client.0 _ll_register_callbacks cb 0x560ace008b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42910/0 (socket says 172.21.15.44:42910) 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 -- 172.21.15.44:0/1629196678 learned_addr learned my addr 172.21.15.44:0/1629196678 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e28c00640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 -- 172.21.15.44:0/1629196678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 msgr2=0x560ace0eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 -- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 msgr2=0x560ace0ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0ef150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 -- 172.21.15.44:0/1629196678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560acde277e0 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e28c00640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.561+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0ee6d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009dc0 tx=0x7f7e1c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37009adf35a3fab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1629196678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c053060 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1629196678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7e1c032840 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1c032d40 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1629196678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7e1c039040 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e2ec31740 10 client.17115 fetch_fsmap learned FSMap version 54 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560acde87f70 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.562+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1629196678 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7e1c032d40 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1629196678 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f7e1c056020 con 0x560ace0e80f0 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e2ec31740 10 client.17115 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:29:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e2ec31740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e2ec31740 20 client.17115 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace101860 0x560ace103c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e2ec31740 4 client.17115 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ace05a740 con 0x560ace101860 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.563+0000 7f7e28c00640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace101860 0x560ace103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.564+0000 7f7e28c00640 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace101860 0x560ace103c50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a45cc0f2fcecfc server_cookie=1fc7340a1f36ccc9 in_seq=0 out_seq=0 2024-04-20T11:29:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.564+0000 7f7e0aa00640 10 client.17115 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1629196678 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ace05a740 con 0x560ace101860 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e0aa00640 10 client.17115 handle_command_reply: tid=0 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 client.17115 shutdown 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 msgr2=0x560ace0ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0ee6d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009dc0 tx=0x7f7e1c009540 comp rx=0 tx=0).stop 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 shutdown_connections 2024-04-20T11:29:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace101860 0x560ace103c50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1629196678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0ee6d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e0aa00640 0 client.17115 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 >> 172.21.15.44:0/1629196678 conn(0x560ace0e2820 msgr2=0x560ace0e4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 shutdown_connections 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1629196678 wait complete. 2024-04-20T11:29:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.565+0000 7f7e2ec31740 20 client.17115 trim_cache size 0 max 16384 2024-04-20T11:29:00.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 Processor -- start 2024-04-20T11:29:00.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 -- start start 2024-04-20T11:29:00.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241afc40 con 0x560ace0e80f0 2024-04-20T11:29:00.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b17b0 con 0x560ace0f15f0 2024-04-20T11:29:00.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b0310 con 0x560ace0e8a10 2024-04-20T11:29:00.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42922/0 (socket says 172.21.15.44:42922) 2024-04-20T11:29:00.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e2a000640 1 -- 172.21.15.44:0/2766081685 learned_addr learned my addr 172.21.15.44:0/2766081685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.585+0000 7f7e28c00640 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e2a000640 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e28c00640 1 -- 172.21.15.44:0/2766081685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 msgr2=0x560ace0e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e28c00640 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e28c00640 1 -- 172.21.15.44:0/2766081685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 msgr2=0x560ace0f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e28c00640 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0f0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e28c00640 1 -- 172.21.15.44:0/2766081685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ace0f3d50 con 0x560ace0e8a10 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e2a000640 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e28c00640 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0fd240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x560ace0efca0 tx=0x7f7e10005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971b92c64a16676c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.586+0000 7f7e15c00640 1 -- 172.21.15.44:0/2766081685 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10025070 con 0x560ace0e8a10 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e15c00640 1 -- 172.21.15.44:0/2766081685 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7e10004220 con 0x560ace0e8a10 2024-04-20T11:29:00.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e15c00640 1 -- 172.21.15.44:0/2766081685 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100039f0 con 0x560ace0e8a10 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 -- 172.21.15.44:0/2766081685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 msgr2=0x560ace0fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0fd240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x560ace0efca0 tx=0x7f7e10005f90 comp rx=0 tx=0).stop 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 -- 172.21.15.44:0/2766081685 shutdown_connections 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0f15f0 0x560ace0e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0fd240 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/2766081685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0e80f0 0x560ace0f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.587+0000 7f7e2ec31740 1 -- 172.21.15.44:0/2766081685 >> 172.21.15.44:0/2766081685 conn(0x560ace0e2820 msgr2=0x560ace0e3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.588+0000 7f7e2ec31740 1 -- 172.21.15.44:0/2766081685 shutdown_connections 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.588+0000 7f7e2ec31740 1 -- 172.21.15.44:0/2766081685 wait complete. 2024-04-20T11:29:00.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.588+0000 7f7e2ec31740 1 -- start start 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.588+0000 7f7e2ec31740 1 Processor -- start 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0ec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 0x560ace166a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b10b0 con 0x560ace0f15f0 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b09e0 con 0x560ace0e80f0 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241adc30 con 0x560ace0e8a10 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2ec31740 10 client.0 _ll_register_callbacks cb 0x7f7e180d4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 0x560ace166a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e28c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 0x560ace166a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42926/0 (socket says 172.21.15.44:42926) 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 -- 172.21.15.44:0/1171214915 learned_addr learned my addr 172.21.15.44:0/1171214915 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 -- 172.21.15.44:0/1171214915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 msgr2=0x560ace0ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0ec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 -- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 msgr2=0x560ace0ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.589+0000 7f7e2a000640 1 -- 172.21.15.44:0/1171214915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560acdd1f170 con 0x560ace0f15f0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e2a000640 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 0x560ace166a60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c02c620 tx=0x7f7e1c036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3859b9e4b410a84c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1171214915 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c043040 con 0x560ace0f15f0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1171214915 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f7e1c058440 con 0x560ace0f15f0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e10004220 con 0x560ace0f15f0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1171214915 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7e1c053020 con 0x560ace0f15f0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e2ec31740 10 client.17121 fetch_fsmap learned FSMap version 54 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.590+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560ace168140 con 0x560ace0f15f0 2024-04-20T11:29:00.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1171214915 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7e10004220 con 0x560ace0f15f0 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1171214915 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f7e1c03f560 con 0x560ace0f15f0 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e2ec31740 10 client.17121 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e2ec31740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e2ec31740 20 client.17121 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace168740 0x560ace16ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e2ec31740 4 client.17121 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ace0545b0 con 0x560ace168740 2024-04-20T11:29:00.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.591+0000 7f7e28c00640 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace168740 0x560ace16ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.592+0000 7f7e28c00640 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace168740 0x560ace16ab60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325cd17d0e083c17 server_cookie=32b2821000287fed in_seq=0 out_seq=0 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.592+0000 7f7e0aa00640 10 client.17121 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e0aa00640 1 -- 172.21.15.44:0/1171214915 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ace0545b0 con 0x560ace168740 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e0aa00640 10 client.17121 handle_command_reply: tid=0 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 client.17121 shutdown 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 msgr2=0x560ace166a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 0x560ace166a60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c02c620 tx=0x7f7e1c036040 comp rx=0 tx=0).stop 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 shutdown_connections 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560ace168740 0x560ace16ab60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ace0f15f0 0x560ace166a60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.593+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ace0e8a10 0x560ace0ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.594+0000 7f7e2ec31740 1 --2- 172.21.15.44:0/1171214915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ace0e80f0 0x560ace0ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.594+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 >> 172.21.15.44:0/1171214915 conn(0x560ace0e2820 msgr2=0x560ace0e1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.594+0000 7f7e0aa00640 0 client.17121 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.594+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 shutdown_connections 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.594+0000 7f7e2ec31740 1 -- 172.21.15.44:0/1171214915 wait complete. 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.594+0000 7f7e2ec31740 20 client.17121 trim_cache size 0 max 16384 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "a6db97ea-dd10-4bb4-bef8-59dc040bdd48", 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:29:00.597 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7e0c0a3d20 msgr2=0x7f7e0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7e0c0a3d20 0x7f7e0c0a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e1809caa0 tx=0x7f7e180d4560 comp rx=0 tx=0).stop 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 msgr2=0x7f7e241a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 0x7f7e241a06b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800e9f0 tx=0x7f7e1800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 shutdown_connections 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e2419a820 0x7f7e2419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f7e0c0a3d20 0x7f7e0c0a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e24109350 0x7f7e241a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 --2- 172.21.15.44:0/1884579927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e24104280 0x7f7e241a06b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 >> 172.21.15.44:0/1884579927 conn(0x7f7e240ffaa0 msgr2=0x7f7e24100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 shutdown_connections 2024-04-20T11:29:00.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.595+0000 7f7e2aa00640 1 -- 172.21.15.44:0/1884579927 wait complete. 2024-04-20T11:29:00.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:29:00.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:29:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 -- 172.21.15.44:0/605334889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 msgr2=0x7f14fc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 --2- 172.21.15.44:0/605334889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 0x7f14fc102460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec02bc30 comp rx=0 tx=0).stop 2024-04-20T11:29:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 -- 172.21.15.44:0/605334889 shutdown_connections 2024-04-20T11:29:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 --2- 172.21.15.44:0/605334889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc107150 0x7f14fc10ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 --2- 172.21.15.44:0/605334889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc102a30 0x7f14fc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 --2- 172.21.15.44:0/605334889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 0x7f14fc102460 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 -- 172.21.15.44:0/605334889 >> 172.21.15.44:0/605334889 conn(0x7f14fc0fd900 msgr2=0x7f14fc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.834+0000 7f14fb200640 1 -- 172.21.15.44:0/605334889 shutdown_connections 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.835+0000 7f14fb200640 1 -- 172.21.15.44:0/605334889 wait complete. 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.836+0000 7f14fb200640 1 Processor -- start 2024-04-20T11:29:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.836+0000 7f14fb200640 1 -- start start 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 0x7f14fc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 0x7f14fc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc19a870 0x7f14fc19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1122f0 con 0x7f14fc107150 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112170 con 0x7f14fc102080 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112470 con 0x7f14fc19a870 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 0x7f14fc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 0x7f14fc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 0x7f14fc1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42952/0 (socket says 172.21.15.44:42952) 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 -- 172.21.15.44:0/130380325 learned_addr learned my addr 172.21.15.44:0/130380325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fa800640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc19a870 0x7f14fc19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 -- 172.21.15.44:0/130380325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc19a870 msgr2=0x7f14fc19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc19a870 0x7f14fc19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 -- 172.21.15.44:0/130380325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 msgr2=0x7f14fc1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 0x7f14fc1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14f9400640 1 -- 172.21.15.44:0/130380325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc19b390 con 0x7f14fc107150 2024-04-20T11:29:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.837+0000 7f14fa800640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc19a870 0x7f14fc19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.838+0000 7f14f9400640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 0x7f14fc1a0c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9c0 tx=0x7f14e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.838+0000 7f14f2a00640 1 -- 172.21.15.44:0/130380325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800cd80 con 0x7f14fc107150 2024-04-20T11:29:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.838+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc19b680 con 0x7f14fc107150 2024-04-20T11:29:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.838+0000 7f14f2a00640 1 -- 172.21.15.44:0/130380325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e8005520 con 0x7f14fc107150 2024-04-20T11:29:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.838+0000 7f14f2a00640 1 -- 172.21.15.44:0/130380325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8018630 con 0x7f14fc107150 2024-04-20T11:29:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.838+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a7920 con 0x7f14fc107150 2024-04-20T11:29:00.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.841+0000 7f1503c19740 1 Processor -- start 2024-04-20T11:29:00.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 -- start start 2024-04-20T11:29:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 0x559deea6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6eca0 0x559deea77340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea79c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559dee9d4150 con 0x559deea6e380 2024-04-20T11:29:00.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559dee97a370 con 0x559deea77880 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f1503c19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559dee9d0110 con 0x559deea6eca0 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f14fa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 0x559deea6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f14fa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 0x559deea6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42956/0 (socket says 172.21.15.44:42956) 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f14fa800640 1 -- 172.21.15.44:0/1418314459 learned_addr learned my addr 172.21.15.44:0/1418314459 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.842+0000 7f14f2a00640 1 -- 172.21.15.44:0/130380325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f14e80046c0 con 0x7f14fc107150 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14f9e00640 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6eca0 0x559deea77340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14fa800640 1 -- 172.21.15.44:0/1418314459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6eca0 msgr2=0x559deea77340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14fa800640 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6eca0 0x559deea77340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14fa800640 1 -- 172.21.15.44:0/1418314459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 msgr2=0x559deea79c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14fa800640 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea79c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14fa800640 1 -- 172.21.15.44:0/1418314459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dee7b07d0 con 0x559deea6e380 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.843+0000 7f14fa800640 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 0x559deea6e760 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009fd0 tx=0x7f14f4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bd925b56b49b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.844+0000 7f14e7400640 1 -- 172.21.15.44:0/1418314459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4011700 con 0x559deea6e380 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.844+0000 7f14e7400640 1 -- 172.21.15.44:0/1418314459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14f4011da0 con 0x559deea6e380 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.844+0000 7f14e7400640 1 -- 172.21.15.44:0/1418314459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f400e5c0 con 0x559deea6e380 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.844+0000 7f14f2a00640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14e00a3d20 0x7f14e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.844+0000 7f1503c19740 1 -- 172.21.15.44:0/1418314459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 msgr2=0x559deea6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.844+0000 7f1503c19740 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 0x559deea6e760 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009fd0 tx=0x7f14f4010860 comp rx=0 tx=0).stop 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f14f2a00640 1 -- 172.21.15.44:0/130380325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f14e80d0d20 con 0x7f14fc107150 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f14f9e00640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14e00a3d20 0x7f14e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 -- 172.21.15.44:0/1418314459 shutdown_connections 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea79c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6eca0 0x559deea77340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 --2- 172.21.15.44:0/1418314459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6e380 0x559deea6e760 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 -- 172.21.15.44:0/1418314459 >> 172.21.15.44:0/1418314459 conn(0x559deea68ab0 msgr2=0x559deea68e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 -- 172.21.15.44:0/1418314459 shutdown_connections 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.845+0000 7f1503c19740 1 -- 172.21.15.44:0/1418314459 wait complete. 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f14f9e00640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14e00a3d20 0x7f14e00a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14fc19b9b0 tx=0x7f14ec005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 -- start start 2024-04-20T11:29:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 Processor -- start 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea6e380 0x559deea74980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea74ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 0x559deea75400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559deea7a670 con 0x559deea6eca0 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559dee98d740 con 0x559deea6e380 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559deea7a7f0 con 0x559deea77880 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.846+0000 7f1503c19740 10 client.0 _ll_register_callbacks cb 0x559dee988b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea74ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 0x559deea75400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea74ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42966/0 (socket says 172.21.15.44:42966) 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14f9400640 1 -- 172.21.15.44:0/2551414013 learned_addr learned my addr 172.21.15.44:0/2551414013 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 0x559deea75400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41386/0 (socket says 172.21.15.44:41386) 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea6e380 0x559deea74980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 -- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea6e380 msgr2=0x559deea74980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea6e380 0x559deea74980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 -- 172.21.15.44:0/2551414013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 msgr2=0x559deea74ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea74ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 -- 172.21.15.44:0/2551414013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dee6e63f0 con 0x559deea77880 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14f9400640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea74ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.847+0000 7f14fa800640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 0x559deea75400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f14f4011010 tx=0x7f14f4004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1328ecdf7d373 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f14dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f14dbe00640 1 -- 172.21.15.44:0/2551414013 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f402b060 con 0x559deea77880 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f14dbe00640 1 -- 172.21.15.44:0/2551414013 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f14f4011710 con 0x559deea77880 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f4011da0 con 0x559deea77880 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f14dbe00640 1 -- 172.21.15.44:0/2551414013 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f14f4011910 con 0x559deea77880 2024-04-20T11:29:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f1503c19740 10 client.27262 fetch_fsmap learned FSMap version 54 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559dee6a60f0 con 0x559deea77880 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.848+0000 7f14dbe00640 1 -- 172.21.15.44:0/2551414013 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14f4011da0 con 0x559deea77880 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f14dbe00640 1 -- 172.21.15.44:0/2551414013 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f14f402e020 con 0x559deea77880 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f1503c19740 10 client.27262 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f1503c19740 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f1503c19740 20 client.27262 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f1503c19740 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deea87c80 0x559deea8a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f1503c19740 4 client.27262 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dee9e1b70 con 0x559deea87c80 2024-04-20T11:29:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.849+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deea87c80 0x559deea8a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.850+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deea87c80 0x559deea8a070 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b0804dc2aebf76 server_cookie=5de3b9bcabe0ce8a in_seq=0 out_seq=0 2024-04-20T11:29:00.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.850+0000 7f14dbe00640 10 client.27262 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f14dbe00640 1 -- 172.21.15.44:0/2551414013 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dee9e1b70 con 0x559deea87c80 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f14dbe00640 10 client.27262 handle_command_reply: tid=0 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 client.27262 shutdown 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 msgr2=0x559deea75400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 0x559deea75400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f14f4011010 tx=0x7f14f4004cf0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 shutdown_connections 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deea87c80 0x559deea8a070 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea77880 0x559deea75400 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea74ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f14dbe00640 0 client.27262 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 --2- 172.21.15.44:0/2551414013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea6e380 0x559deea74980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 >> 172.21.15.44:0/2551414013 conn(0x559deea68ab0 msgr2=0x559deea6a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 shutdown_connections 2024-04-20T11:29:00.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 1 -- 172.21.15.44:0/2551414013 wait complete. 2024-04-20T11:29:00.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.851+0000 7f1503c19740 20 client.27262 trim_cache size 0 max 16384 2024-04-20T11:29:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.870+0000 7f1503c19740 1 Processor -- start 2024-04-20T11:29:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 -- start start 2024-04-20T11:29:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1afd30 con 0x559deea6eca0 2024-04-20T11:29:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b18a0 con 0x559deea77880 2024-04-20T11:29:00.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f1503c19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b0400 con 0x559deea6e380 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea83530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42968/0 (socket says 172.21.15.44:42968) 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f14f9400640 1 -- 172.21.15.44:0/1792240583 learned_addr learned my addr 172.21.15.44:0/1792240583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.871+0000 7f14f9e00640 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14fa800640 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14f9400640 1 -- 172.21.15.44:0/1792240583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 msgr2=0x559deea770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14f9400640 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14f9400640 1 -- 172.21.15.44:0/1792240583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 msgr2=0x559deea738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14f9400640 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14f9400640 1 -- 172.21.15.44:0/1792240583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559deea776b0 con 0x559deea6eca0 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14fa800640 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.872+0000 7f14f9400640 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea83530 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f14e809ccc0 tx=0x7f14e809cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4717748a8d6bdf0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f14e6a00640 1 -- 172.21.15.44:0/1792240583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e809d430 con 0x559deea6eca0 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f14e6a00640 1 -- 172.21.15.44:0/1792240583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e80b6070 con 0x559deea6eca0 2024-04-20T11:29:00.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f14e6a00640 1 -- 172.21.15.44:0/1792240583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e80993a0 con 0x559deea6eca0 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 -- 172.21.15.44:0/1792240583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 msgr2=0x559deea83530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea83530 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f14e809ccc0 tx=0x7f14e809cb00 comp rx=0 tx=0).stop 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 -- 172.21.15.44:0/1792240583 shutdown_connections 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea83530 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 --2- 172.21.15.44:0/1792240583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 -- 172.21.15.44:0/1792240583 >> 172.21.15.44:0/1792240583 conn(0x559deea68ab0 msgr2=0x559deea751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.873+0000 7f1503c19740 1 -- 172.21.15.44:0/1792240583 shutdown_connections 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.874+0000 7f1503c19740 1 -- 172.21.15.44:0/1792240583 wait complete. 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.874+0000 7f1503c19740 1 -- start start 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.874+0000 7f1503c19740 1 Processor -- start 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.874+0000 7f1503c19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea7fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f1503c19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f1503c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea71810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f1503c19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b11a0 con 0x559deea6eca0 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f1503c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b0ad0 con 0x559deea77880 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f1503c19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1add20 con 0x559deea6e380 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f1503c19740 10 client.0 _ll_register_callbacks cb 0x7f14f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42972/0 (socket says 172.21.15.44:42972) 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14fa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea71810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 -- 172.21.15.44:0/2558462666 learned_addr learned my addr 172.21.15.44:0/2558462666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea7fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 -- 172.21.15.44:0/2558462666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 msgr2=0x559deea7fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea7fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 -- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 msgr2=0x559deea71810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea71810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14f9400640 1 -- 172.21.15.44:0/2558462666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559deea6d670 con 0x559deea6eca0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.875+0000 7f14fa800640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea71810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f14f9400640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea712d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f14e80097d0 tx=0x7f14e809b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41500bb6d7230fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f14dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f14dbe00640 1 -- 172.21.15.44:0/2558462666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e80ba070 con 0x559deea6eca0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f14dbe00640 1 -- 172.21.15.44:0/2558462666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8411+0+0 (secure 0 0 0) 0x7f14e80b6220 con 0x559deea6eca0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e80b6070 con 0x559deea6eca0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f14dbe00640 1 -- 172.21.15.44:0/2558462666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f14e80a10e0 con 0x559deea6eca0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f1503c19740 10 client.17139 fetch_fsmap learned FSMap version 54 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559deeaedcb0 con 0x559deea6eca0 2024-04-20T11:29:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.876+0000 7f14dbe00640 1 -- 172.21.15.44:0/2558462666 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14e80b6070 con 0x559deea6eca0 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f14dbe00640 1 -- 172.21.15.44:0/2558462666 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 54) ==== 8359+0+0 (secure 0 0 0) 0x7f14e807bb50 con 0x559deea6eca0 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f1503c19740 10 client.17139 fetch_fsmap finished waiting for FSMap version 54 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f1503c19740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f1503c19740 20 client.17139 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f1503c19740 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deeaee410 0x559deeaf0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f1503c19740 4 client.17139 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559dee9db9e0 con 0x559deeaee410 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.877+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deeaee410 0x559deeaf0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f14f9e00640 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deeaee410 0x559deeaf0830 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c03e8e5f9764d1 server_cookie=73004e70663bfc3a in_seq=0 out_seq=0 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f14dbe00640 10 client.17139 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f14dbe00640 1 -- 172.21.15.44:0/2558462666 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559dee9db9e0 con 0x559deeaee410 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f14dbe00640 10 client.17139 handle_command_reply: tid=0 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 client.17139 shutdown 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 msgr2=0x559deea712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea712d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f14e80097d0 tx=0x7f14e809b240 comp rx=0 tx=0).stop 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 shutdown_connections 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559deeaee410 0x559deeaf0830 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559deea77880 0x559deea71810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559deea6eca0 0x559deea712d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 --2- 172.21.15.44:0/2558462666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559deea6e380 0x559deea7fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f14dbe00640 0 client.17139 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 >> 172.21.15.44:0/2558462666 conn(0x559deea68ab0 msgr2=0x559deea69db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 shutdown_connections 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 1 -- 172.21.15.44:0/2558462666 wait complete. 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.878+0000 7f1503c19740 20 client.17139 trim_cache size 0 max 16384 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "a6db97ea-dd10-4bb4-bef8-59dc040bdd48": { 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "a6db97ea-dd10-4bb4-bef8-59dc040bdd48", 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:29:00.883 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:29:00.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14e00a3d20 msgr2=0x7f14e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14e00a3d20 0x7f14e00a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14fc19b9b0 tx=0x7f14ec005fd0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 msgr2=0x7f14fc1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:00.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 0x7f14fc1a0c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9c0 tx=0x7f14e800ee80 comp rx=0 tx=0).stop 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 shutdown_connections 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14fc19a870 0x7f14fc19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f14e00a3d20 0x7f14e00a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14fc107150 0x7f14fc1a0c00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 --2- 172.21.15.44:0/130380325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14fc102080 0x7f14fc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 >> 172.21.15.44:0/130380325 conn(0x7f14fc0fd900 msgr2=0x7f14fc0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 shutdown_connections 2024-04-20T11:29:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:00.879+0000 7f14fb200640 1 -- 172.21.15.44:0/130380325 wait complete. 2024-04-20T11:29:00.898 INFO:tasks.cephfs.filesystem:scrub status for tag:a6db97ea-dd10-4bb4-bef8-59dc040bdd48 - {'path': '/', 'tag': 'a6db97ea-dd10-4bb4-bef8-59dc040bdd48', 'options': 'recursive,force'} 2024-04-20T11:29:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:01 smithi078 ceph-mon[21464]: pgmap v195: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:01.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:01 smithi044 ceph-mon[16855]: pgmap v195: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:01 smithi165 ceph-mon[21975]: pgmap v195: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:02.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:02 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:29:02.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:02.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:02 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:29:02.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:02 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:02 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:29:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:02 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:29:02.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:02 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:02 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:29:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:02 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:29:02.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:02 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:03.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:03 smithi078 ceph-mon[21464]: pgmap v196: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-20T11:29:03.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:03 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:29:03.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:03 smithi044 ceph-mon[16855]: pgmap v196: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-20T11:29:03.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:03 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:29:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:03 smithi165 ceph-mon[21975]: pgmap v196: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-20T11:29:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:03 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:29:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:04 smithi078 ceph-mon[21464]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:29:04.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:04 smithi044 ceph-mon[16855]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:29:04.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:04 smithi165 ceph-mon[21975]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-20T11:29:05.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:05 smithi078 ceph-mon[21464]: pgmap v197: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-20T11:29:05.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:05 smithi044 ceph-mon[16855]: pgmap v197: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-20T11:29:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:05 smithi165 ceph-mon[21975]: pgmap v197: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-20T11:29:07.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:07 smithi078 ceph-mon[21464]: pgmap v198: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-20T11:29:07.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:07 smithi044 ceph-mon[16855]: pgmap v198: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-20T11:29:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:07 smithi165 ceph-mon[21975]: pgmap v198: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-20T11:29:09.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:09 smithi078 ceph-mon[21464]: pgmap v199: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-20T11:29:09.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:09 smithi078 ceph-mon[21464]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T11:29:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:09 smithi044 ceph-mon[16855]: pgmap v199: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-20T11:29:09.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:09 smithi044 ceph-mon[16855]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T11:29:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:09 smithi165 ceph-mon[21975]: pgmap v199: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-20T11:29:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:09 smithi165 ceph-mon[21975]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T11:29:11.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:11 smithi078 ceph-mon[21464]: pgmap v201: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-20T11:29:11.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:11 smithi044 ceph-mon[16855]: pgmap v201: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-20T11:29:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:11 smithi165 ceph-mon[21975]: pgmap v201: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-20T11:29:13.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:13 smithi078 ceph-mon[21464]: pgmap v202: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:29:13.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:13 smithi078 ceph-mon[21464]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T11:29:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:13 smithi044 ceph-mon[16855]: pgmap v202: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:29:13.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:13 smithi044 ceph-mon[16855]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T11:29:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:13 smithi165 ceph-mon[21975]: pgmap v202: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:29:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:13 smithi165 ceph-mon[21975]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T11:29:15.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:15 smithi044 ceph-mon[16855]: pgmap v204: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:29:15.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:15 smithi165 ceph-mon[21975]: pgmap v204: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:29:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:15 smithi078 ceph-mon[21464]: pgmap v204: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:29:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:17 smithi165 ceph-mon[21975]: pgmap v205: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-20T11:29:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:17 smithi078 ceph-mon[21464]: pgmap v205: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-20T11:29:17.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:17 smithi044 ceph-mon[16855]: pgmap v205: 97 pgs: 97 active+clean; 267 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-20T11:29:19.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:19 smithi165 ceph-mon[21975]: pgmap v206: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-20T11:29:19.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:19 smithi078 ceph-mon[21464]: pgmap v206: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-20T11:29:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:19 smithi044 ceph-mon[16855]: pgmap v206: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2024-04-20T11:29:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:21 smithi165 ceph-mon[21975]: pgmap v207: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:29:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:21 smithi078 ceph-mon[21464]: pgmap v207: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:29:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:21 smithi044 ceph-mon[16855]: pgmap v207: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:29:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:23 smithi165 ceph-mon[21975]: pgmap v208: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:29:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:23 smithi078 ceph-mon[21464]: pgmap v208: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:29:23.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:23 smithi044 ceph-mon[16855]: pgmap v208: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:29:25.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:25 smithi165 ceph-mon[21975]: pgmap v209: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:29:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:25 smithi078 ceph-mon[21464]: pgmap v209: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:29:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:25 smithi044 ceph-mon[16855]: pgmap v209: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:29:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:27 smithi078 ceph-mon[21464]: pgmap v210: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:29:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:27 smithi044 ceph-mon[16855]: pgmap v210: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:29:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:27 smithi165 ceph-mon[21975]: pgmap v210: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:29:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:29 smithi078 ceph-mon[21464]: pgmap v211: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:29:29.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:29 smithi044 ceph-mon[16855]: pgmap v211: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:29:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:29 smithi165 ceph-mon[21975]: pgmap v211: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:29:30.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:29:30.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:29:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 -- 172.21.15.44:0/1993844685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 msgr2=0x7fcc08108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1993844685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 0x7fcc08108960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009860 tx=0x7fcbf002bc10 comp rx=0 tx=0).stop 2024-04-20T11:29:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 -- 172.21.15.44:0/1993844685 shutdown_connections 2024-04-20T11:29:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1993844685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc08108ea0 0x7fcc081152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1993844685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 0x7fcc08108960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1993844685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0810d3d0 0x7fcc0810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 -- 172.21.15.44:0/1993844685 >> 172.21.15.44:0/1993844685 conn(0x7fcc08077d80 msgr2=0x7fcc08106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 -- 172.21.15.44:0/1993844685 shutdown_connections 2024-04-20T11:29:31.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.126+0000 7fcc0d600640 1 -- 172.21.15.44:0/1993844685 wait complete. 2024-04-20T11:29:31.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.127+0000 7fcc0d600640 1 Processor -- start 2024-04-20T11:29:31.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 -- start start 2024-04-20T11:29:31.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 0x7fcc0819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 0x7fcc0819f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc0810d3d0 0x7fcc081a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08118940 con 0x7fcc08108ea0 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081187c0 con 0x7fcc0810d3d0 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08118ac0 con 0x7fcc08108500 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 0x7fcc0819f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 0x7fcc0819f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51300/0 (socket says 172.21.15.44:51300) 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcbffe00640 1 -- 172.21.15.44:0/1535129302 learned_addr learned my addr 172.21.15.44:0/1535129302 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc0810d3d0 0x7fcc081a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.128+0000 7fcc07e00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 0x7fcc0819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcbffe00640 1 -- 172.21.15.44:0/1535129302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 msgr2=0x7fcc0819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcbffe00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 0x7fcc0819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcbffe00640 1 -- 172.21.15.44:0/1535129302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc0810d3d0 msgr2=0x7fcc081a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcbffe00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc0810d3d0 0x7fcc081a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcbffe00640 1 -- 172.21.15.44:0/1535129302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc081a3df0 con 0x7fcc08108ea0 2024-04-20T11:29:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc0810d3d0 0x7fcc081a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcbffe00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 0x7fcc0819f340 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00057c0 tx=0x7fcbf002c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcc05600640 1 -- 172.21.15.44:0/1535129302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0041020 con 0x7fcc08108ea0 2024-04-20T11:29:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.129+0000 7fcc05600640 1 -- 172.21.15.44:0/1535129302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcbf0034940 con 0x7fcc08108ea0 2024-04-20T11:29:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.130+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc081a4080 con 0x7fcc08108ea0 2024-04-20T11:29:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.130+0000 7fcc05600640 1 -- 172.21.15.44:0/1535129302 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0039040 con 0x7fcc08108ea0 2024-04-20T11:29:31.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.130+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc081ab970 con 0x7fcc08108ea0 2024-04-20T11:29:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.133+0000 7fcc11854740 1 Processor -- start 2024-04-20T11:29:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.133+0000 7fcc11854740 1 -- start start 2024-04-20T11:29:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.133+0000 7fcc05600640 1 -- 172.21.15.44:0/1535129302 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcbf0044020 con 0x7fcc08108ea0 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc11854740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d6081fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc11854740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081faca0 0x55d608203340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc11854740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608205c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc11854740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d608160150 con 0x55d6081fa380 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc11854740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d608106370 con 0x55d608203880 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc11854740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d60815c110 con 0x55d6081faca0 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d6081fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d6081fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51304/0 (socket says 172.21.15.44:51304) 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 -- 172.21.15.44:0/3849585210 learned_addr learned my addr 172.21.15.44:0/3849585210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcbffe00640 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608205c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc07e00640 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081faca0 0x55d608203340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 -- 172.21.15.44:0/3849585210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081faca0 msgr2=0x55d608203340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081faca0 0x55d608203340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 -- 172.21.15.44:0/3849585210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 msgr2=0x55d608205c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608205c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.134+0000 7fcc0cc00640 1 -- 172.21.15.44:0/3849585210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d607f3c7d0 con 0x55d6081fa380 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d6081fa760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc000117f0 tx=0x7fcc00011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbebbe0d34694a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcc05600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcbec0a3d20 0x7fcbec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcbfd200640 1 -- 172.21.15.44:0/3849585210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc000112f0 con 0x55d6081fa380 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcbfd200640 1 -- 172.21.15.44:0/3849585210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc0000f9a0 con 0x55d6081fa380 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcbfd200640 1 -- 172.21.15.44:0/3849585210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00004610 con 0x55d6081fa380 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcc05600640 1 -- 172.21.15.44:0/1535129302 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fcbf00f1cd0 con 0x7fcc08108ea0 2024-04-20T11:29:31.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.135+0000 7fcc07e00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcbec0a3d20 0x7fcbec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 -- 172.21.15.44:0/3849585210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 msgr2=0x55d6081fa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d6081fa760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc000117f0 tx=0x7fcc00011cb0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 -- 172.21.15.44:0/3849585210 shutdown_connections 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608205c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081faca0 0x55d608203340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 --2- 172.21.15.44:0/3849585210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d6081fa760 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 -- 172.21.15.44:0/3849585210 >> 172.21.15.44:0/3849585210 conn(0x55d6081f4ab0 msgr2=0x55d6081f4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc07e00640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcbec0a3d20 0x7fcbec0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55d6081f9e50 tx=0x7fcbf8009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 -- 172.21.15.44:0/3849585210 shutdown_connections 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.136+0000 7fcc11854740 1 -- 172.21.15.44:0/3849585210 wait complete. 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 -- start start 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 Processor -- start 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081fa380 0x55d608200980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d608200ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608201400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d608206670 con 0x55d6081faca0 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d608119740 con 0x55d608203880 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.137+0000 7fcc11854740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d6082067f0 con 0x55d6081fa380 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcc11854740 10 client.0 _ll_register_callbacks cb 0x55d608114b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d608200ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d608200ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51320/0 (socket says 172.21.15.44:51320) 2024-04-20T11:29:31.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 -- 172.21.15.44:0/1925999405 learned_addr learned my addr 172.21.15.44:0/1925999405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608201400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcc07e00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081fa380 0x55d608200980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 -- 172.21.15.44:0/1925999405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081fa380 msgr2=0x55d608200980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081fa380 0x55d608200980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 -- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 msgr2=0x55d608201400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608201400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcbffe00640 1 -- 172.21.15.44:0/1925999405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d607e723f0 con 0x55d6081faca0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.138+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608201400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcbffe00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d608200ec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcbf002c220 tx=0x7fcbf00f5370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350a9d90c3b3789f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcbf5600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcbf5600640 1 -- 172.21.15.44:0/1925999405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00ba000 con 0x55d6081faca0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcbf5600640 1 -- 172.21.15.44:0/1925999405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fcbf00ef680 con 0x55d6081faca0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0000f9a0 con 0x55d6081faca0 2024-04-20T11:29:31.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcbf5600640 1 -- 172.21.15.44:0/1925999405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcbf00bd6c0 con 0x55d6081faca0 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcc11854740 10 client.17151 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d607e320f0 con 0x55d6081faca0 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.139+0000 7fcbf5600640 1 -- 172.21.15.44:0/1925999405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc0000f9a0 con 0x55d6081faca0 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcbf5600640 1 -- 172.21.15.44:0/1925999405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fcbf003c030 con 0x55d6081faca0 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc11854740 10 client.17151 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc11854740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc11854740 20 client.17151 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc11854740 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d608213c70 0x55d6082160b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc11854740 4 client.17151 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d60816db70 con 0x55d608213c70 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.140+0000 7fcc07e00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d608213c70 0x55d6082160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.141+0000 7fcc07e00640 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d608213c70 0x55d6082160b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526aab6b688eeab6 server_cookie=fb120405cb480f68 in_seq=0 out_seq=0 2024-04-20T11:29:31.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.141+0000 7fcbf5600640 10 client.17151 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:31.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcbf5600640 1 -- 172.21.15.44:0/1925999405 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d60816db70 con 0x55d608213c70 2024-04-20T11:29:31.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcbf5600640 10 client.17151 handle_command_reply: tid=0 2024-04-20T11:29:31.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 client.17151 shutdown 2024-04-20T11:29:31.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 msgr2=0x55d608200ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d608200ec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcbf002c220 tx=0x7fcbf00f5370 comp rx=0 tx=0).stop 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 shutdown_connections 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d608213c70 0x55d6082160b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d608203880 0x55d608201400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d608200ec0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 --2- 172.21.15.44:0/1925999405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6081fa380 0x55d608200980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 >> 172.21.15.44:0/1925999405 conn(0x55d6081f4ab0 msgr2=0x55d6081f6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcbf5600640 0 client.17151 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 shutdown_connections 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.142+0000 7fcc11854740 1 -- 172.21.15.44:0/1925999405 wait complete. 2024-04-20T11:29:31.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.143+0000 7fcc11854740 20 client.17151 trim_cache size 0 max 16384 2024-04-20T11:29:31.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.166+0000 7fcc11854740 1 Processor -- start 2024-04-20T11:29:31.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.166+0000 7fcc11854740 1 -- start start 2024-04-20T11:29:31.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc11854740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d60820f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc11854740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081faca0 0x55d608203040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc11854740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d6081ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc11854740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b4120 con 0x55d6081fa380 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc11854740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b5c60 con 0x55d6081faca0 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc11854740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b47f0 con 0x55d608203880 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d60820f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081faca0 0x55d608203040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc0cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d6081ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d60820f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51324/0 (socket says 172.21.15.44:51324) 2024-04-20T11:29:31.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.167+0000 7fcc07e00640 1 -- 172.21.15.44:0/872767578 learned_addr learned my addr 172.21.15.44:0/872767578 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbffe00640 1 -- 172.21.15.44:0/872767578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 msgr2=0x55d6081ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbffe00640 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d6081ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbffe00640 1 -- 172.21.15.44:0/872767578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 msgr2=0x55d60820f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbffe00640 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d60820f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbffe00640 1 -- 172.21.15.44:0/872767578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d608216730 con 0x55d6081faca0 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbffe00640 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081faca0 0x55d608203040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00ba6e0 tx=0x7fcbf0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ae83d51debd640 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbf7e00640 1 -- 172.21.15.44:0/872767578 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00bd8a0 con 0x55d6081faca0 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbf7e00640 1 -- 172.21.15.44:0/872767578 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcbf00be460 con 0x55d6081faca0 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.168+0000 7fcbf7e00640 1 -- 172.21.15.44:0/872767578 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00bdb30 con 0x55d6081faca0 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 -- 172.21.15.44:0/872767578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081faca0 msgr2=0x55d608203040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081faca0 0x55d608203040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00ba6e0 tx=0x7fcbf0005f90 comp rx=0 tx=0).stop 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 -- 172.21.15.44:0/872767578 shutdown_connections 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d6081ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081faca0 0x55d608203040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 --2- 172.21.15.44:0/872767578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081fa380 0x55d60820f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 -- 172.21.15.44:0/872767578 >> 172.21.15.44:0/872767578 conn(0x55d6081f4ab0 msgr2=0x55d6081f3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 -- 172.21.15.44:0/872767578 shutdown_connections 2024-04-20T11:29:31.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.169+0000 7fcc11854740 1 -- 172.21.15.44:0/872767578 wait complete. 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 -- start start 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 Processor -- start 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081fa380 0x55d60820baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d60820bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d608208a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b5590 con 0x55d6081faca0 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b4ec0 con 0x55d6081fa380 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b2110 con 0x55d608203880 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.170+0000 7fcc11854740 10 client.0 _ll_register_callbacks cb 0x7fcbf00f5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d60820bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081fa380 0x55d60820baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d60820bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51334/0 (socket says 172.21.15.44:51334) 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcbffe00640 1 -- 172.21.15.44:0/3001310874 learned_addr learned my addr 172.21.15.44:0/3001310874 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc0cc00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d608208a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc07e00640 1 -- 172.21.15.44:0/3001310874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 msgr2=0x55d608208a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc07e00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d608208a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc07e00640 1 -- 172.21.15.44:0/3001310874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 msgr2=0x55d60820bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc07e00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d60820bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcc07e00640 1 -- 172.21.15.44:0/3001310874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d608209030 con 0x55d6081fa380 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.171+0000 7fcbffe00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d60820bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcc07e00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081fa380 0x55d60820baa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8004250 tx=0x7fcbf80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a150b8416fa13872 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcbf5600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcbf5600640 1 -- 172.21.15.44:0/3001310874 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf802e070 con 0x55d6081fa380 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcbf5600640 1 -- 172.21.15.44:0/3001310874 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fcbf8003bc0 con 0x55d6081fa380 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf00be460 con 0x55d6081fa380 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcbf5600640 1 -- 172.21.15.44:0/3001310874 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcbf8003dc0 con 0x55d6081fa380 2024-04-20T11:29:31.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcc11854740 10 client.27281 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.172+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d60827ab50 con 0x55d6081fa380 2024-04-20T11:29:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcbf5600640 1 -- 172.21.15.44:0/3001310874 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcbf00be460 con 0x55d6081fa380 2024-04-20T11:29:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcbf5600640 1 -- 172.21.15.44:0/3001310874 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fcbf801eb00 con 0x55d6081fa380 2024-04-20T11:29:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcc11854740 10 client.27281 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcc11854740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcc11854740 20 client.27281 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:31.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcc11854740 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d60827b230 0x55d60827d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcc11854740 4 client.27281 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6081679e0 con 0x55d60827b230 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.173+0000 7fcbffe00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d60827b230 0x55d60827d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcbffe00640 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d60827b230 0x55d60827d650 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9871de1c584160 server_cookie=deeb718e6b2e33d7 in_seq=0 out_seq=0 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcbf5600640 10 client.27281 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcbf5600640 1 -- 172.21.15.44:0/3001310874 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d6081679e0 con 0x55d60827b230 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcbf5600640 10 client.27281 handle_command_reply: tid=0 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 client.27281 shutdown 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081fa380 msgr2=0x55d60820baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081fa380 0x55d60820baa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8004250 tx=0x7fcbf80046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 shutdown_connections 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55d60827b230 0x55d60827d650 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d608203880 0x55d608208a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d6081faca0 0x55d60820bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 --2- 172.21.15.44:0/3001310874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6081fa380 0x55d60820baa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcbf5600640 0 client.27281 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 >> 172.21.15.44:0/3001310874 conn(0x55d6081f4ab0 msgr2=0x55d608200940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 shutdown_connections 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.174+0000 7fcc11854740 1 -- 172.21.15.44:0/3001310874 wait complete. 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc11854740 20 client.27281 trim_cache size 0 max 16384 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:29:31.177 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcbec0a3d20 msgr2=0x7fcbec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcbec0a3d20 0x7fcbec0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55d6081f9e50 tx=0x7fcbf8009260 comp rx=0 tx=0).stop 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 msgr2=0x7fcc0819f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 0x7fcc0819f340 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00057c0 tx=0x7fcbf002c630 comp rx=0 tx=0).stop 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 shutdown_connections 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc0810d3d0 0x7fcc081a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcbec0a3d20 0x7fcbec0a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc08108ea0 0x7fcc0819f340 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 --2- 172.21.15.44:0/1535129302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc08108500 0x7fcc0819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.175+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 >> 172.21.15.44:0/1535129302 conn(0x7fcc08077d80 msgr2=0x7fcc080783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.176+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 shutdown_connections 2024-04-20T11:29:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.176+0000 7fcc0d600640 1 -- 172.21.15.44:0/1535129302 wait complete. 2024-04-20T11:29:31.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:29:31.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 -- 172.21.15.44:0/2659426304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb481092c0 msgr2=0x7fdb48110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2659426304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb481092c0 0x7fdb48110e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009be0 tx=0x7fdb4002bef0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 -- 172.21.15.44:0/2659426304 shutdown_connections 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2659426304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb481092c0 0x7fdb48110e90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2659426304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb48104ba0 0x7fdb48108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2659426304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb481041f0 0x7fdb481045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.413+0000 7fdb4d800640 1 -- 172.21.15.44:0/2659426304 >> 172.21.15.44:0/2659426304 conn(0x7fdb480ffa30 msgr2=0x7fdb48101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.414+0000 7fdb4d800640 1 -- 172.21.15.44:0/2659426304 shutdown_connections 2024-04-20T11:29:31.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.414+0000 7fdb4d800640 1 -- 172.21.15.44:0/2659426304 wait complete. 2024-04-20T11:29:31.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.415+0000 7fdb4d800640 1 Processor -- start 2024-04-20T11:29:31.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.415+0000 7fdb4d800640 1 -- start start 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 0x7fdb481a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb481092c0 0x7fdb481a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4819a7a0 0x7fdb4819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48114500 con 0x7fdb481041f0 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4d800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48114380 con 0x7fdb4819a7a0 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48114680 con 0x7fdb481092c0 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 0x7fdb481a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 0x7fdb481a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51344/0 (socket says 172.21.15.44:51344) 2024-04-20T11:29:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb47e00640 1 -- 172.21.15.44:0/2499165638 learned_addr learned my addr 172.21.15.44:0/2499165638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb47400640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb481092c0 0x7fdb481a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb4ce00640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4819a7a0 0x7fdb4819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb47e00640 1 -- 172.21.15.44:0/2499165638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb481092c0 msgr2=0x7fdb481a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.416+0000 7fdb47e00640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb481092c0 0x7fdb481a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb47e00640 1 -- 172.21.15.44:0/2499165638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4819a7a0 msgr2=0x7fdb4819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb47e00640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4819a7a0 0x7fdb4819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb47e00640 1 -- 172.21.15.44:0/2499165638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4819b3a0 con 0x7fdb481041f0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb47e00640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 0x7fdb481a0690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9f0 tx=0x7fdb3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800cd80 con 0x7fdb481041f0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb38005520 con 0x7fdb481041f0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb481a7430 con 0x7fdb481041f0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38010690 con 0x7fdb481041f0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.417+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a77e0 con 0x7fdb481041f0 2024-04-20T11:29:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.419+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb481056c0 con 0x7fdb481041f0 2024-04-20T11:29:31.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.421+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdb380046c0 con 0x7fdb481041f0 2024-04-20T11:29:31.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.423+0000 7fdb44c00640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb2c0a3d20 0x7fdb2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.423+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fdb380d0e30 con 0x7fdb481041f0 2024-04-20T11:29:31.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.423+0000 7fdb47400640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb2c0a3d20 0x7fdb2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.424+0000 7fdb47400640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb2c0a3d20 0x7fdb2c0a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb340096c0 tx=0x7fdb34009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.425+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdb38053ff0 con 0x7fdb481041f0 2024-04-20T11:29:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.676+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb48068ca0 con 0x7fdb481041f0 2024-04-20T11:29:31.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.678+0000 7fdb44c00640 1 -- 172.21.15.44:0/2499165638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fdb3809d4b0 con 0x7fdb481041f0 2024-04-20T11:29:31.677 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:29:31.678 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:29:31.678 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:29:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.680+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb2c0a3d20 msgr2=0x7fdb2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.680+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb2c0a3d20 0x7fdb2c0a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb340096c0 tx=0x7fdb34009260 comp rx=0 tx=0).stop 2024-04-20T11:29:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.680+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 msgr2=0x7fdb481a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.680+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 0x7fdb481a0690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9f0 tx=0x7fdb3800edf0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.680+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 shutdown_connections 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4819a7a0 0x7fdb4819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb481092c0 0x7fdb481a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdb2c0a3d20 0x7fdb2c0a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 --2- 172.21.15.44:0/2499165638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb481041f0 0x7fdb481a0690 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 >> 172.21.15.44:0/2499165638 conn(0x7fdb480ffa30 msgr2=0x7fdb48100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 shutdown_connections 2024-04-20T11:29:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.681+0000 7fdb4d800640 1 -- 172.21.15.44:0/2499165638 wait complete. 2024-04-20T11:29:31.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:31 smithi044 ceph-mon[16855]: pgmap v212: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:31.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:29:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 -- 172.21.15.44:0/1826663347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104dc0 msgr2=0x7fe734108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 --2- 172.21.15.44:0/1826663347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104dc0 0x7fe734108e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009900 tx=0x7fe71c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:29:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 -- 172.21.15.44:0/1826663347 shutdown_connections 2024-04-20T11:29:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 --2- 172.21.15.44:0/1826663347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 --2- 172.21.15.44:0/1826663347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104dc0 0x7fe734108e00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 --2- 172.21.15.44:0/1826663347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe734104410 0x7fe7341047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 -- 172.21.15.44:0/1826663347 >> 172.21.15.44:0/1826663347 conn(0x7fe7340ffcf0 msgr2=0x7fe734102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 -- 172.21.15.44:0/1826663347 shutdown_connections 2024-04-20T11:29:31.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.921+0000 7fe739800640 1 -- 172.21.15.44:0/1826663347 wait complete. 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.922+0000 7fe739800640 1 Processor -- start 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.922+0000 7fe739800640 1 -- start start 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe739800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104410 0x7fe7341a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe739800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe739800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe73419aa90 0x7fe73419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe739800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341146b0 con 0x7fe73419aa90 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe739800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe734114530 con 0x7fe7341094e0 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe739800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe734114830 con 0x7fe734104410 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe72a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe72a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:60994/0 (socket says 172.21.15.44:60994) 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe72a000640 1 -- 172.21.15.44:0/1208078803 learned_addr learned my addr 172.21.15.44:0/1208078803 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe733400640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe73419aa90 0x7fe73419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.923+0000 7fe732a00640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104410 0x7fe7341a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72a000640 1 -- 172.21.15.44:0/1208078803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104410 msgr2=0x7fe7341a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72a000640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104410 0x7fe7341a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72a000640 1 -- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe73419aa90 msgr2=0x7fe73419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72a000640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe73419aa90 0x7fe73419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72a000640 1 -- 172.21.15.44:0/1208078803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73419b5b0 con 0x7fe7341094e0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72a000640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341a0e20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe71c000c00 tx=0x7fe71c02dca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72be00640 1 -- 172.21.15.44:0/1208078803 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c043020 con 0x7fe7341094e0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72be00640 1 -- 172.21.15.44:0/1208078803 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe71c004250 con 0x7fe7341094e0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe72be00640 1 -- 172.21.15.44:0/1208078803 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c003970 con 0x7fe7341094e0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73419b840 con 0x7fe7341094e0 2024-04-20T11:29:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.924+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7341a7a90 con 0x7fe7341094e0 2024-04-20T11:29:31.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 Processor -- start 2024-04-20T11:29:31.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe72be00640 1 -- 172.21.15.44:0/1208078803 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe71c046020 con 0x7fe7341094e0 2024-04-20T11:29:31.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 -- start start 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 0x559e5d7f6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d7fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d8018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e5d75b050 con 0x559e5d7f68d0 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e5d701490 con 0x559e5d7ff4b0 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.928+0000 7fe73c59f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559e5d757010 con 0x559e5d7f5fb0 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 0x559e5d7f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe732a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d7fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 0x559e5d7f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59418/0 (socket says 172.21.15.44:59418) 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 -- 172.21.15.44:0/1778882402 learned_addr learned my addr 172.21.15.44:0/1778882402 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe72a000640 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d8018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 -- 172.21.15.44:0/1778882402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 msgr2=0x559e5d8018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d8018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 -- 172.21.15.44:0/1778882402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 msgr2=0x559e5d7fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d7fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 -- 172.21.15.44:0/1778882402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5d709070 con 0x559e5d7f5fb0 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe72a000640 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d8018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe72be00640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7180a3d20 0x7fe7180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe733400640 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 0x559e5d7f6390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009f60 tx=0x7fe72c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1e24a6d5782a1df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.929+0000 7fe72be00640 1 -- 172.21.15.44:0/1208078803 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fe71c0f25f0 con 0x7fe7341094e0 2024-04-20T11:29:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe732a00640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7180a3d20 0x7fe7180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe723e00640 1 -- 172.21.15.44:0/1778882402 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c01e070 con 0x559e5d7f5fb0 2024-04-20T11:29:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe723e00640 1 -- 172.21.15.44:0/1778882402 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe72c003c30 con 0x559e5d7f5fb0 2024-04-20T11:29:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe723e00640 1 -- 172.21.15.44:0/1778882402 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00fa70 con 0x559e5d7f5fb0 2024-04-20T11:29:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe73c59f740 1 -- 172.21.15.44:0/1778882402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 msgr2=0x559e5d7f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe732a00640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7180a3d20 0x7fe7180a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x559e5d7f5a80 tx=0x7fe724005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.930+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 0x559e5d7f6390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009f60 tx=0x7fe72c009f90 comp rx=0 tx=0).stop 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 -- 172.21.15.44:0/1778882402 shutdown_connections 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d8018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d7fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1778882402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f5fb0 0x559e5d7f6390 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 -- 172.21.15.44:0/1778882402 >> 172.21.15.44:0/1778882402 conn(0x559e5d7f06e0 msgr2=0x559e5d7f0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 -- 172.21.15.44:0/1778882402 shutdown_connections 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 -- 172.21.15.44:0/1778882402 wait complete. 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.931+0000 7fe73c59f740 1 -- start start 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 Processor -- start 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d7fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 0x559e5d7fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e5d8022a0 con 0x559e5d7ff4b0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e5d714860 con 0x559e5d7f5fb0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559e5d802420 con 0x559e5d7f68d0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe73c59f740 10 client.0 _ll_register_callbacks cb 0x559e5d6e2fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.932+0000 7fe733400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 0x559e5d7fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 0x559e5d7fd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51376/0 (socket says 172.21.15.44:51376) 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe732a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 -- 172.21.15.44:0/474695615 learned_addr learned my addr 172.21.15.44:0/474695615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe72a000640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d7fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 -- 172.21.15.44:0/474695615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 msgr2=0x559e5d7fcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d7fcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 -- 172.21.15.44:0/474695615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 msgr2=0x559e5d7fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 -- 172.21.15.44:0/474695615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5d718950 con 0x559e5d7ff4b0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe732a00640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe733400640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 0x559e5d7fd020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009b70 tx=0x7fe72c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7afdf2ff8184a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.933+0000 7fe720c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe720c00640 1 -- 172.21.15.44:0/474695615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c027020 con 0x559e5d7ff4b0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe720c00640 1 -- 172.21.15.44:0/474695615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fe72c004e00 con 0x559e5d7ff4b0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c003c30 con 0x559e5d7ff4b0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe720c00640 1 -- 172.21.15.44:0/474695615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe72c032050 con 0x559e5d7ff4b0 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe73c59f740 10 client.17175 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559e5d6e6820 con 0x559e5d7ff4b0 2024-04-20T11:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe720c00640 1 -- 172.21.15.44:0/474695615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe72c003c30 con 0x559e5d7ff4b0 2024-04-20T11:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.934+0000 7fe720c00640 1 -- 172.21.15.44:0/474695615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe72c005040 con 0x559e5d7ff4b0 2024-04-20T11:29:31.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe73c59f740 10 client.17175 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe73c59f740 10 client.17175 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe73c59f740 20 client.17175 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe73c59f740 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d80f500 0x559e5d8118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe73c59f740 4 client.17175 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e5d768a70 con 0x559e5d80f500 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.935+0000 7fe732a00640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d80f500 0x559e5d8118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.936+0000 7fe732a00640 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d80f500 0x559e5d8118f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75d4c57df0925188 server_cookie=ad82274c41bdff22 in_seq=0 out_seq=0 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.936+0000 7fe720c00640 10 client.17175 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.936+0000 7fe720c00640 1 -- 172.21.15.44:0/474695615 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e5d768a70 con 0x559e5d80f500 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.936+0000 7fe720c00640 10 client.17175 handle_command_reply: tid=0 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.936+0000 7fe73c59f740 1 client.17175 shutdown 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 msgr2=0x559e5d7fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 0x559e5d7fd020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009b70 tx=0x7fe72c004840 comp rx=0 tx=0).stop 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 shutdown_connections 2024-04-20T11:29:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7ff4b0 0x559e5d7fd020 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d7fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d80f500 0x559e5d8118f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 --2- 172.21.15.44:0/474695615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 >> 172.21.15.44:0/474695615 conn(0x559e5d7f06e0 msgr2=0x559e5d7f2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe720c00640 0 client.17175 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 shutdown_connections 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 1 -- 172.21.15.44:0/474695615 wait complete. 2024-04-20T11:29:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.937+0000 7fe73c59f740 20 client.17175 trim_cache size 0 max 16384 2024-04-20T11:29:31.940 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:31 smithi078 ceph-mon[21464]: pgmap v212: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.952+0000 7fe73c59f740 1 Processor -- start 2024-04-20T11:29:31.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.952+0000 7fe73c59f740 1 -- start start 2024-04-20T11:29:31.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.952+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d80b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe73c59f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341aff60 con 0x559e5d7f5fb0 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe73c59f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341b1aa0 con 0x559e5d7ff4b0 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe73c59f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341b0630 con 0x559e5d7f68d0 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe733400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe732a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe72a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d80b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe733400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32778/0 (socket says 172.21.15.44:32778) 2024-04-20T11:29:31.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe733400640 1 -- 172.21.15.44:0/1680544723 learned_addr learned my addr 172.21.15.44:0/1680544723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe72a000640 1 -- 172.21.15.44:0/1680544723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 msgr2=0x559e5d7f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe72a000640 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe72a000640 1 -- 172.21.15.44:0/1680544723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f5fb0 msgr2=0x559e5d7fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe72a000640 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.953+0000 7fe72a000640 1 -- 172.21.15.44:0/1680544723 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5d46d3f0 con 0x559e5d7f68d0 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.954+0000 7fe732a00640 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.954+0000 7fe72a000640 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d80b160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0efe00 tx=0x7fe71c0a9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43603f0b3a1625e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.954+0000 7fe723400640 1 -- 172.21.15.44:0/1680544723 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0ba920 con 0x559e5d7f68d0 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.954+0000 7fe723400640 1 -- 172.21.15.44:0/1680544723 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe71c0bed80 con 0x559e5d7f68d0 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.954+0000 7fe723400640 1 -- 172.21.15.44:0/1680544723 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0bdfe0 con 0x559e5d7f68d0 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 -- 172.21.15.44:0/1680544723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 msgr2=0x559e5d80b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d80b160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0efe00 tx=0x7fe71c0a9040 comp rx=0 tx=0).stop 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 -- 172.21.15.44:0/1680544723 shutdown_connections 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7f68d0 0x559e5d80b160 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 --2- 172.21.15.44:0/1680544723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f5fb0 0x559e5d7fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 -- 172.21.15.44:0/1680544723 >> 172.21.15.44:0/1680544723 conn(0x559e5d7f06e0 msgr2=0x559e5d7f1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 -- 172.21.15.44:0/1680544723 shutdown_connections 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 -- 172.21.15.44:0/1680544723 wait complete. 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 -- start start 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.955+0000 7fe73c59f740 1 Processor -- start 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d807670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d807bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341b13d0 con 0x559e5d7f68d0 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341b0d00 con 0x559e5d7f5fb0 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341adf50 con 0x559e5d7ff4b0 2024-04-20T11:29:31.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe73c59f740 10 client.0 _ll_register_callbacks cb 0x7fe72c009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe72a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d807bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe72a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d807bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51392/0 (socket says 172.21.15.44:51392) 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe732a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d807670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe72a000640 1 -- 172.21.15.44:0/707648800 learned_addr learned my addr 172.21.15.44:0/707648800 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.956+0000 7fe733400640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe732a00640 1 -- 172.21.15.44:0/707648800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7ff4b0 msgr2=0x559e5d7f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe732a00640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe732a00640 1 -- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 msgr2=0x559e5d807bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe732a00640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d807bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe732a00640 1 -- 172.21.15.44:0/707648800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5d5377d0 con 0x559e5d7f5fb0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe733400640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe732a00640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d807670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x559e5d811ff0 tx=0x7fe7240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af69a447c3a029c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe720c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:31.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe720c00640 1 -- 172.21.15.44:0/707648800 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724005020 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71c0bed80 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.957+0000 7fe720c00640 1 -- 172.21.15.44:0/707648800 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fe7240051c0 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe720c00640 1 -- 172.21.15.44:0/707648800 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe7240053c0 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 10 client.27305 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559e5d87d7a0 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe720c00640 1 -- 172.21.15.44:0/707648800 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe71c0bed80 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe720c00640 1 -- 172.21.15.44:0/707648800 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe72402e270 con 0x559e5d7f5fb0 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 10 client.27305 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 10 client.27305 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 20 client.27305 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d87de70 0x559e5d880290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 4 client.27305 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:29:31.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e5d7628e0 con 0x559e5d87de70 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.958+0000 7fe72a000640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d87de70 0x559e5d880290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe72a000640 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d87de70 0x559e5d880290 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cdf0fd985971bb51 server_cookie=2307aaad0c8e843d in_seq=0 out_seq=0 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe720c00640 10 client.27305 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe720c00640 1 -- 172.21.15.44:0/707648800 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e5d7628e0 con 0x559e5d87de70 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe720c00640 10 client.27305 handle_command_reply: tid=0 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 client.27305 shutdown 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 msgr2=0x559e5d807670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d807670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x559e5d811ff0 tx=0x7fe7240046c0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 shutdown_connections 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e5d7ff4b0 0x559e5d7f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e5d7f68d0 0x559e5d807bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x559e5d87de70 0x559e5d880290 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 --2- 172.21.15.44:0/707648800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e5d7f5fb0 0x559e5d807670 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe720c00640 0 client.27305 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.959+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 >> 172.21.15.44:0/707648800 conn(0x559e5d7f06e0 msgr2=0x559e5d7fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 shutdown_connections 2024-04-20T11:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe73c59f740 1 -- 172.21.15.44:0/707648800 wait complete. 2024-04-20T11:29:31.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe73c59f740 20 client.27305 trim_cache size 0 max 16384 2024-04-20T11:29:31.964 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7180a3d20 msgr2=0x7fe7180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe739800640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7180a3d20 0x7fe7180a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x559e5d7f5a80 tx=0x7fe724005f90 comp rx=0 tx=0).stop 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 msgr2=0x7fe7341a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.960+0000 7fe739800640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341a0e20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe71c000c00 tx=0x7fe71c02dca0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 shutdown_connections 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe7180a3d20 0x7fe7180a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe73419aa90 0x7fe73419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7341094e0 0x7fe7341a0e20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 --2- 172.21.15.44:0/1208078803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe734104410 0x7fe7341a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 >> 172.21.15.44:0/1208078803 conn(0x7fe7340ffcf0 msgr2=0x7fe734101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 shutdown_connections 2024-04-20T11:29:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:31.961+0000 7fe739800640 1 -- 172.21.15.44:0/1208078803 wait complete. 2024-04-20T11:29:31.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:31 smithi165 ceph-mon[21975]: pgmap v212: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.183+0000 7fe0b2000640 1 -- 172.21.15.44:0/3764315246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac102a20 msgr2=0x7fe0ac106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.183+0000 7fe0b2000640 1 --2- 172.21.15.44:0/3764315246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac102a20 0x7fe0ac106a60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe098009900 tx=0x7fe09802bc30 comp rx=0 tx=0).stop 2024-04-20T11:29:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 -- 172.21.15.44:0/3764315246 shutdown_connections 2024-04-20T11:29:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 --2- 172.21.15.44:0/3764315246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0ac107140 0x7fe0ac10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 --2- 172.21.15.44:0/3764315246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac102a20 0x7fe0ac106a60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 --2- 172.21.15.44:0/3764315246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 0x7fe0ac102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 -- 172.21.15.44:0/3764315246 >> 172.21.15.44:0/3764315246 conn(0x7fe0ac0fd930 msgr2=0x7fe0ac0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 -- 172.21.15.44:0/3764315246 shutdown_connections 2024-04-20T11:29:32.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.184+0000 7fe0b2000640 1 -- 172.21.15.44:0/3764315246 wait complete. 2024-04-20T11:29:32.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 Processor -- start 2024-04-20T11:29:32.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 -- start start 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 0x7fe0ac19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 0x7fe0ac19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0ac198600 0x7fe0ac198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1122a0 con 0x7fe0ac107140 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac112120 con 0x7fe0ac102070 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.185+0000 7fe0b2000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac112420 con 0x7fe0ac198600 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 0x7fe0ac19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 0x7fe0ac19e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51404/0 (socket says 172.21.15.44:51404) 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 -- 172.21.15.44:0/65905845 learned_addr learned my addr 172.21.15.44:0/65905845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 0x7fe0ac19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0b1600640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0ac198600 0x7fe0ac198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 -- 172.21.15.44:0/65905845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0ac198600 msgr2=0x7fe0ac198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0ac198600 0x7fe0ac198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 -- 172.21.15.44:0/65905845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 msgr2=0x7fe0ac19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 0x7fe0ac19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 -- 172.21.15.44:0/65905845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac199120 con 0x7fe0ac107140 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 0x7fe0ac19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:29:32.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.186+0000 7fe0abe00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 0x7fe0ac19e990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe098009230 tx=0x7fe098004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.187+0000 7fe0a9600640 1 -- 172.21.15.44:0/65905845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098043020 con 0x7fe0ac107140 2024-04-20T11:29:32.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.187+0000 7fe0a9600640 1 -- 172.21.15.44:0/65905845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe098004e50 con 0x7fe0ac107140 2024-04-20T11:29:32.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.187+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac1993b0 con 0x7fe0ac107140 2024-04-20T11:29:32.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.187+0000 7fe0a9600640 1 -- 172.21.15.44:0/65905845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098003a10 con 0x7fe0ac107140 2024-04-20T11:29:32.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.187+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac1a5600 con 0x7fe0ac107140 2024-04-20T11:29:32.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.190+0000 7fe0b641b740 1 Processor -- start 2024-04-20T11:29:32.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.190+0000 7fe0a9600640 1 -- 172.21.15.44:0/65905845 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe098046020 con 0x7fe0ac107140 2024-04-20T11:29:32.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 -- start start 2024-04-20T11:29:32.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa8cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa9a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa9ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562b4f9f2050 con 0x562b4fa9aa80 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562b4f998490 con 0x562b4fa8d500 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b641b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562b4f9ee010 con 0x562b4fa8cb50 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa9ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa9ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51420/0 (socket says 172.21.15.44:51420) 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0abe00640 1 -- 172.21.15.44:0/1140439813 learned_addr learned my addr 172.21.15.44:0/1140439813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa8cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.191+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa9a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0b1600640 1 -- 172.21.15.44:0/1140439813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 msgr2=0x562b4fa9a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa9a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0b1600640 1 -- 172.21.15.44:0/1140439813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 msgr2=0x562b4fa9ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa9ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0b1600640 1 -- 172.21.15.44:0/1140439813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4f9a0070 con 0x562b4fa8cb50 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0a9600640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe0940a3d20 0x7fe0940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa8cf30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009f00 tx=0x7fe0a4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb3b0f53f7edcb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe0a9600640 1 -- 172.21.15.44:0/65905845 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fe0980f2700 con 0x7fe0ac107140 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.192+0000 7fe09dc00640 1 -- 172.21.15.44:0/1140439813 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004d20 con 0x562b4fa8cb50 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.193+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe0940a3d20 0x7fe0940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.193+0000 7fe09dc00640 1 -- 172.21.15.44:0/1140439813 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0a4004050 con 0x562b4fa8cb50 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.193+0000 7fe09dc00640 1 -- 172.21.15.44:0/1140439813 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4005260 con 0x562b4fa8cb50 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.193+0000 7fe0b641b740 1 -- 172.21.15.44:0/1140439813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 msgr2=0x562b4fa8cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.193+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe0940a3d20 0x7fe0940a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009db0 tx=0x7fe0a0005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.193+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa8cf30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009f00 tx=0x7fe0a4011810 comp rx=0 tx=0).stop 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 -- 172.21.15.44:0/1140439813 shutdown_connections 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa9ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa9a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1140439813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa8cf30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 -- 172.21.15.44:0/1140439813 >> 172.21.15.44:0/1140439813 conn(0x562b4fa878a0 msgr2=0x562b4fa87c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 -- 172.21.15.44:0/1140439813 shutdown_connections 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 -- 172.21.15.44:0/1140439813 wait complete. 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 -- start start 2024-04-20T11:29:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.194+0000 7fe0b641b740 1 Processor -- start 2024-04-20T11:29:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa97ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa98000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa98540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562b4fa9d840 con 0x562b4fa9aa80 2024-04-20T11:29:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562b4f9ab860 con 0x562b4fa8d500 2024-04-20T11:29:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562b4fa9d9c0 con 0x562b4fa8cb50 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b641b740 10 client.0 _ll_register_callbacks cb 0x562b4f979fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa98540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa98540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51426/0 (socket says 172.21.15.44:51426) 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b1600640 1 -- 172.21.15.44:0/1744350536 learned_addr learned my addr 172.21.15.44:0/1744350536 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa97ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.195+0000 7fe0abe00640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b1600640 1 -- 172.21.15.44:0/1744350536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 msgr2=0x562b4fa97ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa97ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b1600640 1 -- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 msgr2=0x562b4fa98000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa98000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b1600640 1 -- 172.21.15.44:0/1744350536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4f9af950 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b1600640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa98540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40044b0 tx=0x7fe0a40044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa54894d9f9025 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe092000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe092000640 1 -- 172.21.15.44:0/1744350536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4029020 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4004050 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.196+0000 7fe092000640 1 -- 172.21.15.44:0/1744350536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fe0a4004d30 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe092000640 1 -- 172.21.15.44:0/1744350536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe0a4034050 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe0b641b740 10 client.17193 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562b4f97d820 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe092000640 1 -- 172.21.15.44:0/1744350536 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0a4004050 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe092000640 1 -- 172.21.15.44:0/1744350536 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe0a40050e0 con 0x562b4fa9aa80 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe0b641b740 10 client.17193 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe0b641b740 10 client.17193 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe0b641b740 20 client.17193 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.197+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4faaaa40 0x562b4faace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.198+0000 7fe0b641b740 4 client.17193 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.198+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4f9ffa70 con 0x562b4faaaa40 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.198+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4faaaa40 0x562b4faace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.199+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4faaaa40 0x562b4faace30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d607e21ae6fe98d6 server_cookie=570f531b81f67859 in_seq=0 out_seq=0 2024-04-20T11:29:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.199+0000 7fe092000640 10 client.17193 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:29:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.199+0000 7fe092000640 1 -- 172.21.15.44:0/1744350536 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b4f9ffa70 con 0x562b4faaaa40 2024-04-20T11:29:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.199+0000 7fe092000640 10 client.17193 handle_command_reply: tid=0 2024-04-20T11:29:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.199+0000 7fe0b641b740 1 client.17193 shutdown 2024-04-20T11:29:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 msgr2=0x562b4fa98540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa98540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40044b0 tx=0x7fe0a40044e0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 shutdown_connections 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4faaaa40 0x562b4faace30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa98540 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa98000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 --2- 172.21.15.44:0/1744350536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4fa97ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe092000640 0 client.17193 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 >> 172.21.15.44:0/1744350536 conn(0x562b4fa878a0 msgr2=0x562b4fa895f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 shutdown_connections 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 1 -- 172.21.15.44:0/1744350536 wait complete. 2024-04-20T11:29:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.200+0000 7fe0b641b740 20 client.17193 trim_cache size 0 max 16384 2024-04-20T11:29:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.215+0000 7fe0b641b740 1 Processor -- start 2024-04-20T11:29:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 -- start start 2024-04-20T11:29:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa983a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fab25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1ab930 con 0x562b4fa9aa80 2024-04-20T11:29:32.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1ad470 con 0x562b4fa8d500 2024-04-20T11:29:32.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b641b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1ac000 con 0x562b4fa8cb50 2024-04-20T11:29:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fab25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fab25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51436/0 (socket says 172.21.15.44:51436) 2024-04-20T11:29:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b1600640 1 -- 172.21.15.44:0/2570289670 learned_addr learned my addr 172.21.15.44:0/2570289670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0abe00640 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.216+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe0b1600640 1 -- 172.21.15.44:0/2570289670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 msgr2=0x562b4faa6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe0b1600640 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe0b1600640 1 -- 172.21.15.44:0/2570289670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 msgr2=0x562b4fa983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe0b1600640 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe0b1600640 1 -- 172.21.15.44:0/2570289670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4f7043f0 con 0x562b4fa9aa80 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe0b1600640 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fab25a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009540 tx=0x7fe0a4004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e991f129329c379 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe09d200640 1 -- 172.21.15.44:0/2570289670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4029040 con 0x562b4fa9aa80 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe09d200640 1 -- 172.21.15.44:0/2570289670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0a4011d10 con 0x562b4fa9aa80 2024-04-20T11:29:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.217+0000 7fe09d200640 1 -- 172.21.15.44:0/2570289670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4003fe0 con 0x562b4fa9aa80 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 -- 172.21.15.44:0/2570289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 msgr2=0x562b4fab25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fab25a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009540 tx=0x7fe0a4004830 comp rx=0 tx=0).stop 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 -- 172.21.15.44:0/2570289670 shutdown_connections 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fab25a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 --2- 172.21.15.44:0/2570289670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 -- 172.21.15.44:0/2570289670 >> 172.21.15.44:0/2570289670 conn(0x562b4fa878a0 msgr2=0x562b4fa89580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 -- 172.21.15.44:0/2570289670 shutdown_connections 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.218+0000 7fe0b641b740 1 -- 172.21.15.44:0/2570289670 wait complete. 2024-04-20T11:29:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 -- start start 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 Processor -- start 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1acda0 con 0x562b4fa9aa80 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1ac6d0 con 0x562b4fa8d500 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1a9940 con 0x562b4fa8cb50 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.219+0000 7fe0b641b740 10 client.0 _ll_register_callbacks cb 0x7fe0a4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa948f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51444/0 (socket says 172.21.15.44:51444) 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 -- 172.21.15.44:0/4226461848 learned_addr learned my addr 172.21.15.44:0/4226461848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0abe00640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 -- 172.21.15.44:0/4226461848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 msgr2=0x562b4faa2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 -- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 msgr2=0x562b4fa943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa943b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 -- 172.21.15.44:0/4226461848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4f7ce7d0 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe0b1600640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa948f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4006010 tx=0x7fe0a40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbabbc02610347b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.220+0000 7fe092000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe092000640 1 -- 172.21.15.44:0/4226461848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a403c070 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe092000640 1 -- 172.21.15.44:0/4226461848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fe0a4003fc0 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4011d10 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe092000640 1 -- 172.21.15.44:0/4226461848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe0a4004b30 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe0b641b740 10 client.17199 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562b4fb10a30 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.221+0000 7fe092000640 1 -- 172.21.15.44:0/4226461848 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0a4011d10 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe092000640 1 -- 172.21.15.44:0/4226461848 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe0a4025c70 con 0x562b4fa9aa80 2024-04-20T11:29:32.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b641b740 10 client.17199 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b641b740 10 client.17199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b641b740 20 client.17199 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b641b740 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4fb11040 0x562b4fb13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b641b740 4 client.17199 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b4f9f98e0 con 0x562b4fb11040 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.222+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4fb11040 0x562b4fb13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b0c00640 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4fb11040 0x562b4fb13460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23c61012317a8332 server_cookie=13b431d067946da2 in_seq=0 out_seq=0 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe092000640 10 client.17199 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe092000640 1 -- 172.21.15.44:0/4226461848 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b4f9f98e0 con 0x562b4fb11040 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe092000640 10 client.17199 handle_command_reply: tid=0 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 client.17199 shutdown 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 msgr2=0x562b4fa948f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa948f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4006010 tx=0x7fe0a40039e0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 shutdown_connections 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x562b4fb11040 0x562b4fb13460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4fa9aa80 0x562b4fa948f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562b4fa8d500 0x562b4fa943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 --2- 172.21.15.44:0/4226461848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b4fa8cb50 0x562b4faa2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe092000640 0 client.17199 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 >> 172.21.15.44:0/4226461848 conn(0x562b4fa878a0 msgr2=0x562b4fa860d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 shutdown_connections 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 1 -- 172.21.15.44:0/4226461848 wait complete. 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.223+0000 7fe0b641b740 20 client.17199 trim_cache size 0 max 16384 2024-04-20T11:29:32.228 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:29:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe0940a3d20 msgr2=0x7fe0940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe0940a3d20 0x7fe0940a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009db0 tx=0x7fe0a0005f90 comp rx=0 tx=0).stop 2024-04-20T11:29:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 msgr2=0x7fe0ac19e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 0x7fe0ac19e990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe098009230 tx=0x7fe098004c30 comp rx=0 tx=0).stop 2024-04-20T11:29:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 shutdown_connections 2024-04-20T11:29:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0ac198600 0x7fe0ac198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe0940a3d20 0x7fe0940a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0ac107140 0x7fe0ac19e990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 --2- 172.21.15.44:0/65905845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0ac102070 0x7fe0ac19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 >> 172.21.15.44:0/65905845 conn(0x7fe0ac0fd930 msgr2=0x7fe0ac0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 shutdown_connections 2024-04-20T11:29:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.224+0000 7fe0b2000640 1 -- 172.21.15.44:0/65905845 wait complete. 2024-04-20T11:29:32.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:29:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4070887131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c010e770 msgr2=0x7fd5c0110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4070887131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c010e770 0x7fd5c0110b60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009be0 tx=0x7fd5b802bef0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4070887131 shutdown_connections 2024-04-20T11:29:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4070887131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c010e770 0x7fd5c0110b60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4070887131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4070887131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c0100bc0 0x7fd5c0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4070887131 >> 172.21.15.44:0/4070887131 conn(0x7fd5c00fc7e0 msgr2=0x7fd5c00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4070887131 shutdown_connections 2024-04-20T11:29:32.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.450+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4070887131 wait complete. 2024-04-20T11:29:32.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.451+0000 7fd5bfe00640 1 Processor -- start 2024-04-20T11:29:32.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 -- start start 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0100bc0 0x7fd5c01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c010e770 0x7fd5c019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c0114530 con 0x7fd5c0101570 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01143b0 con 0x7fd5c0100bc0 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5bfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01146b0 con 0x7fd5c010e770 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c01a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51456/0 (socket says 172.21.15.44:51456) 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.452+0000 7fd5be000640 1 -- 172.21.15.44:0/4090504532 learned_addr learned my addr 172.21.15.44:0/4090504532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5bea00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0100bc0 0x7fd5c01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5bf400640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c010e770 0x7fd5c019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5be000640 1 -- 172.21.15.44:0/4090504532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c010e770 msgr2=0x7fd5c019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5be000640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c010e770 0x7fd5c019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5be000640 1 -- 172.21.15.44:0/4090504532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0100bc0 msgr2=0x7fd5c01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5be000640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0100bc0 0x7fd5c01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5be000640 1 -- 172.21.15.44:0/4090504532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c019b000 con 0x7fd5c0101570 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5bea00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0100bc0 0x7fd5c01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5be000640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c01a0bb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00e9c0 tx=0x7fd5ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.453+0000 7fd5b7400640 1 -- 172.21.15.44:0/4090504532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00ccc0 con 0x7fd5c0101570 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.454+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c019b2f0 con 0x7fd5c0101570 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.454+0000 7fd5b7400640 1 -- 172.21.15.44:0/4090504532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5ac005520 con 0x7fd5c0101570 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.454+0000 7fd5b7400640 1 -- 172.21.15.44:0/4090504532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac018630 con 0x7fd5c0101570 2024-04-20T11:29:32.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.454+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c01a74c0 con 0x7fd5c0101570 2024-04-20T11:29:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.457+0000 7fd5c85a2740 1 Processor -- start 2024-04-20T11:29:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.457+0000 7fd5c85a2740 1 -- start start 2024-04-20T11:29:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad5a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558dbacbf050 con 0x558dbad5aa90 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dbac65490 con 0x558dbad5a170 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558dbacbb010 con 0x558dbad63670 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5b7400640 1 -- 172.21.15.44:0/4090504532 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd5ac0046c0 con 0x7fd5c0101570 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad63130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51460/0 (socket says 172.21.15.44:51460) 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5bea00640 1 -- 172.21.15.44:0/3613981225 learned_addr learned my addr 172.21.15.44:0/3613981225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.458+0000 7fd5bf400640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad5a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bea00640 1 -- 172.21.15.44:0/3613981225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 msgr2=0x558dbad65a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5be000640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bea00640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad65a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bea00640 1 -- 172.21.15.44:0/3613981225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 msgr2=0x558dbad5a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bea00640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad5a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bea00640 1 -- 172.21.15.44:0/3613981225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dbac6d070 con 0x558dbad5aa90 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bf400640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad5a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5be000640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad65a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5bea00640 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad63130 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00115c0 tx=0x7fd5b0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff67b683e663616 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5abe00640 1 -- 172.21.15.44:0/3613981225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000f640 con 0x558dbad5aa90 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5abe00640 1 -- 172.21.15.44:0/3613981225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5b000fce0 con 0x558dbad5aa90 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.459+0000 7fd5abe00640 1 -- 172.21.15.44:0/3613981225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00183b0 con 0x558dbad5aa90 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5b7400640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5b7400640 1 -- 172.21.15.44:0/4090504532 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fd5ac0d0b80 con 0x7fd5c0101570 2024-04-20T11:29:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5bf400640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3613981225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 msgr2=0x558dbad63130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad63130 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00115c0 tx=0x7fd5b0011a80 comp rx=0 tx=0).stop 2024-04-20T11:29:32.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3613981225 shutdown_connections 2024-04-20T11:29:32.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad65a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad63130 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3613981225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad5a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.460+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3613981225 >> 172.21.15.44:0/3613981225 conn(0x558dbad548a0 msgr2=0x558dbad54c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.461+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3613981225 shutdown_connections 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.461+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3613981225 wait complete. 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.461+0000 7fd5bf400640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8033490 tx=0x7fd5b8005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.461+0000 7fd5c85a2740 1 -- start start 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.461+0000 7fd5c85a2740 1 Processor -- start 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558dbad66460 con 0x558dbad5aa90 2024-04-20T11:29:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dbac78860 con 0x558dbad5a170 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558dbad665e0 con 0x558dbad63670 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5c85a2740 10 client.0 _ll_register_callbacks cb 0x558dbac46fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32890/0 (socket says 172.21.15.44:32890) 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5bea00640 1 -- 172.21.15.44:0/3460170246 learned_addr learned my addr 172.21.15.44:0/3460170246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.462+0000 7fd5bf400640 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5bea00640 1 -- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 msgr2=0x558dbad61120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5bea00640 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad61120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5bea00640 1 -- 172.21.15.44:0/3460170246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 msgr2=0x558dbad60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5bea00640 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad60be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5bea00640 1 -- 172.21.15.44:0/3460170246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dbac7c950 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5bea00640 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad606a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5b001b040 tx=0x7fd5b0019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b35c94ec390afef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5a8c00640 1 -- 172.21.15.44:0/3460170246 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0027060 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5a8c00640 1 -- 172.21.15.44:0/3460170246 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fd5b0004250 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.463+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b000fce0 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5a8c00640 1 -- 172.21.15.44:0/3460170246 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd5b000fe90 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 10 client.27335 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558dbac4a820 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5a8c00640 1 -- 172.21.15.44:0/3460170246 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5b000fce0 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5a8c00640 1 -- 172.21.15.44:0/3460170246 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd5b00053d0 con 0x558dbad5a170 2024-04-20T11:29:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 10 client.27335 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 10 client.27335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 20 client.27335 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbad73600 0x558dbad759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 4 client.27335 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.464+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dbaccca70 con 0x558dbad73600 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.465+0000 7fd5be000640 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbad73600 0x558dbad759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.465+0000 7fd5be000640 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbad73600 0x558dbad759f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ead8a890c0e774f server_cookie=a0fe16da8f271cfd in_seq=0 out_seq=0 2024-04-20T11:29:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.465+0000 7fd5a8c00640 10 client.27335 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.466+0000 7fd5a8c00640 1 -- 172.21.15.44:0/3460170246 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dbaccca70 con 0x558dbad73600 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.466+0000 7fd5a8c00640 10 client.27335 handle_command_reply: tid=0 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 client.27335 shutdown 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 msgr2=0x558dbad606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad606a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5b001b040 tx=0x7fd5b0019640 comp rx=0 tx=0).stop 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 shutdown_connections 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbad73600 0x558dbad759f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad63670 0x558dbad61120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5a8c00640 0 client.27335 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:29:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/3460170246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad5a170 0x558dbad606a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 >> 172.21.15.44:0/3460170246 conn(0x558dbad548a0 msgr2=0x558dbad565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 shutdown_connections 2024-04-20T11:29:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 1 -- 172.21.15.44:0/3460170246 wait complete. 2024-04-20T11:29:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.467+0000 7fd5c85a2740 20 client.27335 trim_cache size 0 max 16384 2024-04-20T11:29:32.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.482+0000 7fd5c85a2740 1 Processor -- start 2024-04-20T11:29:32.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.482+0000 7fd5c85a2740 1 -- start start 2024-04-20T11:29:32.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5a170 0x558dbad62dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad6f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad7b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01afda0 con 0x558dbad5aa90 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b18e0 con 0x558dbad63670 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b0470 con 0x558dbad5a170 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad6f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5a170 0x558dbad62dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad6f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51476/0 (socket says 172.21.15.44:51476) 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5a170 0x558dbad62dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59546/0 (socket says 172.21.15.44:59546) 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 -- 172.21.15.44:0/2195682119 learned_addr learned my addr 172.21.15.44:0/2195682119 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5bf400640 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad7b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 -- 172.21.15.44:0/2195682119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5a170 msgr2=0x558dbad62dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5a170 0x558dbad62dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 -- 172.21.15.44:0/2195682119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 msgr2=0x558dbad7b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad7b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.483+0000 7fd5be000640 1 -- 172.21.15.44:0/2195682119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dbad5a8c0 con 0x558dbad5aa90 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.484+0000 7fd5be000640 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad6f350 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0058c0 tx=0x7fd5ac087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfd11f077569590 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.484+0000 7fd5ab400640 1 -- 172.21.15.44:0/2195682119 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0a0eb0 con 0x558dbad5aa90 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.484+0000 7fd5ab400640 1 -- 172.21.15.44:0/2195682119 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5ac0a5770 con 0x558dbad5aa90 2024-04-20T11:29:32.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.484+0000 7fd5ab400640 1 -- 172.21.15.44:0/2195682119 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac09bf50 con 0x558dbad5aa90 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 -- 172.21.15.44:0/2195682119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 msgr2=0x558dbad6f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad6f350 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0058c0 tx=0x7fd5ac087040 comp rx=0 tx=0).stop 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 -- 172.21.15.44:0/2195682119 shutdown_connections 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad7b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5aa90 0x558dbad6f350 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/2195682119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5a170 0x558dbad62dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 -- 172.21.15.44:0/2195682119 >> 172.21.15.44:0/2195682119 conn(0x558dbad548a0 msgr2=0x558dbad5fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 -- 172.21.15.44:0/2195682119 shutdown_connections 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 -- 172.21.15.44:0/2195682119 wait complete. 2024-04-20T11:29:32.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.485+0000 7fd5c85a2740 1 -- start start 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 Processor -- start 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 0x558dbad6b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5aa90 0x558dbad6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad5cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b1210 con 0x558dbad5a170 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b0b40 con 0x558dbad63670 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01add90 con 0x558dbad5aa90 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5c85a2740 10 client.0 _ll_register_callbacks cb 0x7fd5b0019f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 0x558dbad6b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 0x558dbad6b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51480/0 (socket says 172.21.15.44:51480) 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5bea00640 1 -- 172.21.15.44:0/926073022 learned_addr learned my addr 172.21.15.44:0/926073022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.486+0000 7fd5be000640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5aa90 0x558dbad6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bf400640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad5cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bea00640 1 -- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5aa90 msgr2=0x558dbad6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bea00640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5aa90 0x558dbad6bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bea00640 1 -- 172.21.15.44:0/926073022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 msgr2=0x558dbad5cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bea00640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad5cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bea00640 1 -- 172.21.15.44:0/926073022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dba9d13f0 con 0x558dbad5a170 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5bea00640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 0x558dbad6b860 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00115c0 tx=0x7fd5b0009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ba31bfd74a1b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5a8c00640 1 -- 172.21.15.44:0/926073022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b003a060 con 0x558dbad5a170 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5a8c00640 1 -- 172.21.15.44:0/926073022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fd5b0011040 con 0x558dbad5a170 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.487+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac0a5770 con 0x558dbad5a170 2024-04-20T11:29:32.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5a8c00640 1 -- 172.21.15.44:0/926073022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd5b0019a10 con 0x558dbad5a170 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 10 client.17217 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558dbaa9b7d0 con 0x558dbad5a170 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5a8c00640 1 -- 172.21.15.44:0/926073022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5ac0a5770 con 0x558dbad5a170 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5a8c00640 1 -- 172.21.15.44:0/926073022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd5b00035f0 con 0x558dbad5a170 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 10 client.17217 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 10 client.17217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 20 client.17217 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbadd98e0 0x558dbaddbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 4 client.17217 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.488+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dbacc68e0 con 0x558dbadd98e0 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.489+0000 7fd5be000640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbadd98e0 0x558dbaddbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.489+0000 7fd5be000640 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbadd98e0 0x558dbaddbd00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2150f41e8e52d1ca server_cookie=f3bcaeb001f25b06 in_seq=0 out_seq=0 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.489+0000 7fd5a8c00640 10 client.17217 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.489+0000 7fd5a8c00640 1 -- 172.21.15.44:0/926073022 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dbacc68e0 con 0x558dbadd98e0 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.489+0000 7fd5a8c00640 10 client.17217 handle_command_reply: tid=0 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 client.17217 shutdown 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 msgr2=0x558dbad6b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 0x558dbad6b860 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00115c0 tx=0x7fd5b0009e40 comp rx=0 tx=0).stop 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 shutdown_connections 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x558dbadd98e0 0x558dbaddbd00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dbad63670 0x558dbad5cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558dbad5aa90 0x558dbad6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 --2- 172.21.15.44:0/926073022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558dbad5a170 0x558dbad6b860 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 >> 172.21.15.44:0/926073022 conn(0x558dbad548a0 msgr2=0x558dbad53960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5a8c00640 0 client.17217 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:29:32.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 shutdown_connections 2024-04-20T11:29:32.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 1 -- 172.21.15.44:0/926073022 wait complete. 2024-04-20T11:29:32.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.490+0000 7fd5c85a2740 20 client.17217 trim_cache size 0 max 16384 2024-04-20T11:29:32.496 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5a40a3d20 msgr2=0x7fd5a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8033490 tx=0x7fd5b8005fd0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 msgr2=0x7fd5c01a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c01a0bb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00e9c0 tx=0x7fd5ac00ee80 comp rx=0 tx=0).stop 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 shutdown_connections 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c010e770 0x7fd5c019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5a40a3d20 0x7fd5a40a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5c0101570 0x7fd5c01a0bb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 --2- 172.21.15.44:0/4090504532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0100bc0 0x7fd5c01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 >> 172.21.15.44:0/4090504532 conn(0x7fd5c00fc7e0 msgr2=0x7fd5c010fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 shutdown_connections 2024-04-20T11:29:32.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.491+0000 7fd5bfe00640 1 -- 172.21.15.44:0/4090504532 wait complete. 2024-04-20T11:29:32.510 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:29:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 -- 172.21.15.44:0/4125318805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9484104410 msgr2=0x7f94841047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 --2- 172.21.15.44:0/4125318805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9484104410 0x7f94841047f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9474009ca0 tx=0x7f947402c100 comp rx=0 tx=0).stop 2024-04-20T11:29:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 -- 172.21.15.44:0/4125318805 shutdown_connections 2024-04-20T11:29:32.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 --2- 172.21.15.44:0/4125318805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 0x7f94841110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 --2- 172.21.15.44:0/4125318805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104dc0 0x7f9484108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 --2- 172.21.15.44:0/4125318805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9484104410 0x7f94841047f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 -- 172.21.15.44:0/4125318805 >> 172.21.15.44:0/4125318805 conn(0x7f94840ffcf0 msgr2=0x7f9484102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 -- 172.21.15.44:0/4125318805 shutdown_connections 2024-04-20T11:29:32.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.720+0000 7f9489400640 1 -- 172.21.15.44:0/4125318805 wait complete. 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.721+0000 7f9489400640 1 Processor -- start 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 -- start start 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 0x7f94841a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 0x7f94841a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f948419aa10 0x7f948419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94841146b0 con 0x7f9484104410 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9484114530 con 0x7f948419aa10 2024-04-20T11:29:32.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9489400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9484114830 con 0x7f94841094e0 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9482800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 0x7f94841a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9482800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 0x7f94841a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51506/0 (socket says 172.21.15.44:51506) 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9482800640 1 -- 172.21.15.44:0/3216062723 learned_addr learned my addr 172.21.15.44:0/3216062723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9481e00640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 0x7f94841a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.722+0000 7f9483200640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f948419aa10 0x7f948419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9482800640 1 -- 172.21.15.44:0/3216062723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 msgr2=0x7f94841a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9482800640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 0x7f94841a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9482800640 1 -- 172.21.15.44:0/3216062723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f948419aa10 msgr2=0x7f948419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9482800640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f948419aa10 0x7f948419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9482800640 1 -- 172.21.15.44:0/3216062723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948419b530 con 0x7f9484104410 2024-04-20T11:29:32.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9481e00640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 0x7f94841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f9482800640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 0x7f94841a0860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9474005e30 tx=0x7f94740093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.723+0000 7f947b400640 1 -- 172.21.15.44:0/3216062723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474039040 con 0x7f9484104410 2024-04-20T11:29:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.724+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948419b7c0 con 0x7f9484104410 2024-04-20T11:29:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.724+0000 7f947b400640 1 -- 172.21.15.44:0/3216062723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9474044070 con 0x7f9484104410 2024-04-20T11:29:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.724+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94841a79c0 con 0x7f9484104410 2024-04-20T11:29:32.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.724+0000 7f947b400640 1 -- 172.21.15.44:0/3216062723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474004340 con 0x7f9484104410 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.727+0000 7f948c015740 1 Processor -- start 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.727+0000 7f947b400640 1 -- 172.21.15.44:0/3216062723 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9474003a90 con 0x7f9484104410 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.727+0000 7f948c015740 1 -- start start 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f948c015740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9efb0 0x560ed9e9f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f948c015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f948c015740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eaa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f948c015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560ed9e04050 con 0x560ed9e9f8d0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f948c015740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560ed9daa490 con 0x560ed9ea84b0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f948c015740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560ed9e00010 con 0x560ed9e9efb0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9482800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eaa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9483200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9efb0 0x560ed9e9f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eaa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:32944/0 (socket says 172.21.15.44:32944) 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9482800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51518/0 (socket says 172.21.15.44:51518) 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 -- 172.21.15.44:0/1800665882 learned_addr learned my addr 172.21.15.44:0/1800665882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 -- 172.21.15.44:0/1800665882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9efb0 msgr2=0x560ed9e9f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9efb0 0x560ed9e9f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 -- 172.21.15.44:0/1800665882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 msgr2=0x560ed9ea7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.728+0000 7f9481e00640 1 -- 172.21.15.44:0/1800665882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed9db2070 con 0x560ed9ea84b0 2024-04-20T11:29:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f947b400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f94680a3d20 0x7f94680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f9481e00640 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eaa8a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f946c011470 tx=0x7f946c011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3fda0c45097cc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f947b400640 1 -- 172.21.15.44:0/3216062723 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f94740f2a30 con 0x7f9484104410 2024-04-20T11:29:32.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f9473400640 1 -- 172.21.15.44:0/1800665882 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c01e070 con 0x560ed9ea84b0 2024-04-20T11:29:32.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f9473400640 1 -- 172.21.15.44:0/1800665882 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f946c0035f0 con 0x560ed9ea84b0 2024-04-20T11:29:32.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f9473400640 1 -- 172.21.15.44:0/1800665882 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00f8f0 con 0x560ed9ea84b0 2024-04-20T11:29:32.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.729+0000 7f9483200640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f94680a3d20 0x7f94680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 -- 172.21.15.44:0/1800665882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 msgr2=0x560ed9eaa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eaa8a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f946c011470 tx=0x7f946c011930 comp rx=0 tx=0).stop 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f9483200640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f94680a3d20 0x7f94680a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f947c009d20 tx=0x7f947c005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 -- 172.21.15.44:0/1800665882 shutdown_connections 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eaa8a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 --2- 172.21.15.44:0/1800665882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9efb0 0x560ed9e9f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 -- 172.21.15.44:0/1800665882 >> 172.21.15.44:0/1800665882 conn(0x560ed9e996e0 msgr2=0x560ed9e99ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 -- 172.21.15.44:0/1800665882 shutdown_connections 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.730+0000 7f948c015740 1 -- 172.21.15.44:0/1800665882 wait complete. 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 -- start start 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 Processor -- start 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 0x560ed9ea5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560ed9eab2a0 con 0x560ed9ea84b0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560ed9dbd860 con 0x560ed9e9efb0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560ed9eab420 con 0x560ed9e9f8d0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.731+0000 7f948c015740 10 client.0 _ll_register_callbacks cb 0x560ed9d8bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9483200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 0x560ed9ea5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9483200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 0x560ed9ea5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51528/0 (socket says 172.21.15.44:51528) 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9483200640 1 -- 172.21.15.44:0/3055875767 learned_addr learned my addr 172.21.15.44:0/3055875767 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9481e00640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 -- 172.21.15.44:0/3055875767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 msgr2=0x560ed9ea59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 -- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 msgr2=0x560ed9ea5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 0x560ed9ea5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 -- 172.21.15.44:0/3055875767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed9dc1950 con 0x560ed9e9efb0 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9483200640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 0x560ed9ea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9482800640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea54b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f94740f0270 tx=0x7f9474009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff72253ec011d95f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.732+0000 7f9467e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f9467e00640 1 -- 172.21.15.44:0/3055875767 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740be600 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f9467e00640 1 -- 172.21.15.44:0/3055875767 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f94740bed20 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946c0035f0 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f9467e00640 1 -- 172.21.15.44:0/3055875767 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f94740be420 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f948c015740 10 client.27353 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560ed9d8f820 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f9467e00640 1 -- 172.21.15.44:0/3055875767 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f946c0035f0 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.733+0000 7f9467e00640 1 -- 172.21.15.44:0/3055875767 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f947403c030 con 0x560ed9e9efb0 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f948c015740 10 client.27353 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f948c015740 10 client.27353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:29:32.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f948c015740 20 client.27353 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f948c015740 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9ec0c40 0x560ed9ec3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f948c015740 4 client.27353 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ed9e11a70 con 0x560ed9ec0c40 2024-04-20T11:29:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.734+0000 7f9481e00640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9ec0c40 0x560ed9ec3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.735+0000 7f9481e00640 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9ec0c40 0x560ed9ec3050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b32c78538c70479d server_cookie=b98f830a138dd11 in_seq=0 out_seq=0 2024-04-20T11:29:32.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.735+0000 7f9467e00640 10 client.27353 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:29:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.735+0000 7f9467e00640 1 -- 172.21.15.44:0/3055875767 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ed9e11a70 con 0x560ed9ec0c40 2024-04-20T11:29:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.735+0000 7f9467e00640 10 client.27353 handle_command_reply: tid=0 2024-04-20T11:29:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.735+0000 7f948c015740 1 client.27353 shutdown 2024-04-20T11:29:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 msgr2=0x560ed9ea54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea54b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f94740f0270 tx=0x7f9474009800 comp rx=0 tx=0).stop 2024-04-20T11:29:32.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 shutdown_connections 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9ec0c40 0x560ed9ec3050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9ea84b0 0x560ed9ea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9ea59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 --2- 172.21.15.44:0/3055875767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea54b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 >> 172.21.15.44:0/3055875767 conn(0x560ed9e996e0 msgr2=0x560ed9e9b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f9467e00640 0 client.27353 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 shutdown_connections 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 1 -- 172.21.15.44:0/3055875767 wait complete. 2024-04-20T11:29:32.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.736+0000 7f948c015740 20 client.27353 trim_cache size 0 max 16384 2024-04-20T11:29:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 Processor -- start 2024-04-20T11:29:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 -- start start 2024-04-20T11:29:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ec8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9ea84b0 0x560ed9ec8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94841aba80 con 0x560ed9e9f8d0 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94841ad5c0 con 0x560ed9e9efb0 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.751+0000 7f948c015740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94841ac150 con 0x560ed9ea84b0 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ec8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ec8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51530/0 (socket says 172.21.15.44:51530) 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 -- 172.21.15.44:0/4152277269 learned_addr learned my addr 172.21.15.44:0/4152277269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9482800640 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9483200640 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9ea84b0 0x560ed9ec8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 -- 172.21.15.44:0/4152277269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9ea84b0 msgr2=0x560ed9ec8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9ea84b0 0x560ed9ec8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 -- 172.21.15.44:0/4152277269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 msgr2=0x560ed9ea7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9481e00640 1 -- 172.21.15.44:0/4152277269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed9b163f0 con 0x560ed9e9f8d0 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.752+0000 7f9482800640 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.753+0000 7f9481e00640 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ec8960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f946c01c040 tx=0x7f946c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e480d9fd7ccf784d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.753+0000 7f9472a00640 1 -- 172.21.15.44:0/4152277269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c02e070 con 0x560ed9e9f8d0 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.753+0000 7f9472a00640 1 -- 172.21.15.44:0/4152277269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f946c01f040 con 0x560ed9e9f8d0 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.753+0000 7f9472a00640 1 -- 172.21.15.44:0/4152277269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c029e60 con 0x560ed9e9f8d0 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.753+0000 7f948c015740 1 -- 172.21.15.44:0/4152277269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 msgr2=0x560ed9ec8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.753+0000 7f948c015740 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ec8960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f946c01c040 tx=0x7f946c0096c0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 -- 172.21.15.44:0/4152277269 shutdown_connections 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9ea84b0 0x560ed9ec8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9f8d0 0x560ed9ec8960 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 --2- 172.21.15.44:0/4152277269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9e9efb0 0x560ed9ea7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 -- 172.21.15.44:0/4152277269 >> 172.21.15.44:0/4152277269 conn(0x560ed9e996e0 msgr2=0x560ed9ea0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 -- 172.21.15.44:0/4152277269 shutdown_connections 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 -- 172.21.15.44:0/4152277269 wait complete. 2024-04-20T11:29:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 -- start start 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.754+0000 7f948c015740 1 Processor -- start 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 0x560ed9eb4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9eb4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eb53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94841acef0 con 0x560ed9e9efb0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94841ac820 con 0x560ed9ea84b0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94841a9a70 con 0x560ed9e9f8d0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f948c015740 10 client.0 _ll_register_callbacks cb 0x7f94740f61d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f9482800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 0x560ed9eb4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f9482800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 0x560ed9eb4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51542/0 (socket says 172.21.15.44:51542) 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f9482800640 1 -- 172.21.15.44:0/2999219474 learned_addr learned my addr 172.21.15.44:0/2999219474 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f9481e00640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9eb4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.755+0000 7f9483200640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eb53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9482800640 1 -- 172.21.15.44:0/2999219474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 msgr2=0x560ed9eb4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9482800640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9eb4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9482800640 1 -- 172.21.15.44:0/2999219474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 msgr2=0x560ed9eb53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9482800640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eb53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9482800640 1 -- 172.21.15.44:0/2999219474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed9be07d0 con 0x560ed9e9efb0 2024-04-20T11:29:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9481e00640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9eb4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9482800640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 0x560ed9eb4920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f94740f6280 tx=0x7f94740099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2440971507fc138e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9467e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9467e00640 1 -- 172.21.15.44:0/2999219474 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740be600 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f9467e00640 1 -- 172.21.15.44:0/2999219474 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f94740f0550 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.756+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946c01f040 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f9467e00640 1 -- 172.21.15.44:0/2999219474 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f94740bf1c0 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 10 client.17235 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560ed9f272e0 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f9467e00640 1 -- 172.21.15.44:0/2999219474 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f946c01f040 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f9467e00640 1 -- 172.21.15.44:0/2999219474 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f9474004b40 con 0x560ed9e9efb0 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 10 client.17235 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 10 client.17235 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 20 client.17235 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9f27810 0x560ed9f29c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 4 client.17235 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ed9e0b8e0 con 0x560ed9f27810 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.757+0000 7f9481e00640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9f27810 0x560ed9f29c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.758+0000 7f9481e00640 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9f27810 0x560ed9f29c00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a06b4c9ec73bf30e server_cookie=af35ac049c8cc7b6 in_seq=0 out_seq=0 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.758+0000 7f9467e00640 10 client.17235 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.758+0000 7f9467e00640 1 -- 172.21.15.44:0/2999219474 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ed9e0b8e0 con 0x560ed9f27810 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.758+0000 7f9467e00640 10 client.17235 handle_command_reply: tid=0 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.758+0000 7f948c015740 1 client.17235 shutdown 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 msgr2=0x560ed9eb4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 0x560ed9eb4920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f94740f6280 tx=0x7f94740099c0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 shutdown_connections 2024-04-20T11:29:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x560ed9f27810 0x560ed9f29c00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560ed9ea84b0 0x560ed9eb53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560ed9e9f8d0 0x560ed9eb4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 --2- 172.21.15.44:0/2999219474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560ed9e9efb0 0x560ed9eb4920 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f9467e00640 0 client.17235 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 >> 172.21.15.44:0/2999219474 conn(0x560ed9e996e0 msgr2=0x560ed9e97eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 shutdown_connections 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 1 -- 172.21.15.44:0/2999219474 wait complete. 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.759+0000 7f948c015740 20 client.17235 trim_cache size 0 max 16384 2024-04-20T11:29:32.763 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f94680a3d20 msgr2=0x7f94680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f94680a3d20 0x7f94680a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f947c009d20 tx=0x7f947c005f90 comp rx=0 tx=0).stop 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 msgr2=0x7f94841a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 0x7f94841a0860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9474005e30 tx=0x7f94740093d0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 shutdown_connections 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f948419aa10 0x7f948419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94841094e0 0x7f94841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f94680a3d20 0x7f94680a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 --2- 172.21.15.44:0/3216062723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9484104410 0x7f94841a0860 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 >> 172.21.15.44:0/3216062723 conn(0x7f94840ffcf0 msgr2=0x7f9484101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 shutdown_connections 2024-04-20T11:29:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.760+0000 7f9489400640 1 -- 172.21.15.44:0/3216062723 wait complete. 2024-04-20T11:29:32.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:32 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2499165638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:29:32.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:29:32.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:32 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2499165638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:29:32.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:32 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2499165638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:29:32.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3075543603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 msgr2=0x7fd99c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3075543603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c10dc00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd994009900 tx=0x7fd99402bc30 comp rx=0 tx=0).stop 2024-04-20T11:29:32.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3075543603 shutdown_connections 2024-04-20T11:29:32.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3075543603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c10dc00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3075543603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c101910 0x7fd99c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3075543603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd99c100f60 0x7fd99c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.987+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3075543603 >> 172.21.15.44:0/3075543603 conn(0x7fd99c0fc7e0 msgr2=0x7fd99c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.988+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3075543603 shutdown_connections 2024-04-20T11:29:32.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.988+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3075543603 wait complete. 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 Processor -- start 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 -- start start 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 0x7fd99c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd99c19d020 0x7fd99c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1122e0 con 0x7fd99c100f60 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c112160 con 0x7fd99c106030 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd9a1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c112460 con 0x7fd99c19d020 2024-04-20T11:29:32.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd99be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 0x7fd99c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd99be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 0x7fd99c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51562/0 (socket says 172.21.15.44:51562) 2024-04-20T11:29:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.989+0000 7fd99be00640 1 -- 172.21.15.44:0/3202357358 learned_addr learned my addr 172.21.15.44:0/3202357358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99b400640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd9a1000640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd99c19d020 0x7fd99c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99be00640 1 -- 172.21.15.44:0/3202357358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd99c19d020 msgr2=0x7fd99c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99be00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd99c19d020 0x7fd99c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99be00640 1 -- 172.21.15.44:0/3202357358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 msgr2=0x7fd99c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99be00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99be00640 1 -- 172.21.15.44:0/3202357358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c19db10 con 0x7fd99c100f60 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99b400640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.990+0000 7fd99be00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 0x7fd99c198760 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00eb90 tx=0x7fd98c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.991+0000 7fd998c00640 1 -- 172.21.15.44:0/3202357358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c004d20 con 0x7fd99c100f60 2024-04-20T11:29:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.991+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c19de00 con 0x7fd99c100f60 2024-04-20T11:29:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.991+0000 7fd998c00640 1 -- 172.21.15.44:0/3202357358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd98c00ee90 con 0x7fd99c100f60 2024-04-20T11:29:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.991+0000 7fd998c00640 1 -- 172.21.15.44:0/3202357358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0108b0 con 0x7fd99c100f60 2024-04-20T11:29:32.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.991+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a55a0 con 0x7fd99c100f60 2024-04-20T11:29:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.994+0000 7fd998c00640 1 -- 172.21.15.44:0/3202357358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd98c005020 con 0x7fd99c100f60 2024-04-20T11:29:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.994+0000 7fd9a5bf6740 1 Processor -- start 2024-04-20T11:29:32.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 -- start start 2024-04-20T11:29:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a2254a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22553540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a22555e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564a224ab050 con 0x564a22553a80 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a22451490 con 0x564a2254aea0 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a224a7010 con 0x564a2254a580 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd99b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a22555e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd99b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a22555e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51574/0 (socket says 172.21.15.44:51574) 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd99b400640 1 -- 172.21.15.44:0/2383681096 learned_addr learned my addr 172.21.15.44:0/2383681096 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.995+0000 7fd9a1000640 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a2254a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99be00640 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22553540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99b400640 1 -- 172.21.15.44:0/2383681096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 msgr2=0x564a2254a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99b400640 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a2254a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99b400640 1 -- 172.21.15.44:0/2383681096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 msgr2=0x564a22553540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99b400640 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22553540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99b400640 1 -- 172.21.15.44:0/2383681096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a22459070 con 0x564a22553a80 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd998c00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd9800a3d20 0x7fd9800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd99b400640 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a22555e70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd988009fd0 tx=0x7fd9880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2576f74eb4f151eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.996+0000 7fd998c00640 1 -- 172.21.15.44:0/3202357358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fd98c0d0e50 con 0x7fd99c100f60 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd9a1000640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd9800a3d20 0x7fd9800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd991600640 1 -- 172.21.15.44:0/2383681096 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988005020 con 0x564a22553a80 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd991600640 1 -- 172.21.15.44:0/2383681096 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9880047a0 con 0x564a22553a80 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd991600640 1 -- 172.21.15.44:0/2383681096 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98800fa40 con 0x564a22553a80 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/2383681096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 msgr2=0x564a22555e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:32.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd9a1000640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd9800a3d20 0x7fd9800a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd994005ab0 tx=0x7fd9940058d0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.997+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a22555e70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd988009fd0 tx=0x7fd9880118d0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/2383681096 shutdown_connections 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a22555e70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22553540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/2383681096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a2254a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/2383681096 >> 172.21.15.44:0/2383681096 conn(0x564a22544cb0 msgr2=0x564a22545090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/2383681096 shutdown_connections 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/2383681096 wait complete. 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 -- start start 2024-04-20T11:29:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.998+0000 7fd9a5bf6740 1 Processor -- start 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a22550b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22551060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a225515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564a22556870 con 0x564a22553a80 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a22464860 con 0x564a2254aea0 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a225569f0 con 0x564a2254a580 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a5bf6740 10 client.0 _ll_register_callbacks cb 0x564a22432fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a225515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd99b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22551060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd99b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22551060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:33020/0 (socket says 172.21.15.44:33020) 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd99b400640 1 -- 172.21.15.44:0/90364580 learned_addr learned my addr 172.21.15.44:0/90364580 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd9a1000640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a225515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51578/0 (socket says 172.21.15.44:51578) 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:32.999+0000 7fd99be00640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a22550b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd99b400640 1 -- 172.21.15.44:0/90364580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 msgr2=0x564a22550b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd99b400640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a22550b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd99b400640 1 -- 172.21.15.44:0/90364580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 msgr2=0x564a225515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd99b400640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a225515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd99b400640 1 -- 172.21.15.44:0/90364580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a22468950 con 0x564a2254aea0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd99b400640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22551060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd98801b040 tx=0x7fd9880096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599bf0cf334f2cff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd986000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd986000640 1 -- 172.21.15.44:0/90364580 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988028070 con 0x564a2254aea0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9880047a0 con 0x564a2254aea0 2024-04-20T11:29:33.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.000+0000 7fd986000640 1 -- 172.21.15.44:0/90364580 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fd9880055a0 con 0x564a2254aea0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd986000640 1 -- 172.21.15.44:0/90364580 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd988034050 con 0x564a2254aea0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 10 client.27371 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564a22436820 con 0x564a2254aea0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd986000640 1 -- 172.21.15.44:0/90364580 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9880047a0 con 0x564a2254aea0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd986000640 1 -- 172.21.15.44:0/90364580 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd9880245d0 con 0x564a2254aea0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 10 client.27371 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 10 client.27371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 20 client.27371 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a22563a70 0x564a22565e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 4 client.27371 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.001+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a224b8a70 con 0x564a22563a70 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.002+0000 7fd99be00640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a22563a70 0x564a22565e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.002+0000 7fd99be00640 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a22563a70 0x564a22565e60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90de21308b33c9a9 server_cookie=a705dcb9c7a772a2 in_seq=0 out_seq=0 2024-04-20T11:29:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.002+0000 7fd986000640 10 client.27371 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd986000640 1 -- 172.21.15.44:0/90364580 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a224b8a70 con 0x564a22563a70 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd986000640 10 client.27371 handle_command_reply: tid=0 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 client.27371 shutdown 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 msgr2=0x564a22551060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22551060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd98801b040 tx=0x7fd9880096c0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 shutdown_connections 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a22563a70 0x564a22565e60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a22553a80 0x564a225515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254aea0 0x564a22551060 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/90364580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254a580 0x564a22550b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd986000640 0 client.27371 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.003+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 >> 172.21.15.44:0/90364580 conn(0x564a22544cb0 msgr2=0x564a22546a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.004+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 shutdown_connections 2024-04-20T11:29:33.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.004+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/90364580 wait complete. 2024-04-20T11:29:33.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.004+0000 7fd9a5bf6740 20 client.27371 trim_cache size 0 max 16384 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 Processor -- start 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 -- start start 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 0x564a22553250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254aea0 0x564a2255f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a22553a80 0x564a2256b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1ab880 con 0x564a2254a580 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1ad3c0 con 0x564a22553a80 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1abf50 con 0x564a2254aea0 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd99be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 0x564a22553250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd99be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 0x564a22553250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51586/0 (socket says 172.21.15.44:51586) 2024-04-20T11:29:33.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd99be00640 1 -- 172.21.15.44:0/3882419047 learned_addr learned my addr 172.21.15.44:0/3882419047 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:33.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd9a1000640 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a22553a80 0x564a2256b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.019+0000 7fd99b400640 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254aea0 0x564a2255f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd99be00640 1 -- 172.21.15.44:0/3882419047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254aea0 msgr2=0x564a2255f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd99be00640 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254aea0 0x564a2255f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd99be00640 1 -- 172.21.15.44:0/3882419047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a22553a80 msgr2=0x564a2256b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd99be00640 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a22553a80 0x564a2256b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd99be00640 1 -- 172.21.15.44:0/3882419047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2254acd0 con 0x564a2254a580 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd99be00640 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 0x564a22553250 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00cba0 tx=0x7fd98c0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc091e67a54d7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd990c00640 1 -- 172.21.15.44:0/3882419047 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c09c730 con 0x564a2254a580 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd990c00640 1 -- 172.21.15.44:0/3882419047 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd98c09d4d0 con 0x564a2254a580 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.020+0000 7fd990c00640 1 -- 172.21.15.44:0/3882419047 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c099440 con 0x564a2254a580 2024-04-20T11:29:33.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/3882419047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 msgr2=0x564a22553250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 0x564a22553250 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00cba0 tx=0x7fd98c0bd040 comp rx=0 tx=0).stop 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/3882419047 shutdown_connections 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a22553a80 0x564a2256b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a2254aea0 0x564a2255f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/3882419047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254a580 0x564a22553250 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/3882419047 >> 172.21.15.44:0/3882419047 conn(0x564a22544cb0 msgr2=0x564a2254b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/3882419047 shutdown_connections 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.021+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/3882419047 wait complete. 2024-04-20T11:29:33.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 -- start start 2024-04-20T11:29:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 Processor -- start 2024-04-20T11:29:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 0x564a2255beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254aea0 0x564a22557200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a22553a80 0x564a22557740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1accf0 con 0x564a2254aea0 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1ac620 con 0x564a2254a580 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1a9870 con 0x564a22553a80 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.022+0000 7fd9a5bf6740 10 client.0 _ll_register_callbacks cb 0x7fd988004ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254aea0 0x564a22557200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 0x564a2255beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 0x564a2255beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:33038/0 (socket says 172.21.15.44:33038) 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 -- 172.21.15.44:0/1749949423 learned_addr learned my addr 172.21.15.44:0/1749949423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd9a1000640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a22553a80 0x564a22557740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 -- 172.21.15.44:0/1749949423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a22553a80 msgr2=0x564a22557740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a22553a80 0x564a22557740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 -- 172.21.15.44:0/1749949423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254aea0 msgr2=0x564a22557200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254aea0 0x564a22557200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 -- 172.21.15.44:0/1749949423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a221bd3f0 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99b400640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254aea0 0x564a22557200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd99be00640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 0x564a2255beb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd98c099230 tx=0x7fd98c005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f85ad48848c943 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.023+0000 7fd986000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd986000640 1 -- 172.21.15.44:0/1749949423 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0bb060 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd986000640 1 -- 172.21.15.44:0/1749949423 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fd98c005310 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c09d4d0 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd986000640 1 -- 172.21.15.44:0/1749949423 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd98c0b5070 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 10 client.27377 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564a222877d0 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd986000640 1 -- 172.21.15.44:0/1749949423 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd98c09d4d0 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd986000640 1 -- 172.21.15.44:0/1749949423 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd98c083020 con 0x564a2254a580 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 10 client.27377 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 10 client.27377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:33.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 20 client.27377 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a225c9fc0 0x564a225cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 4 client.27377 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.024+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a224b28e0 con 0x564a225c9fc0 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.025+0000 7fd99b400640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a225c9fc0 0x564a225cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.025+0000 7fd99b400640 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a225c9fc0 0x564a225cc3e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c2023c993cfa6e server_cookie=1df2001c7e02fe1b in_seq=0 out_seq=0 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.025+0000 7fd986000640 10 client.27377 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.025+0000 7fd986000640 1 -- 172.21.15.44:0/1749949423 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a224b28e0 con 0x564a225c9fc0 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.025+0000 7fd986000640 10 client.27377 handle_command_reply: tid=0 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.025+0000 7fd9a5bf6740 1 client.27377 shutdown 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 msgr2=0x564a2255beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 0x564a2255beb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd98c099230 tx=0x7fd98c005640 comp rx=0 tx=0).stop 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 shutdown_connections 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564a225c9fc0 0x564a225cc3e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a22553a80 0x564a22557740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd986000640 0 client.27377 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a2254aea0 0x564a22557200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 --2- 172.21.15.44:0/1749949423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2254a580 0x564a2255beb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 >> 172.21.15.44:0/1749949423 conn(0x564a22544cb0 msgr2=0x564a2253f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 shutdown_connections 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 1 -- 172.21.15.44:0/1749949423 wait complete. 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.026+0000 7fd9a5bf6740 20 client.27377 trim_cache size 0 max 16384 2024-04-20T11:29:33.030 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd9800a3d20 msgr2=0x7fd9800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd9800a3d20 0x7fd9800a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd994005ab0 tx=0x7fd9940058d0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 msgr2=0x7fd99c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 0x7fd99c198760 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00eb90 tx=0x7fd98c00c6a0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 shutdown_connections 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd99c19d020 0x7fd99c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd99c106030 0x7fd99c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd9800a3d20 0x7fd9800a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 --2- 172.21.15.44:0/3202357358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c100f60 0x7fd99c198760 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:33.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 >> 172.21.15.44:0/3202357358 conn(0x7fd99c0fc7e0 msgr2=0x7fd99c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:33.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 shutdown_connections 2024-04-20T11:29:33.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:33.027+0000 7fd9a1a00640 1 -- 172.21.15.44:0/3202357358 wait complete. 2024-04-20T11:29:33.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:33 smithi078 ceph-mon[21464]: pgmap v213: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:33.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:33 smithi044 ceph-mon[16855]: pgmap v213: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:33 smithi165 ceph-mon[21975]: pgmap v213: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:34.044 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:29:34.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:29:34.045 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:29:34.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.280+0000 7f9a1be00640 1 -- 172.21.15.44:0/1282261773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c1038e0 msgr2=0x7f9a1c10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.280+0000 7f9a1be00640 1 --2- 172.21.15.44:0/1282261773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c10bdc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009860 tx=0x7f9a0402bc10 comp rx=0 tx=0).stop 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 -- 172.21.15.44:0/1282261773 shutdown_connections 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 --2- 172.21.15.44:0/1282261773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c10c3e0 0x7f9a1c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 --2- 172.21.15.44:0/1282261773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c10bdc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 --2- 172.21.15.44:0/1282261773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c102fc0 0x7f9a1c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 -- 172.21.15.44:0/1282261773 >> 172.21.15.44:0/1282261773 conn(0x7f9a1c0fc7e0 msgr2=0x7f9a1c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 -- 172.21.15.44:0/1282261773 shutdown_connections 2024-04-20T11:29:34.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.281+0000 7f9a1be00640 1 -- 172.21.15.44:0/1282261773 wait complete. 2024-04-20T11:29:34.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.282+0000 7f9a1be00640 1 Processor -- start 2024-04-20T11:29:34.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.282+0000 7f9a1be00640 1 -- start start 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c102fc0 0x7f9a1c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c10c3e0 0x7f9a1c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c112300 con 0x7f9a1c10c3e0 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c112180 con 0x7f9a1c102fc0 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c112480 con 0x7f9a1c1038e0 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c10c3e0 0x7f9a1c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c10c3e0 0x7f9a1c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51600/0 (socket says 172.21.15.44:51600) 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1b400640 1 -- 172.21.15.44:0/3286333136 learned_addr learned my addr 172.21.15.44:0/3286333136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a12000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.283+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c102fc0 0x7f9a1c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a12000640 1 -- 172.21.15.44:0/3286333136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c102fc0 msgr2=0x7f9a1c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a12000640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c102fc0 0x7f9a1c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a12000640 1 -- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c10c3e0 msgr2=0x7f9a1c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a12000640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c10c3e0 0x7f9a1c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a12000640 1 -- 172.21.15.44:0/3286333136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c19afe0 con 0x7f9a1c1038e0 2024-04-20T11:29:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a12000640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c1a0be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a0402c150 tx=0x7f9a0402c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.284+0000 7f9a20a00640 1 -- 172.21.15.44:0/3286333136 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04041020 con 0x7f9a1c1038e0 2024-04-20T11:29:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.285+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c19b270 con 0x7f9a1c1038e0 2024-04-20T11:29:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.285+0000 7f9a20a00640 1 -- 172.21.15.44:0/3286333136 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a04034940 con 0x7f9a1c1038e0 2024-04-20T11:29:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.285+0000 7f9a20a00640 1 -- 172.21.15.44:0/3286333136 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04039040 con 0x7f9a1c1038e0 2024-04-20T11:29:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.285+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1a74f0 con 0x7f9a1c1038e0 2024-04-20T11:29:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.288+0000 7f9a24be6740 1 Processor -- start 2024-04-20T11:29:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.288+0000 7f9a24be6740 1 -- start start 2024-04-20T11:29:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.288+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5b0f0 0x55a93ea5b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.288+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5ba10 0x55a93ea640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea645f0 0x55a93ea669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a24be6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a93e9bfd20 con 0x55a93ea5ba10 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a24be6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a93e9663e0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a24be6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a93e9bbce0 con 0x55a93ea645f0 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5ba10 0x55a93ea640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5ba10 0x55a93ea640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51608/0 (socket says 172.21.15.44:51608) 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1aa00640 1 -- 172.21.15.44:0/519100724 learned_addr learned my addr 172.21.15.44:0/519100724 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1b400640 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5b0f0 0x55a93ea5b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1b400640 1 -- 172.21.15.44:0/519100724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea645f0 msgr2=0x55a93ea669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1b400640 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea645f0 0x55a93ea669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1b400640 1 -- 172.21.15.44:0/519100724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5ba10 msgr2=0x55a93ea640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1b400640 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5ba10 0x55a93ea640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.289+0000 7f9a1b400640 1 -- 172.21.15.44:0/519100724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93e985980 con 0x55a93ea5b0f0 2024-04-20T11:29:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a1b400640 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5b0f0 0x55a93ea5b4d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009f30 tx=0x7f9a140118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c3adf5c4978c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a20a00640 1 -- 172.21.15.44:0/3286333136 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9a04044020 con 0x7f9a1c1038e0 2024-04-20T11:29:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a0be00640 1 -- 172.21.15.44:0/519100724 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14004b20 con 0x55a93ea5b0f0 2024-04-20T11:29:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a0be00640 1 -- 172.21.15.44:0/519100724 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a14004cc0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a0be00640 1 -- 172.21.15.44:0/519100724 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14005410 con 0x55a93ea5b0f0 2024-04-20T11:29:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a24be6740 1 -- 172.21.15.44:0/519100724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5b0f0 msgr2=0x55a93ea5b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.290+0000 7f9a24be6740 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5b0f0 0x55a93ea5b4d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009f30 tx=0x7f9a140118d0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 -- 172.21.15.44:0/519100724 shutdown_connections 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea645f0 0x55a93ea669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5ba10 0x55a93ea640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 --2- 172.21.15.44:0/519100724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5b0f0 0x55a93ea5b4d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 -- 172.21.15.44:0/519100724 >> 172.21.15.44:0/519100724 conn(0x55a93ea55820 msgr2=0x55a93ea55c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 -- 172.21.15.44:0/519100724 shutdown_connections 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 -- 172.21.15.44:0/519100724 wait complete. 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a20a00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9a000a3cb0 0x7f9a000a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a24be6740 1 -- start start 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.291+0000 7f9a20a00640 1 -- 172.21.15.44:0/3286333136 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f9a040f19d0 con 0x7f9a1c1038e0 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a1b400640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9a000a3cb0 0x7f9a000a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 Processor -- start 2024-04-20T11:29:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea61600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a93ea673e0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a93e9797a0 con 0x55a93ea645f0 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a93ea67560 con 0x55a93ea5ba10 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a24be6740 10 client.0 _ll_register_callbacks cb 0x55a93e97bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a1b400640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9a000a3cb0 0x7f9a000a6170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a1401b040 tx=0x7f9a14005bd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.292+0000 7f9a12000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea61600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a12000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea61b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59694/0 (socket says 172.21.15.44:59694) 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a12000640 1 -- 172.21.15.44:0/2788473964 learned_addr learned my addr 172.21.15.44:0/2788473964 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1b400640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 -- 172.21.15.44:0/2788473964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 msgr2=0x55a93ea61b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea61b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 -- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 msgr2=0x55a93ea62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea62080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 -- 172.21.15.44:0/2788473964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93e79a7e0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1b400640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea61600 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c011670 tx=0x7f9a0c011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3e8763f35cb526 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.293+0000 7f9a08c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a08c00640 1 -- 172.21.15.44:0/2788473964 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01e020 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a14004cc0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a08c00640 1 -- 172.21.15.44:0/2788473964 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f9a0c00f8b0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a08c00640 1 -- 172.21.15.44:0/2788473964 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9a0c00fab0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a24be6740 10 client.17265 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a93e7faf70 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.294+0000 7f9a08c00640 1 -- 172.21.15.44:0/2788473964 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a14004cc0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a08c00640 1 -- 172.21.15.44:0/2788473964 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f9a0c019070 con 0x55a93ea5b0f0 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a24be6740 10 client.17265 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a24be6740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a24be6740 20 client.17265 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a24be6740 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93ea74960 0x55a93ea76d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a24be6740 4 client.17265 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93e9cd740 con 0x55a93ea74960 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.295+0000 7f9a12000640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93ea74960 0x55a93ea76d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.296+0000 7f9a12000640 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93ea74960 0x55a93ea76d90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9b48c48f209b77 server_cookie=c5910d10d5a09369 in_seq=0 out_seq=0 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.296+0000 7f9a08c00640 10 client.17265 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a08c00640 1 -- 172.21.15.44:0/2788473964 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a93e9cd740 con 0x55a93ea74960 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a08c00640 10 client.17265 handle_command_reply: tid=0 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 client.17265 shutdown 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 msgr2=0x55a93ea61600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea61600 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c011670 tx=0x7f9a0c011a70 comp rx=0 tx=0).stop 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 shutdown_connections 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93ea74960 0x55a93ea76d90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea61b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 --2- 172.21.15.44:0/2788473964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea61600 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a08c00640 0 client.17265 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 >> 172.21.15.44:0/2788473964 conn(0x55a93ea55820 msgr2=0x55a93ea57570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 shutdown_connections 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 1 -- 172.21.15.44:0/2788473964 wait complete. 2024-04-20T11:29:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.297+0000 7f9a24be6740 20 client.17265 trim_cache size 0 max 16384 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.317+0000 7f9a24be6740 1 Processor -- start 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.317+0000 7f9a24be6740 1 -- start start 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 0x55a93ea63cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 0x55a93ea702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea645f0 0x55a93ea7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a24be6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1afc60 con 0x55a93ea645f0 2024-04-20T11:29:34.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a24be6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b17a0 con 0x55a93ea5ba10 2024-04-20T11:29:34.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a24be6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b0330 con 0x55a93ea5b0f0 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 0x55a93ea63cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a12000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 0x55a93ea702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a12000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 0x55a93ea702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:33090/0 (socket says 172.21.15.44:33090) 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a12000640 1 -- 172.21.15.44:0/3675298500 learned_addr learned my addr 172.21.15.44:0/3675298500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 0x55a93ea63cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59698/0 (socket says 172.21.15.44:59698) 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1b400640 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea645f0 0x55a93ea7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 -- 172.21.15.44:0/3675298500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 msgr2=0x55a93ea702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 0x55a93ea702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 -- 172.21.15.44:0/3675298500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea645f0 msgr2=0x55a93ea7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea645f0 0x55a93ea7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a1aa00640 1 -- 172.21.15.44:0/3675298500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93ea5b840 con 0x55a93ea5b0f0 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.318+0000 7f9a12000640 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 0x55a93ea702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:34.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.319+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 0x55a93ea63cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009e20 tx=0x7f9a0c00fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54f9ccea65b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.319+0000 7f9a0b400640 1 -- 172.21.15.44:0/3675298500 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c000e60 con 0x55a93ea5b0f0 2024-04-20T11:29:34.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.319+0000 7f9a0b400640 1 -- 172.21.15.44:0/3675298500 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a0c005540 con 0x55a93ea5b0f0 2024-04-20T11:29:34.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.319+0000 7f9a0b400640 1 -- 172.21.15.44:0/3675298500 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01e070 con 0x55a93ea5b0f0 2024-04-20T11:29:34.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 -- 172.21.15.44:0/3675298500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 msgr2=0x55a93ea63cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 0x55a93ea63cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009e20 tx=0x7f9a0c00fa40 comp rx=0 tx=0).stop 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 -- 172.21.15.44:0/3675298500 shutdown_connections 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea645f0 0x55a93ea7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea5ba10 0x55a93ea702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3675298500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5b0f0 0x55a93ea63cc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 -- 172.21.15.44:0/3675298500 >> 172.21.15.44:0/3675298500 conn(0x55a93ea55820 msgr2=0x55a93ea574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 -- 172.21.15.44:0/3675298500 shutdown_connections 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 -- 172.21.15.44:0/3675298500 wait complete. 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.320+0000 7f9a24be6740 1 -- start start 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 Processor -- start 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea6ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea68630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b10d0 con 0x55a93ea5b0f0 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b0a00 con 0x55a93ea645f0 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1adc50 con 0x55a93ea5ba10 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a24be6740 10 client.0 _ll_register_callbacks cb 0x7f9a0c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea6ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea6ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51636/0 (socket says 172.21.15.44:51636) 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.321+0000 7f9a1aa00640 1 -- 172.21.15.44:0/3697516729 learned_addr learned my addr 172.21.15.44:0/3697516729 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a12000640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a1b400640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea68630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a12000640 1 -- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 msgr2=0x55a93ea68630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a12000640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea68630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a12000640 1 -- 172.21.15.44:0/3697516729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 msgr2=0x55a93ea6ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a12000640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea6ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.322+0000 7f9a12000640 1 -- 172.21.15.44:0/3697516729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93e692170 con 0x55a93ea5ba10 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.323+0000 7f9a12000640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea68020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a040bd870 tx=0x7f9a04005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a779ac9c2c827212 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.323+0000 7f9a08c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.323+0000 7f9a08c00640 1 -- 172.21.15.44:0/3697516729 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a040d6020 con 0x55a93ea5ba10 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.323+0000 7f9a08c00640 1 -- 172.21.15.44:0/3697516729 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f9a0402ca20 con 0x55a93ea5ba10 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.323+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0c005540 con 0x55a93ea5ba10 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.323+0000 7f9a08c00640 1 -- 172.21.15.44:0/3697516729 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9a0402ce60 con 0x55a93ea5ba10 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 10 client.27394 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:34.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a93eadae70 con 0x55a93ea5ba10 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a08c00640 1 -- 172.21.15.44:0/3697516729 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a0c005540 con 0x55a93ea5ba10 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a08c00640 1 -- 172.21.15.44:0/3697516729 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f9a0403c030 con 0x55a93ea5ba10 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 10 client.27394 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 10 client.27394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 20 client.27394 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93eadb5a0 0x55a93eadd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 4 client.27394 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.324+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a93e9c75b0 con 0x55a93eadb5a0 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.325+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93eadb5a0 0x55a93eadd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.325+0000 7f9a1aa00640 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93eadb5a0 0x55a93eadd9c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f559b33fd2b2ce2 server_cookie=e4624f1d52ea71c8 in_seq=0 out_seq=0 2024-04-20T11:29:34.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.325+0000 7f9a08c00640 10 client.27394 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.326+0000 7f9a08c00640 1 -- 172.21.15.44:0/3697516729 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a93e9c75b0 con 0x55a93eadb5a0 2024-04-20T11:29:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.326+0000 7f9a08c00640 10 client.27394 handle_command_reply: tid=0 2024-04-20T11:29:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.326+0000 7f9a24be6740 1 client.27394 shutdown 2024-04-20T11:29:34.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 msgr2=0x55a93ea68020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea68020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a040bd870 tx=0x7f9a04005f90 comp rx=0 tx=0).stop 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 shutdown_connections 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55a93eadb5a0 0x55a93eadd9c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a93ea645f0 0x55a93ea68630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a93ea5ba10 0x55a93ea68020 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 --2- 172.21.15.44:0/3697516729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a93ea5b0f0 0x55a93ea6ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 >> 172.21.15.44:0/3697516729 conn(0x55a93ea55820 msgr2=0x55a93ea54170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a08c00640 0 client.27394 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 shutdown_connections 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 1 -- 172.21.15.44:0/3697516729 wait complete. 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.327+0000 7f9a24be6740 20 client.27394 trim_cache size 0 max 16384 2024-04-20T11:29:34.328 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:29:34.329 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:29:34.329 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "bfbdeb3a-cfe4-4ca3-bdb4-a38584b22298", 2024-04-20T11:29:34.329 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:29:34.329 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9a000a3cb0 msgr2=0x7f9a000a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9a000a3cb0 0x7f9a000a6170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a1401b040 tx=0x7f9a14005bd0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c1038e0 msgr2=0x7f9a1c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c1a0be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a0402c150 tx=0x7f9a0402c630 comp rx=0 tx=0).stop 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 shutdown_connections 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9a000a3cb0 0x7f9a000a6170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c10c3e0 0x7f9a1c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c1a0be0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 --2- 172.21.15.44:0/3286333136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a1c102fc0 0x7f9a1c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 >> 172.21.15.44:0/3286333136 conn(0x7f9a1c0fc7e0 msgr2=0x7f9a1c100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 shutdown_connections 2024-04-20T11:29:34.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.328+0000 7f9a1be00640 1 -- 172.21.15.44:0/3286333136 wait complete. 2024-04-20T11:29:34.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:29:34.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:29:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 -- 172.21.15.44:0/962609570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5584108ea0 msgr2=0x7f55841152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 --2- 172.21.15.44:0/962609570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5584108ea0 0x7f55841152c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f557c009900 tx=0x7f557c02bcd0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 -- 172.21.15.44:0/962609570 shutdown_connections 2024-04-20T11:29:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 --2- 172.21.15.44:0/962609570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5584108ea0 0x7f55841152c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 --2- 172.21.15.44:0/962609570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108500 0x7f5584108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 --2- 172.21.15.44:0/962609570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f558410d3d0 0x7f558410d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 -- 172.21.15.44:0/962609570 >> 172.21.15.44:0/962609570 conn(0x7f5584077d80 msgr2=0x7f5584106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 -- 172.21.15.44:0/962609570 shutdown_connections 2024-04-20T11:29:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.565+0000 7f558a800640 1 -- 172.21.15.44:0/962609570 wait complete. 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.566+0000 7f558a800640 1 Processor -- start 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.566+0000 7f558a800640 1 -- start start 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f558a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5584108500 0x7f558419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f558a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108ea0 0x7f558419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f558a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558410d3d0 0x7f55841a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f558a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5584118980 con 0x7f558410d3d0 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f558a800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5584118800 con 0x7f5584108ea0 2024-04-20T11:29:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f558a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5584118b00 con 0x7f5584108500 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f5589e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558410d3d0 0x7f55841a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f5589400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5584108500 0x7f558419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f5589400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5584108500 0x7f558419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59724/0 (socket says 172.21.15.44:59724) 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f5588a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108ea0 0x7f558419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f5589400640 1 -- 172.21.15.44:0/2684100410 learned_addr learned my addr 172.21.15.44:0/2684100410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.567+0000 7f5589e00640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558410d3d0 0x7f55841a3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51652/0 (socket says 172.21.15.44:51652) 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5588a00640 1 -- 172.21.15.44:0/2684100410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5584108500 msgr2=0x7f558419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5588a00640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5584108500 0x7f558419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5588a00640 1 -- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558410d3d0 msgr2=0x7f55841a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5588a00640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558410d3d0 0x7f55841a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5588a00640 1 -- 172.21.15.44:0/2684100410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55841a3d40 con 0x7f5584108ea0 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5588a00640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108ea0 0x7f558419f2e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f557000e9f0 tx=0x7f557000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5582000640 1 -- 172.21.15.44:0/2684100410 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557000ccc0 con 0x7f5584108ea0 2024-04-20T11:29:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5582000640 1 -- 172.21.15.44:0/2684100410 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5570010930 con 0x7f5584108ea0 2024-04-20T11:29:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55841a4030 con 0x7f5584108ea0 2024-04-20T11:29:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.568+0000 7f5582000640 1 -- 172.21.15.44:0/2684100410 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55700189f0 con 0x7f5584108ea0 2024-04-20T11:29:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.569+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55841ab8c0 con 0x7f5584108ea0 2024-04-20T11:29:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 Processor -- start 2024-04-20T11:29:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 -- start start 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f5582000640 1 -- 172.21.15.44:0/2684100410 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5570010ae0 con 0x7f5584108ea0 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1648760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d1651340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d1653c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649d15ae150 con 0x5649d1648380 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649d1554370 con 0x5649d1648ca0 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f558eb68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649d15aa110 con 0x5649d1651880 2024-04-20T11:29:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f5589e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1648760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.572+0000 7f5588a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d1653c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d1651340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1648760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51660/0 (socket says 172.21.15.44:51660) 2024-04-20T11:29:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589e00640 1 -- 172.21.15.44:0/2603856915 learned_addr learned my addr 172.21.15.44:0/2603856915 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 -- 172.21.15.44:0/2603856915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 msgr2=0x5649d1653c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d1653c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 -- 172.21.15.44:0/2603856915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 msgr2=0x5649d1648760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1648760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 -- 172.21.15.44:0/2603856915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d138a7d0 con 0x5649d1648ca0 2024-04-20T11:29:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589e00640 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1648760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:34.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.573+0000 7f5589400640 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d1651340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5574009f70 tx=0x7f5574011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=795358e638976ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.574+0000 7f557a600640 1 -- 172.21.15.44:0/2603856915 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574003b70 con 0x5649d1648ca0 2024-04-20T11:29:34.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.574+0000 7f557a600640 1 -- 172.21.15.44:0/2603856915 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5574003d10 con 0x5649d1648ca0 2024-04-20T11:29:34.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.574+0000 7f557a600640 1 -- 172.21.15.44:0/2603856915 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574012bb0 con 0x5649d1648ca0 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.574+0000 7f5582000640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55680a3d20 0x7f55680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.574+0000 7f5582000640 1 -- 172.21.15.44:0/2684100410 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f55700d0ac0 con 0x7f5584108ea0 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.574+0000 7f5589e00640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55680a3d20 0x7f55680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 -- 172.21.15.44:0/2603856915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 msgr2=0x5649d1651340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d1651340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5574009f70 tx=0x7f5574011870 comp rx=0 tx=0).stop 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 -- 172.21.15.44:0/2603856915 shutdown_connections 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d1653c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d1651340 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 --2- 172.21.15.44:0/2603856915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1648760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 -- 172.21.15.44:0/2603856915 >> 172.21.15.44:0/2603856915 conn(0x5649d1642ab0 msgr2=0x5649d1642e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 -- 172.21.15.44:0/2603856915 shutdown_connections 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f5589e00640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55680a3d20 0x7f55680a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f557c005ec0 tx=0x7f557c005a40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 -- 172.21.15.44:0/2603856915 wait complete. 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.575+0000 7f558eb68740 1 -- start start 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 Processor -- start 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648380 0x5649d164e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d164ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 0x5649d164f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649d1654670 con 0x5649d1651880 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649d1567740 con 0x5649d1648ca0 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649d16547f0 con 0x5649d1648380 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.576+0000 7f558eb68740 10 client.0 _ll_register_callbacks cb 0x5649d1562b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 0x5649d164f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648380 0x5649d164e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 0x5649d164f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51668/0 (socket says 172.21.15.44:51668) 2024-04-20T11:29:34.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 -- 172.21.15.44:0/3203514898 learned_addr learned my addr 172.21.15.44:0/3203514898 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5588a00640 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d164ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 -- 172.21.15.44:0/3203514898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648380 msgr2=0x5649d164e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648380 0x5649d164e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 -- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 msgr2=0x5649d164ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d164ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 -- 172.21.15.44:0/3203514898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d12c03f0 con 0x5649d1651880 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f5589e00640 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 0x5649d164f1f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f557c005ab0 tx=0x7f557c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54763914e98b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.577+0000 7f556ea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f556ea00640 1 -- 172.21.15.44:0/3203514898 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c048040 con 0x5649d1651880 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5574003d10 con 0x5649d1651880 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f556ea00640 1 -- 172.21.15.44:0/3203514898 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f557c004380 con 0x5649d1651880 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f556ea00640 1 -- 172.21.15.44:0/3203514898 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f557c0046c0 con 0x5649d1651880 2024-04-20T11:29:34.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f558eb68740 10 client.17283 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5649d12800f0 con 0x5649d1651880 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.578+0000 7f556ea00640 1 -- 172.21.15.44:0/3203514898 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5574003d10 con 0x5649d1651880 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f556ea00640 1 -- 172.21.15.44:0/3203514898 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f557c003bc0 con 0x5649d1651880 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f558eb68740 10 client.17283 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f558eb68740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f558eb68740 20 client.17283 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f558eb68740 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d1661b00 0x5649d1663f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f558eb68740 4 client.17283 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d15bbb70 con 0x5649d1661b00 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.579+0000 7f5589400640 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d1661b00 0x5649d1663f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.580+0000 7f5589400640 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d1661b00 0x5649d1663f30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeda9abd686273d3 server_cookie=9c5481ec6eebffd in_seq=0 out_seq=0 2024-04-20T11:29:34.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.580+0000 7f556ea00640 10 client.17283 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f556ea00640 1 -- 172.21.15.44:0/3203514898 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649d15bbb70 con 0x5649d1661b00 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f556ea00640 10 client.17283 handle_command_reply: tid=0 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 client.17283 shutdown 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 msgr2=0x5649d164f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 0x5649d164f1f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f557c005ab0 tx=0x7f557c02bcd0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 shutdown_connections 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d1661b00 0x5649d1663f30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1651880 0x5649d164f1f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d164ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 --2- 172.21.15.44:0/3203514898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648380 0x5649d164e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f556ea00640 0 client.17283 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 >> 172.21.15.44:0/3203514898 conn(0x5649d1642ab0 msgr2=0x5649d1644800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 shutdown_connections 2024-04-20T11:29:34.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 1 -- 172.21.15.44:0/3203514898 wait complete. 2024-04-20T11:29:34.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.581+0000 7f558eb68740 20 client.17283 trim_cache size 0 max 16384 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.601+0000 7f558eb68740 1 Processor -- start 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.601+0000 7f558eb68740 1 -- start start 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f558eb68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1650ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f558eb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d165d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f558eb68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d164d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f558eb68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b40b0 con 0x5649d1648380 2024-04-20T11:29:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f558eb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b5bf0 con 0x5649d1648ca0 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f558eb68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b4780 con 0x5649d1651880 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f5589400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1650ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f5588a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d165d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f5589400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1650ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51676/0 (socket says 172.21.15.44:51676) 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f5589400640 1 -- 172.21.15.44:0/1320780684 learned_addr learned my addr 172.21.15.44:0/1320780684 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.602+0000 7f5589e00640 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d164d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5588a00640 1 -- 172.21.15.44:0/1320780684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 msgr2=0x5649d164d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5588a00640 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d164d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5588a00640 1 -- 172.21.15.44:0/1320780684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 msgr2=0x5649d1650ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5588a00640 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1650ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5588a00640 1 -- 172.21.15.44:0/1320780684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d16428e0 con 0x5649d1648ca0 2024-04-20T11:29:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5589400640 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1650ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:29:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5588a00640 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d165d530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f55700ce1c0 tx=0x7f55700d4570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a856898071633a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5579c00640 1 -- 172.21.15.44:0/1320780684 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557009c630 con 0x5649d1648ca0 2024-04-20T11:29:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5579c00640 1 -- 172.21.15.44:0/1320780684 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f55700bd0b0 con 0x5649d1648ca0 2024-04-20T11:29:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.603+0000 7f5579c00640 1 -- 172.21.15.44:0/1320780684 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557009c450 con 0x5649d1648ca0 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 -- 172.21.15.44:0/1320780684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 msgr2=0x5649d165d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d165d530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f55700ce1c0 tx=0x7f55700d4570 comp rx=0 tx=0).stop 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 -- 172.21.15.44:0/1320780684 shutdown_connections 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1651880 0x5649d164d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1648ca0 0x5649d165d530 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 --2- 172.21.15.44:0/1320780684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1650ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 -- 172.21.15.44:0/1320780684 >> 172.21.15.44:0/1320780684 conn(0x5649d1642ab0 msgr2=0x5649d1644720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 -- 172.21.15.44:0/1320780684 shutdown_connections 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 -- 172.21.15.44:0/1320780684 wait complete. 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.604+0000 7f558eb68740 1 -- start start 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 Processor -- start 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1655720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648ca0 0x5649d1655c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1651880 0x5649d16561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b5520 con 0x5649d1648380 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b4e50 con 0x5649d1651880 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55841b20a0 con 0x5649d1648ca0 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f558eb68740 10 client.0 _ll_register_callbacks cb 0x7f557c02c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:29:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f5589400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1655720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f5589e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1651880 0x5649d16561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f5589400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1655720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51692/0 (socket says 172.21.15.44:51692) 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f5589400640 1 -- 172.21.15.44:0/3829594558 learned_addr learned my addr 172.21.15.44:0/3829594558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.605+0000 7f5588a00640 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648ca0 0x5649d1655c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f5589e00640 1 -- 172.21.15.44:0/3829594558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648ca0 msgr2=0x5649d1655c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f5589e00640 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648ca0 0x5649d1655c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f5589e00640 1 -- 172.21.15.44:0/3829594558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 msgr2=0x5649d1655720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f5589e00640 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1655720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f5589e00640 1 -- 172.21.15.44:0/3829594558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d16c7340 con 0x5649d1651880 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f5589e00640 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1651880 0x5649d16561c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f557c02c840 tx=0x7f557c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ac6472f33d4d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f556ea00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f556ea00640 1 -- 172.21.15.44:0/3829594558 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c048040 con 0x5649d1651880 2024-04-20T11:29:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f556ea00640 1 -- 172.21.15.44:0/3829594558 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f557c0409e0 con 0x5649d1651880 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.606+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55700bd0b0 con 0x5649d1651880 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f556ea00640 1 -- 172.21.15.44:0/3829594558 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f557c040be0 con 0x5649d1651880 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 10 client.27413 fetch_fsmap learned FSMap version 55 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5649d16c8650 con 0x5649d1651880 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f556ea00640 1 -- 172.21.15.44:0/3829594558 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f55700bd0b0 con 0x5649d1651880 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f556ea00640 1 -- 172.21.15.44:0/3829594558 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f557c003680 con 0x5649d1651880 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 10 client.27413 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 10 client.27413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 20 client.27413 populate_metadata read hostname 'smithi044' 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d16c8dc0 0x5649d16cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 4 client.27413 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.607+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5649d15b59e0 con 0x5649d16c8dc0 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.608+0000 7f5589400640 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d16c8dc0 0x5649d16cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.608+0000 7f5589400640 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d16c8dc0 0x5649d16cb1e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43139aaeebd34d57 server_cookie=8a39f1989b6338e9 in_seq=0 out_seq=0 2024-04-20T11:29:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.608+0000 7f556ea00640 10 client.27413 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.608+0000 7f556ea00640 1 -- 172.21.15.44:0/3829594558 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5649d15b59e0 con 0x5649d16c8dc0 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.608+0000 7f556ea00640 10 client.27413 handle_command_reply: tid=0 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.608+0000 7f558eb68740 1 client.27413 shutdown 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1651880 msgr2=0x5649d16561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1651880 0x5649d16561c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f557c02c840 tx=0x7f557c005830 comp rx=0 tx=0).stop 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 shutdown_connections 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5649d16c8dc0 0x5649d16cb1e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649d1651880 0x5649d16561c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649d1648ca0 0x5649d1655c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f556ea00640 0 client.27413 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 --2- 172.21.15.44:0/3829594558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d1648380 0x5649d1655720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 >> 172.21.15.44:0/3829594558 conn(0x5649d1642ab0 msgr2=0x5649d164e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 shutdown_connections 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 1 -- 172.21.15.44:0/3829594558 wait complete. 2024-04-20T11:29:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.609+0000 7f558eb68740 20 client.27413 trim_cache size 0 max 16384 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: "bfbdeb3a-cfe4-4ca3-bdb4-a38584b22298": { 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "bfbdeb3a-cfe4-4ca3-bdb4-a38584b22298", 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:29:34.614 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:29:34.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55680a3d20 msgr2=0x7f55680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55680a3d20 0x7f55680a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f557c005ec0 tx=0x7f557c005a40 comp rx=0 tx=0).stop 2024-04-20T11:29:34.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108ea0 msgr2=0x7f558419f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:29:34.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108ea0 0x7f558419f2e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f557000e9f0 tx=0x7f557000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 shutdown_connections 2024-04-20T11:29:34.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55680a3d20 0x7f55680a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f558410d3d0 0x7f55841a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5584108ea0 0x7f558419f2e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 --2- 172.21.15.44:0/2684100410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5584108500 0x7f558419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:29:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 >> 172.21.15.44:0/2684100410 conn(0x7f5584077d80 msgr2=0x7f55840782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:29:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 shutdown_connections 2024-04-20T11:29:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:29:34.610+0000 7f558a800640 1 -- 172.21.15.44:0/2684100410 wait complete. 2024-04-20T11:29:34.628 INFO:tasks.cephfs.filesystem:scrub status for tag:bfbdeb3a-cfe4-4ca3-bdb4-a38584b22298 - {'path': '/', 'tag': 'bfbdeb3a-cfe4-4ca3-bdb4-a38584b22298', 'options': 'recursive,force'} 2024-04-20T11:29:34.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:34 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:29:34.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:34 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:34.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:34 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:29:34.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:34 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:34.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:34 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:29:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:34 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:34 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:29:34.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:34 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:34.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:34 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:29:34.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:34 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:34.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:34 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:29:34.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:34 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:29:35.881 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:35 smithi165 ceph-mon[21975]: pgmap v214: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:35.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:35 smithi078 ceph-mon[21464]: pgmap v214: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:35.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:35 smithi044 ceph-mon[16855]: pgmap v214: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:37 smithi078 ceph-mon[21464]: pgmap v215: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:37.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:37 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:29:37.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:37 smithi044 ceph-mon[16855]: pgmap v215: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:37.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:37 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:29:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:37 smithi165 ceph-mon[21975]: pgmap v215: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:37 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:29:39.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:39 smithi078 ceph-mon[21464]: pgmap v216: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:39 smithi044 ceph-mon[16855]: pgmap v216: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:39 smithi165 ceph-mon[21975]: pgmap v216: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:41 smithi078 ceph-mon[21464]: pgmap v217: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:41 smithi044 ceph-mon[16855]: pgmap v217: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:41 smithi165 ceph-mon[21975]: pgmap v217: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:43.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:43 smithi078 ceph-mon[21464]: pgmap v218: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:43.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:43 smithi044 ceph-mon[16855]: pgmap v218: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:43 smithi165 ceph-mon[21975]: pgmap v218: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:45.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:45 smithi078 ceph-mon[21464]: pgmap v219: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:45.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:45 smithi044 ceph-mon[16855]: pgmap v219: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:45 smithi165 ceph-mon[21975]: pgmap v219: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:47.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:47 smithi078 ceph-mon[21464]: pgmap v220: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:47 smithi044 ceph-mon[16855]: pgmap v220: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:47 smithi165 ceph-mon[21975]: pgmap v220: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:29:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:49 smithi078 ceph-mon[21464]: pgmap v221: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:49.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:49 smithi044 ceph-mon[16855]: pgmap v221: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:49 smithi165 ceph-mon[21975]: pgmap v221: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:29:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:51 smithi078 ceph-mon[21464]: pgmap v222: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:51.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:51 smithi044 ceph-mon[16855]: pgmap v222: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:51 smithi165 ceph-mon[21975]: pgmap v222: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:53.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:53 smithi078 ceph-mon[21464]: pgmap v223: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:53.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:53 smithi044 ceph-mon[16855]: pgmap v223: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:53 smithi165 ceph-mon[21975]: pgmap v223: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:55.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:55 smithi078 ceph-mon[21464]: pgmap v224: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:55.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:55 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:29:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:55 smithi044 ceph-mon[16855]: pgmap v224: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:55.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:55 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:29:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:55 smithi165 ceph-mon[21975]: pgmap v224: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:55 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:29:56.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:56 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:29:56.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:56 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:29:56.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:56 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:29:56.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:56 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:29:56.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:56 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:29:56.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:56 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:56 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:56 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:56 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:57 smithi078 ceph-mon[21464]: pgmap v225: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:57.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:57 smithi044 ceph-mon[16855]: pgmap v225: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:57 smithi165 ceph-mon[21975]: pgmap v225: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:29:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:29:59 smithi078 ceph-mon[21464]: pgmap v226: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:59.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:29:59 smithi044 ceph-mon[16855]: pgmap v226: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:29:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:29:59 smithi165 ceph-mon[21975]: pgmap v226: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:00.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:00 smithi078 ceph-mon[21464]: overall HEALTH_OK 2024-04-20T11:30:00.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:00 smithi044 ceph-mon[16855]: overall HEALTH_OK 2024-04-20T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:00 smithi165 ceph-mon[21975]: overall HEALTH_OK 2024-04-20T11:30:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:01 smithi078 ceph-mon[21464]: pgmap v227: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:01.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:01 smithi044 ceph-mon[16855]: pgmap v227: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:01 smithi165 ceph-mon[21975]: pgmap v227: 97 pgs: 97 active+clean; 267 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:03 smithi078 ceph-mon[21464]: pgmap v228: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:30:03.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:03 smithi044 ceph-mon[16855]: pgmap v228: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:30:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:03 smithi165 ceph-mon[21975]: pgmap v228: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:30:04.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:30:04.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:30:04.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.851+0000 7fb48e400640 1 -- 172.21.15.44:0/21589419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488102ba0 msgr2=0x7fb488106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.851+0000 7fb48e400640 1 --2- 172.21.15.44:0/21589419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488102ba0 0x7fb488106be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb470009900 tx=0x7fb47002bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 -- 172.21.15.44:0/21589419 shutdown_connections 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 --2- 172.21.15.44:0/21589419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881072c0 0x7fb48810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 --2- 172.21.15.44:0/21589419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488102ba0 0x7fb488106be0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 --2- 172.21.15.44:0/21589419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4881021f0 0x7fb4881025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 -- 172.21.15.44:0/21589419 >> 172.21.15.44:0/21589419 conn(0x7fb4880fda70 msgr2=0x7fb4880ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 -- 172.21.15.44:0/21589419 shutdown_connections 2024-04-20T11:30:04.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.852+0000 7fb48e400640 1 -- 172.21.15.44:0/21589419 wait complete. 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 Processor -- start 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 -- start start 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 0x7fb488074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4881072c0 0x7fb4880747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488074ce0 0x7fb4881a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb488112440 con 0x7fb488074ce0 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881122c0 con 0x7fb4881021f0 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.853+0000 7fb48e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881125c0 con 0x7fb4881072c0 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 0x7fb488074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 0x7fb488074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41726/0 (socket says 172.21.15.44:41726) 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 -- 172.21.15.44:0/2732724726 learned_addr learned my addr 172.21.15.44:0/2732724726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487e00640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488074ce0 0x7fb4881a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb486a00640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4881072c0 0x7fb4880747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 -- 172.21.15.44:0/2732724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4881072c0 msgr2=0x7fb4880747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4881072c0 0x7fb4880747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 -- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488074ce0 msgr2=0x7fb4881a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488074ce0 0x7fb4881a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 -- 172.21.15.44:0/2732724726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4881a7290 con 0x7fb4881021f0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487e00640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488074ce0 0x7fb4881a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.854+0000 7fb487400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 0x7fb488074260 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb47800eb90 tx=0x7fb47800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.855+0000 7fb47fe00640 1 -- 172.21.15.44:0/2732724726 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478004d20 con 0x7fb4881021f0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.855+0000 7fb47fe00640 1 -- 172.21.15.44:0/2732724726 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb47800ee90 con 0x7fb4881021f0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.855+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4881a7580 con 0x7fb4881021f0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.855+0000 7fb47fe00640 1 -- 172.21.15.44:0/2732724726 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780108b0 con 0x7fb4881021f0 2024-04-20T11:30:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.855+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881a7a50 con 0x7fb4881021f0 2024-04-20T11:30:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.858+0000 7fb4911e7740 1 Processor -- start 2024-04-20T11:30:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.858+0000 7fb4911e7740 1 -- start start 2024-04-20T11:30:04.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.858+0000 7fb47fe00640 1 -- 172.21.15.44:0/2732724726 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb478005020 con 0x7fb4881021f0 2024-04-20T11:30:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed6d9b0 0x560beed6dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed76970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed76eb0 0x560beed792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb4911e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560beeccb150 con 0x560beed6e2d0 2024-04-20T11:30:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb4911e7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560beec71370 con 0x560beed76eb0 2024-04-20T11:30:04.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb4911e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560beecc7110 con 0x560beed6d9b0 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed76970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed76970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44004/0 (socket says 172.21.15.44:44004) 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 -- 172.21.15.44:0/3236025790 learned_addr learned my addr 172.21.15.44:0/3236025790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487e00640 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed6d9b0 0x560beed6dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb486a00640 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed76eb0 0x560beed792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 -- 172.21.15.44:0/3236025790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed6d9b0 msgr2=0x560beed6dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed6d9b0 0x560beed6dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 -- 172.21.15.44:0/3236025790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed76eb0 msgr2=0x560beed792a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed76eb0 0x560beed792a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.859+0000 7fb487400640 1 -- 172.21.15.44:0/3236025790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560beeaa77d0 con 0x560beed6e2d0 2024-04-20T11:30:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.860+0000 7fb487400640 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed76970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb4780ce4c0 tx=0x7fb4780d4e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc09ccb59e99c0ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.860+0000 7fb47cc00640 1 -- 172.21.15.44:0/3236025790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780e4070 con 0x560beed6e2d0 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.860+0000 7fb47fe00640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb46c0a3d20 0x7fb46c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.860+0000 7fb47cc00640 1 -- 172.21.15.44:0/3236025790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb4780d9b90 con 0x560beed6e2d0 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.860+0000 7fb47cc00640 1 -- 172.21.15.44:0/3236025790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780de6a0 con 0x560beed6e2d0 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.860+0000 7fb47fe00640 1 -- 172.21.15.44:0/2732724726 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fb4780d0e50 con 0x7fb4881021f0 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb487e00640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb46c0a3d20 0x7fb46c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 -- 172.21.15.44:0/3236025790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 msgr2=0x560beed76970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed76970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb4780ce4c0 tx=0x7fb4780d4e60 comp rx=0 tx=0).stop 2024-04-20T11:30:04.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 -- 172.21.15.44:0/3236025790 shutdown_connections 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed76eb0 0x560beed792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed76970 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3236025790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed6d9b0 0x560beed6dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.861+0000 7fb4911e7740 1 -- 172.21.15.44:0/3236025790 >> 172.21.15.44:0/3236025790 conn(0x560beed680e0 msgr2=0x560beed684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.862+0000 7fb487e00640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb46c0a3d20 0x7fb46c0a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb480012560 tx=0x7fb480005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.862+0000 7fb4911e7740 1 -- 172.21.15.44:0/3236025790 shutdown_connections 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.862+0000 7fb4911e7740 1 -- 172.21.15.44:0/3236025790 wait complete. 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.862+0000 7fb4911e7740 1 -- start start 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.862+0000 7fb4911e7740 1 Processor -- start 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560beed79ca0 con 0x560beed6e2d0 2024-04-20T11:30:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560beec84740 con 0x560beed6d9b0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560beed79e20 con 0x560beed76eb0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb4911e7740 10 client.0 _ll_register_callbacks cb 0x560beec7fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb486a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed73fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41750/0 (socket says 172.21.15.44:41750) 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 -- 172.21.15.44:0/2996061125 learned_addr learned my addr 172.21.15.44:0/2996061125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487e00640 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 -- 172.21.15.44:0/2996061125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 msgr2=0x560beed74a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 -- 172.21.15.44:0/2996061125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 msgr2=0x560beed744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.863+0000 7fb487400640 1 -- 172.21.15.44:0/2996061125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bee9dd3f0 con 0x560beed6d9b0 2024-04-20T11:30:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb487400640 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed73fa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb4780d94b0 tx=0x7fb4780dff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708f1d6bc56e80ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb475600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb475600640 1 -- 172.21.15.44:0/2996061125 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478099070 con 0x560beed6d9b0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb475600640 1 -- 172.21.15.44:0/2996061125 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fb478005640 con 0x560beed6d9b0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4780d9b90 con 0x560beed6d9b0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb475600640 1 -- 172.21.15.44:0/2996061125 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb47809c730 con 0x560beed6d9b0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb4911e7740 10 client.27425 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.864+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560bee99d0f0 con 0x560beed6d9b0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb475600640 1 -- 172.21.15.44:0/2996061125 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb4780d9b90 con 0x560beed6d9b0 2024-04-20T11:30:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb475600640 1 -- 172.21.15.44:0/2996061125 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fb478099440 con 0x560beed6d9b0 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb4911e7740 10 client.27425 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb4911e7740 10 client.27425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb4911e7740 20 client.27425 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beed87180 0x560beed89570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb4911e7740 4 client.27425 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560beecd8b70 con 0x560beed87180 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.865+0000 7fb486a00640 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beed87180 0x560beed89570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.866+0000 7fb486a00640 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beed87180 0x560beed89570 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4d50355c6867fa server_cookie=38686befbd6bc089 in_seq=0 out_seq=0 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.866+0000 7fb475600640 10 client.27425 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb475600640 1 -- 172.21.15.44:0/2996061125 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560beecd8b70 con 0x560beed87180 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb475600640 10 client.27425 handle_command_reply: tid=0 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 client.27425 shutdown 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 msgr2=0x560beed73fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed73fa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb4780d94b0 tx=0x7fb4780dff00 comp rx=0 tx=0).stop 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 shutdown_connections 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beed87180 0x560beed89570 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2996061125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed73fa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 >> 172.21.15.44:0/2996061125 conn(0x560beed680e0 msgr2=0x560beed69e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb475600640 0 client.27425 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 shutdown_connections 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 1 -- 172.21.15.44:0/2996061125 wait complete. 2024-04-20T11:30:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.867+0000 7fb4911e7740 20 client.27425 trim_cache size 0 max 16384 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 Processor -- start 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 -- start start 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed73100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881abb20 con 0x560beed6e2d0 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881ad660 con 0x560beed6d9b0 2024-04-20T11:30:04.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.887+0000 7fb4911e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881ac1f0 con 0x560beed76eb0 2024-04-20T11:30:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed73100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed73100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44010/0 (socket says 172.21.15.44:44010) 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 -- 172.21.15.44:0/2968684553 learned_addr learned my addr 172.21.15.44:0/2968684553 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb487400640 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb487e00640 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 -- 172.21.15.44:0/2968684553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 msgr2=0x560beed74670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 -- 172.21.15.44:0/2968684553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 msgr2=0x560beed766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb486a00640 1 -- 172.21.15.44:0/2968684553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560beed724f0 con 0x560beed6e2d0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.888+0000 7fb487400640 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.889+0000 7fb486a00640 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed73100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb4700096c0 tx=0x7fb470005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f73b22a4bd400 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.889+0000 7fb477e00640 1 -- 172.21.15.44:0/2968684553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47004f070 con 0x560beed6e2d0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.889+0000 7fb477e00640 1 -- 172.21.15.44:0/2968684553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb470004040 con 0x560beed6e2d0 2024-04-20T11:30:04.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.889+0000 7fb477e00640 1 -- 172.21.15.44:0/2968684553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470005480 con 0x560beed6e2d0 2024-04-20T11:30:04.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.889+0000 7fb4911e7740 1 -- 172.21.15.44:0/2968684553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 msgr2=0x560beed73100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.889+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed73100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb4700096c0 tx=0x7fb470005900 comp rx=0 tx=0).stop 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 -- 172.21.15.44:0/2968684553 shutdown_connections 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6e2d0 0x560beed73100 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 --2- 172.21.15.44:0/2968684553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6d9b0 0x560beed766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 -- 172.21.15.44:0/2968684553 >> 172.21.15.44:0/2968684553 conn(0x560beed680e0 msgr2=0x560beed73f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 -- 172.21.15.44:0/2968684553 shutdown_connections 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 -- 172.21.15.44:0/2968684553 wait complete. 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 -- start start 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.890+0000 7fb4911e7740 1 Processor -- start 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 0x560beed8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6e2d0 0x560beed8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed8aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881acf90 con 0x560beed6d9b0 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881ac8c0 con 0x560beed6e2d0 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881a9b10 con 0x560beed76eb0 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb4911e7740 10 client.0 _ll_register_callbacks cb 0x7fb4780e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 0x560beed8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 0x560beed8a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44024/0 (socket says 172.21.15.44:44024) 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 -- 172.21.15.44:0/3407001465 learned_addr learned my addr 172.21.15.44:0/3407001465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb486a00640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6e2d0 0x560beed8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487e00640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed8aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 -- 172.21.15.44:0/3407001465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 msgr2=0x560beed8aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed8aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 -- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6e2d0 msgr2=0x560beed8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6e2d0 0x560beed8a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb487400640 1 -- 172.21.15.44:0/3407001465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560beedec3b0 con 0x560beed6d9b0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.891+0000 7fb486a00640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6e2d0 0x560beed8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb487400640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 0x560beed8a030 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4780049d0 tx=0x7fb47809b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae02ba13b20483d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb475600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb475600640 1 -- 172.21.15.44:0/3407001465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47809c910 con 0x560beed6d9b0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb475600640 1 -- 172.21.15.44:0/3407001465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7fb4780ce800 con 0x560beed6d9b0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb470004040 con 0x560beed6d9b0 2024-04-20T11:30:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb475600640 1 -- 172.21.15.44:0/3407001465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb47809cba0 con 0x560beed6d9b0 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb4911e7740 10 client.17307 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560beeded6c0 con 0x560beed6d9b0 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.892+0000 7fb475600640 1 -- 172.21.15.44:0/3407001465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb470004040 con 0x560beed6d9b0 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb475600640 1 -- 172.21.15.44:0/3407001465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fb47809d030 con 0x560beed6d9b0 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb4911e7740 10 client.17307 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb4911e7740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb4911e7740 20 client.17307 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beededb40 0x560beedeff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb4911e7740 4 client.17307 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560beecd29e0 con 0x560beededb40 2024-04-20T11:30:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.893+0000 7fb486a00640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beededb40 0x560beedeff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb486a00640 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beededb40 0x560beedeff30 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a228b56b349ce574 server_cookie=6cefadb1232b8026 in_seq=0 out_seq=0 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb475600640 10 client.17307 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb475600640 1 -- 172.21.15.44:0/3407001465 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560beecd29e0 con 0x560beededb40 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb475600640 10 client.17307 handle_command_reply: tid=0 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 client.17307 shutdown 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 msgr2=0x560beed8a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 0x560beed8a030 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4780049d0 tx=0x7fb47809b2e0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 shutdown_connections 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x560beededb40 0x560beedeff30 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560beed76eb0 0x560beed8aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560beed6e2d0 0x560beed8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb475600640 0 client.17307 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 --2- 172.21.15.44:0/3407001465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560beed6d9b0 0x560beed8a030 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 >> 172.21.15.44:0/3407001465 conn(0x560beed680e0 msgr2=0x560beed82900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 shutdown_connections 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.894+0000 7fb4911e7740 1 -- 172.21.15.44:0/3407001465 wait complete. 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb4911e7740 20 client.17307 trim_cache size 0 max 16384 2024-04-20T11:30:04.900 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:30:04.901 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:30:04.901 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:30:04.901 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:30:04.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb46c0a3d20 msgr2=0x7fb46c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb46c0a3d20 0x7fb46c0a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb480012560 tx=0x7fb480005f90 comp rx=0 tx=0).stop 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 msgr2=0x7fb488074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 0x7fb488074260 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb47800eb90 tx=0x7fb47800c6a0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 shutdown_connections 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb46c0a3d20 0x7fb46c0a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb488074ce0 0x7fb4881a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4881072c0 0x7fb4880747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 --2- 172.21.15.44:0/2732724726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4881021f0 0x7fb488074260 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 >> 172.21.15.44:0/2732724726 conn(0x7fb4880fda70 msgr2=0x7fb4880fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.895+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 shutdown_connections 2024-04-20T11:30:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:04.896+0000 7fb48e400640 1 -- 172.21.15.44:0/2732724726 wait complete. 2024-04-20T11:30:04.914 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:30:04.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 -- 172.21.15.44:0/3621817249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c810c330 msgr2=0x7f31c810e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 --2- 172.21.15.44:0/3621817249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c810c330 0x7f31c810e7d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009ca0 tx=0x7f31c002c190 comp rx=0 tx=0).stop 2024-04-20T11:30:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 -- 172.21.15.44:0/3621817249 shutdown_connections 2024-04-20T11:30:05.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 --2- 172.21.15.44:0/3621817249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c810c330 0x7f31c810e7d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 --2- 172.21.15.44:0/3621817249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c8103830 0x7f31c810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 --2- 172.21.15.44:0/3621817249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c81032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 -- 172.21.15.44:0/3621817249 >> 172.21.15.44:0/3621817249 conn(0x7f31c80fc730 msgr2=0x7f31c80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.110+0000 7f31c7e00640 1 -- 172.21.15.44:0/3621817249 shutdown_connections 2024-04-20T11:30:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.111+0000 7f31c7e00640 1 -- 172.21.15.44:0/3621817249 wait complete. 2024-04-20T11:30:05.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.111+0000 7f31c7e00640 1 Processor -- start 2024-04-20T11:30:05.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 -- start start 2024-04-20T11:30:05.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c8100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8103830 0x7f31c81010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c810c330 0x7f31c81015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8112250 con 0x7f31c8102f10 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81120d0 con 0x7f31c8103830 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31c81123d0 con 0x7f31c810c330 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c8100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c8100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44046/0 (socket says 172.21.15.44:44046) 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c6a00640 1 -- 172.21.15.44:0/2352263219 learned_addr learned my addr 172.21.15.44:0/2352263219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31be000640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8103830 0x7f31c81010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.112+0000 7f31c7400640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c810c330 0x7f31c81015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c6a00640 1 -- 172.21.15.44:0/2352263219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c810c330 msgr2=0x7f31c81015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c6a00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c810c330 0x7f31c81015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c6a00640 1 -- 172.21.15.44:0/2352263219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8103830 msgr2=0x7f31c81010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c6a00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8103830 0x7f31c81010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c6a00640 1 -- 172.21.15.44:0/2352263219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c81a6b90 con 0x7f31c8102f10 2024-04-20T11:30:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c7400640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c810c330 0x7f31c81015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c6a00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c8100b60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31b800e9f0 tx=0x7f31b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b800cd80 con 0x7f31c8102f10 2024-04-20T11:30:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31b8010930 con 0x7f31c8102f10 2024-04-20T11:30:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8018b00 con 0x7f31c8102f10 2024-04-20T11:30:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.113+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c81a6e80 con 0x7f31c8102f10 2024-04-20T11:30:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.114+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c81a7390 con 0x7f31c8102f10 2024-04-20T11:30:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.115+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c80690f0 con 0x7f31c8102f10 2024-04-20T11:30:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.117+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f31b8018ca0 con 0x7f31c8102f10 2024-04-20T11:30:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.118+0000 7f31bfe00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31ac0a3dd0 0x7f31ac0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.118+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f31b80d0ab0 con 0x7f31c8102f10 2024-04-20T11:30:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.118+0000 7f31be000640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31ac0a3dd0 0x7f31ac0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.119+0000 7f31be000640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31ac0a3dd0 0x7f31ac0a6290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f31c81025a0 tx=0x7f31b0005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.123+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f31b809cae0 con 0x7f31c8102f10 2024-04-20T11:30:05.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.375+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c802d5d0 con 0x7f31c8102f10 2024-04-20T11:30:05.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.377+0000 7f31bfe00640 1 -- 172.21.15.44:0/2352263219 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7f31b80a0de0 con 0x7f31c8102f10 2024-04-20T11:30:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:30:05.378 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:30:05.378 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:30:05.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31ac0a3dd0 msgr2=0x7f31ac0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31ac0a3dd0 0x7f31ac0a6290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f31c81025a0 tx=0x7f31b0005f90 comp rx=0 tx=0).stop 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 msgr2=0x7f31c8100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c8100b60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31b800e9f0 tx=0x7f31b800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 shutdown_connections 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31c810c330 0x7f31c81015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8103830 0x7f31c81010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31ac0a3dd0 0x7f31ac0a6290 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 --2- 172.21.15.44:0/2352263219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31c8102f10 0x7f31c8100b60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 >> 172.21.15.44:0/2352263219 conn(0x7f31c80fc730 msgr2=0x7f31c80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 shutdown_connections 2024-04-20T11:30:05.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.380+0000 7f31c7e00640 1 -- 172.21.15.44:0/2352263219 wait complete. 2024-04-20T11:30:05.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:30:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 -- 172.21.15.44:0/651834658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd810e600 msgr2=0x7f4fd810e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 --2- 172.21.15.44:0/651834658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd810e600 0x7f4fd810e9e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009900 tx=0x7f4fc802bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 -- 172.21.15.44:0/651834658 shutdown_connections 2024-04-20T11:30:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 --2- 172.21.15.44:0/651834658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fd8108f50 0x7f4fd81153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 --2- 172.21.15.44:0/651834658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd8108590 0x7f4fd8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 --2- 172.21.15.44:0/651834658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd810e600 0x7f4fd810e9e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 -- 172.21.15.44:0/651834658 >> 172.21.15.44:0/651834658 conn(0x7f4fd80764a0 msgr2=0x7f4fd80788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 -- 172.21.15.44:0/651834658 shutdown_connections 2024-04-20T11:30:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.618+0000 7f4fdca00640 1 -- 172.21.15.44:0/651834658 wait complete. 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.619+0000 7f4fdca00640 1 Processor -- start 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.619+0000 7f4fdca00640 1 -- start start 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fdca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fd8108590 0x7f4fd819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fdca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 0x7f4fd819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fdca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd810e600 0x7f4fd81a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fdca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8118a20 con 0x7f4fd8108f50 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fdca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81188a0 con 0x7f4fd810e600 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fdca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8118ba0 con 0x7f4fd8108590 2024-04-20T11:30:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 0x7f4fd819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fd6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd810e600 0x7f4fd81a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 0x7f4fd819f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44058/0 (socket says 172.21.15.44:44058) 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fcd400640 1 -- 172.21.15.44:0/3096455526 learned_addr learned my addr 172.21.15.44:0/3096455526 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.620+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fd8108590 0x7f4fd819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fd6800640 1 -- 172.21.15.44:0/3096455526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fd8108590 msgr2=0x7f4fd819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fd6800640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fd8108590 0x7f4fd819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fd6800640 1 -- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 msgr2=0x7f4fd819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fd6800640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 0x7f4fd819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fd6800640 1 -- 172.21.15.44:0/3096455526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81a3e20 con 0x7f4fd810e600 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fcd400640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 0x7f4fd819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fd6800640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd810e600 0x7f4fd81a36f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009fd0 tx=0x7f4fd000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fcf400640 1 -- 172.21.15.44:0/3096455526 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000ce20 con 0x7f4fd810e600 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a4110 con 0x7f4fd810e600 2024-04-20T11:30:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fcf400640 1 -- 172.21.15.44:0/3096455526 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4fd0004b20 con 0x7f4fd810e600 2024-04-20T11:30:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fcf400640 1 -- 172.21.15.44:0/3096455526 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0010790 con 0x7f4fd810e600 2024-04-20T11:30:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.621+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81ab9a0 con 0x7f4fd810e600 2024-04-20T11:30:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fcf400640 1 -- 172.21.15.44:0/3096455526 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4fd0004cd0 con 0x7f4fd810e600 2024-04-20T11:30:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 Processor -- start 2024-04-20T11:30:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 -- start start 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd88205390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd8820df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd8820e4b0 0x55dd882108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dd8816a050 con 0x55dd8820e4b0 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dd88110490 con 0x55dd882058d0 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.625+0000 7f4fdf771740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dd88166010 con 0x55dd88204fb0 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd88205390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd88205390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46258/0 (socket says 172.21.15.44:46258) 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 -- 172.21.15.44:0/2649151037 learned_addr learned my addr 172.21.15.44:0/2649151037 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd8820df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd8820e4b0 0x55dd882108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 -- 172.21.15.44:0/2649151037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 msgr2=0x55dd8820df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd8820df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 -- 172.21.15.44:0/2649151037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd8820e4b0 msgr2=0x55dd882108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd8820e4b0 0x55dd882108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 -- 172.21.15.44:0/2649151037 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd88118070 con 0x55dd88204fb0 2024-04-20T11:30:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd8820df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.626+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd88205390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0011ca0 tx=0x7f4fc0009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2cc74c04ee3ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.627+0000 7f4fc7000640 1 -- 172.21.15.44:0/2649151037 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000fc50 con 0x55dd88204fb0 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.627+0000 7f4fc7000640 1 -- 172.21.15.44:0/2649151037 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4fc000fdf0 con 0x55dd88204fb0 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.627+0000 7f4fc7000640 1 -- 172.21.15.44:0/2649151037 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000e7c0 con 0x55dd88204fb0 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.627+0000 7f4fcf400640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4fbc0a3d20 0x7f4fbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fcf400640 1 -- 172.21.15.44:0/3096455526 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f4fd00d0ed0 con 0x7f4fd810e600 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4fbc0a3d20 0x7f4fbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 -- 172.21.15.44:0/2649151037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 msgr2=0x55dd88205390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd88205390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0011ca0 tx=0x7f4fc0009c60 comp rx=0 tx=0).stop 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 -- 172.21.15.44:0/2649151037 shutdown_connections 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd8820e4b0 0x55dd882108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd8820df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2649151037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd88205390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 -- 172.21.15.44:0/2649151037 >> 172.21.15.44:0/2649151037 conn(0x55dd881ff6e0 msgr2=0x55dd881ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 -- 172.21.15.44:0/2649151037 shutdown_connections 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fdf771740 1 -- 172.21.15.44:0/2649151037 wait complete. 2024-04-20T11:30:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.628+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4fbc0a3d20 0x7f4fbc0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc8005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.629+0000 7f4fdf771740 1 -- start start 2024-04-20T11:30:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.629+0000 7f4fdf771740 1 Processor -- start 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.629+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.629+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8820b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.629+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd8820bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.629+0000 7f4fdf771740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dd882112a0 con 0x55dd882058d0 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fdf771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dd88123860 con 0x55dd8820e4b0 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fdf771740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dd88211420 con 0x55dd88204fb0 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fdf771740 10 client.0 _ll_register_callbacks cb 0x55dd880f1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8820b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd8820bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46266/0 (socket says 172.21.15.44:46266) 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd5e00640 1 -- 172.21.15.44:0/3412084005 learned_addr learned my addr 172.21.15.44:0/3412084005 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 -- 172.21.15.44:0/3412084005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 msgr2=0x55dd8820b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 -- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 msgr2=0x55dd8820b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8820b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 -- 172.21.15.44:0/3412084005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd88127950 con 0x55dd8820e4b0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.630+0000 7f4fd6800640 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd8820bf10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00ce660 tx=0x7f4fd009d6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd7927d06469df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fbb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fbb400640 1 -- 172.21.15.44:0/3412084005 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00b5020 con 0x55dd8820e4b0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fbb400640 1 -- 172.21.15.44:0/3412084005 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f4fd0099520 con 0x55dd8820e4b0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc000fdf0 con 0x55dd8820e4b0 2024-04-20T11:30:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fbb400640 1 -- 172.21.15.44:0/3412084005 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4fd00b6220 con 0x55dd8820e4b0 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fdf771740 10 client.27449 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dd880f5820 con 0x55dd8820e4b0 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fbb400640 1 -- 172.21.15.44:0/3412084005 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4fc000fdf0 con 0x55dd8820e4b0 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.631+0000 7f4fbb400640 1 -- 172.21.15.44:0/3412084005 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f4fd009c390 con 0x55dd8820e4b0 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fdf771740 10 client.27449 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fdf771740 10 client.27449 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fdf771740 20 client.27449 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fdf771740 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd8821e450 0x55dd88220880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fdf771740 4 client.27449 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd88177a70 con 0x55dd8821e450 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.632+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd8821e450 0x55dd88220880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.633+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd8821e450 0x55dd88220880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2bd533e845316952 server_cookie=350699c2b497f50f in_seq=0 out_seq=0 2024-04-20T11:30:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.633+0000 7f4fbb400640 10 client.27449 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.633+0000 7f4fbb400640 1 -- 172.21.15.44:0/3412084005 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd88177a70 con 0x55dd8821e450 2024-04-20T11:30:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.633+0000 7f4fbb400640 10 client.27449 handle_command_reply: tid=0 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.633+0000 7f4fdf771740 1 client.27449 shutdown 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 msgr2=0x55dd8820bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd8820bf10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00ce660 tx=0x7f4fd009d6f0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 shutdown_connections 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd8821e450 0x55dd88220880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd8820bf10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8820b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fbb400640 0 client.27449 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 --2- 172.21.15.44:0/3412084005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 >> 172.21.15.44:0/3412084005 conn(0x55dd881ff6e0 msgr2=0x55dd88201430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 shutdown_connections 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 1 -- 172.21.15.44:0/3412084005 wait complete. 2024-04-20T11:30:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.634+0000 7f4fdf771740 20 client.27449 trim_cache size 0 max 16384 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 Processor -- start 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 -- start start 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8821a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd88201200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b41f0 con 0x55dd882058d0 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b5d30 con 0x55dd8820e4b0 2024-04-20T11:30:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.649+0000 7f4fdf771740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b48c0 con 0x55dd88204fb0 2024-04-20T11:30:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8821a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8821a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44074/0 (socket says 172.21.15.44:44074) 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd88201200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fcd400640 1 -- 172.21.15.44:0/4004754665 learned_addr learned my addr 172.21.15.44:0/4004754665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 -- 172.21.15.44:0/4004754665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 msgr2=0x55dd8820dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 -- 172.21.15.44:0/4004754665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 msgr2=0x55dd8821a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8821a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 -- 172.21.15.44:0/4004754665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd87e7c3f0 con 0x55dd8820e4b0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fcd400640 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8821a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.650+0000 7f4fd6800640 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd88201200 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00ce710 tx=0x7f4fd00a13a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f607e48aa83effaf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fc6600640 1 -- 172.21.15.44:0/4004754665 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00b5070 con 0x55dd8820e4b0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fc6600640 1 -- 172.21.15.44:0/4004754665 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4fd00055b0 con 0x55dd8820e4b0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fc6600640 1 -- 172.21.15.44:0/4004754665 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00a5690 con 0x55dd8820e4b0 2024-04-20T11:30:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 -- 172.21.15.44:0/4004754665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 msgr2=0x55dd88201200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd88201200 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00ce710 tx=0x7f4fd00a13a0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 -- 172.21.15.44:0/4004754665 shutdown_connections 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8820e4b0 0x55dd88201200 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd882058d0 0x55dd8821a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 --2- 172.21.15.44:0/4004754665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd88204fb0 0x55dd8820dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 -- 172.21.15.44:0/4004754665 >> 172.21.15.44:0/4004754665 conn(0x55dd881ff6e0 msgr2=0x55dd88200af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 -- 172.21.15.44:0/4004754665 shutdown_connections 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.651+0000 7f4fdf771740 1 -- 172.21.15.44:0/4004754665 wait complete. 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.652+0000 7f4fdf771740 1 -- start start 2024-04-20T11:30:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.652+0000 7f4fdf771740 1 Processor -- start 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.652+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 0x55dd88212500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.652+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd88212a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fdf771740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd8820e4b0 0x55dd882839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fdf771740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b5660 con 0x55dd88204fb0 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fdf771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b4f90 con 0x55dd882058d0 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fdf771740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81b21e0 con 0x55dd8820e4b0 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fdf771740 10 client.0 _ll_register_callbacks cb 0x7f4fd009c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 0x55dd88212500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 0x55dd88212500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44090/0 (socket says 172.21.15.44:44090) 2024-04-20T11:30:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 -- 172.21.15.44:0/2412503134 learned_addr learned my addr 172.21.15.44:0/2412503134 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd88212a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd6800640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd8820e4b0 0x55dd882839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 -- 172.21.15.44:0/2412503134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd8820e4b0 msgr2=0x55dd882839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd8820e4b0 0x55dd882839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 -- 172.21.15.44:0/2412503134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 msgr2=0x55dd88212a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd88212a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fd5e00640 1 -- 172.21.15.44:0/2412503134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd87f467d0 con 0x55dd88204fb0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.653+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd88212a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fd5e00640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 0x55dd88212500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4fc802c960 tx=0x7f4fc802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4745e2c95d41b8ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fbb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fbb400640 1 -- 172.21.15.44:0/2412503134 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8044040 con 0x55dd88204fb0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fbb400640 1 -- 172.21.15.44:0/2412503134 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f4fc802ca80 con 0x55dd88204fb0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd00055b0 con 0x55dd88204fb0 2024-04-20T11:30:05.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fbb400640 1 -- 172.21.15.44:0/2412503134 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4fc805f050 con 0x55dd88204fb0 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fdf771740 10 client.17331 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.654+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dd88284ef0 con 0x55dd88204fb0 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fbb400640 1 -- 172.21.15.44:0/2412503134 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4fd00055b0 con 0x55dd88204fb0 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fbb400640 1 -- 172.21.15.44:0/2412503134 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f4fc80506b0 con 0x55dd88204fb0 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fdf771740 10 client.17331 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fdf771740 10 client.17331 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fdf771740 20 client.17331 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd88285640 0x55dd88287a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fdf771740 4 client.17331 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd881718e0 con 0x55dd88285640 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.655+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd88285640 0x55dd88287a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fcd400640 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd88285640 0x55dd88287a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e69cbe935ed80e2d server_cookie=822f11ffb7d3436f in_seq=0 out_seq=0 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fbb400640 10 client.17331 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fbb400640 1 -- 172.21.15.44:0/2412503134 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd881718e0 con 0x55dd88285640 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fbb400640 10 client.17331 handle_command_reply: tid=0 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 client.17331 shutdown 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 msgr2=0x55dd88212500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 0x55dd88212500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4fc802c960 tx=0x7f4fc802bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 shutdown_connections 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd8820e4b0 0x55dd882839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55dd88285640 0x55dd88287a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd882058d0 0x55dd88212a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.656+0000 7f4fdf771740 1 --2- 172.21.15.44:0/2412503134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dd88204fb0 0x55dd88212500 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fbb400640 0 client.17331 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 >> 172.21.15.44:0/2412503134 conn(0x55dd881ff6e0 msgr2=0x55dd881fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 shutdown_connections 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdf771740 1 -- 172.21.15.44:0/2412503134 wait complete. 2024-04-20T11:30:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdf771740 20 client.17331 trim_cache size 0 max 16384 2024-04-20T11:30:05.662 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4fbc0a3d20 msgr2=0x7f4fbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdca00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4fbc0a3d20 0x7f4fbc0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc8005e40 comp rx=0 tx=0).stop 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd810e600 msgr2=0x7f4fd81a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.657+0000 7f4fdca00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd810e600 0x7f4fd81a36f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009fd0 tx=0x7f4fd000ef10 comp rx=0 tx=0).stop 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 shutdown_connections 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd810e600 0x7f4fd81a36f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4fbc0a3d20 0x7f4fbc0a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fd8108f50 0x7f4fd819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 --2- 172.21.15.44:0/3096455526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fd8108590 0x7f4fd819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 >> 172.21.15.44:0/3096455526 conn(0x7f4fd80764a0 msgr2=0x7f4fd8077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 shutdown_connections 2024-04-20T11:30:05.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.658+0000 7f4fdca00640 1 -- 172.21.15.44:0/3096455526 wait complete. 2024-04-20T11:30:05.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:05 smithi044 ceph-mon[16855]: pgmap v229: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:30:05.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:05 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2352263219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:30:05.679 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 -- 172.21.15.44:0/2013167725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff348104280 msgr2=0x7ff348104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 --2- 172.21.15.44:0/2013167725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff348104280 0x7ff348104660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff338009900 tx=0x7ff33802bc50 comp rx=0 tx=0).stop 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 -- 172.21.15.44:0/2013167725 shutdown_connections 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 --2- 172.21.15.44:0/2013167725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 0x7ff348110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 --2- 172.21.15.44:0/2013167725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348104c30 0x7ff348108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 --2- 172.21.15.44:0/2013167725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff348104280 0x7ff348104660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 -- 172.21.15.44:0/2013167725 >> 172.21.15.44:0/2013167725 conn(0x7ff3480ffaa0 msgr2=0x7ff348101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 -- 172.21.15.44:0/2013167725 shutdown_connections 2024-04-20T11:30:05.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.888+0000 7ff34ce00640 1 -- 172.21.15.44:0/2013167725 wait complete. 2024-04-20T11:30:05.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.889+0000 7ff34ce00640 1 Processor -- start 2024-04-20T11:30:05.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.889+0000 7ff34ce00640 1 -- start start 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff34ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348104280 0x7ff3481a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff34ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 0x7ff3481a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff34ce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 0x7ff34819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff34ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff348114520 con 0x7ff348104280 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff34ce00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481143a0 con 0x7ff34819a820 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff34ce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481146a0 con 0x7ff348109350 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff346a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 0x7ff3481a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff347e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 0x7ff34819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff347400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348104280 0x7ff3481a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff347e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 0x7ff34819acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41856/0 (socket says 172.21.15.44:41856) 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff346a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 0x7ff3481a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46296/0 (socket says 172.21.15.44:46296) 2024-04-20T11:30:05.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.890+0000 7ff347e00640 1 -- 172.21.15.44:0/4230663918 learned_addr learned my addr 172.21.15.44:0/4230663918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff347e00640 1 -- 172.21.15.44:0/4230663918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 msgr2=0x7ff3481a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff347e00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 0x7ff3481a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff347e00640 1 -- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348104280 msgr2=0x7ff3481a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff347e00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348104280 0x7ff3481a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff347e00640 1 -- 172.21.15.44:0/4230663918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34819b3b0 con 0x7ff34819a820 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff347e00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 0x7ff34819acd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff340009fd0 tx=0x7ff34000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff33fe00640 1 -- 172.21.15.44:0/4230663918 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340017070 con 0x7ff34819a820 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff33fe00640 1 -- 172.21.15.44:0/4230663918 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff340004020 con 0x7ff34819a820 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff33fe00640 1 -- 172.21.15.44:0/4230663918 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400106c0 con 0x7ff34819a820 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.891+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34819b6a0 con 0x7ff34819a820 2024-04-20T11:30:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.892+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a7830 con 0x7ff34819a820 2024-04-20T11:30:05.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.895+0000 7ff3511c4740 1 Processor -- start 2024-04-20T11:30:05.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.895+0000 7ff33fe00640 1 -- 172.21.15.44:0/4230663918 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff3400041d0 con 0x7ff34819a820 2024-04-20T11:30:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 -- start start 2024-04-20T11:30:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 0x55c5aae25130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae27a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5aad81050 con 0x55c5aae1c170 2024-04-20T11:30:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5aad27490 con 0x55c5aae25670 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff3511c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5aad7d010 con 0x55c5aae1ca90 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff346a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff347400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 0x55c5aae25130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff347400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 0x55c5aae25130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46304/0 (socket says 172.21.15.44:46304) 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.896+0000 7ff347400640 1 -- 172.21.15.44:0/482422667 learned_addr learned my addr 172.21.15.44:0/482422667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff346a00640 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae27a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff347400640 1 -- 172.21.15.44:0/482422667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 msgr2=0x55c5aae27a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff347400640 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae27a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff347400640 1 -- 172.21.15.44:0/482422667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 msgr2=0x55c5aae1c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff347400640 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae1c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff347400640 1 -- 172.21.15.44:0/482422667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5aad2f070 con 0x55c5aae1ca90 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff347400640 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 0x55c5aae25130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff338009900 tx=0x7ff33802c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80f34c93c6bccdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.897+0000 7ff33fe00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff32c0a3da0 0x7ff32c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff33cc00640 1 -- 172.21.15.44:0/482422667 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33802c820 con 0x55c5aae1ca90 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff33cc00640 1 -- 172.21.15.44:0/482422667 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff338004040 con 0x55c5aae1ca90 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff33cc00640 1 -- 172.21.15.44:0/482422667 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3380046c0 con 0x55c5aae1ca90 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff33fe00640 1 -- 172.21.15.44:0/4230663918 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7ff3400d03c0 con 0x7ff34819a820 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff346a00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff32c0a3da0 0x7ff32c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 -- 172.21.15.44:0/482422667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 msgr2=0x55c5aae25130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 0x55c5aae25130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff338009900 tx=0x7ff33802c630 comp rx=0 tx=0).stop 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 -- 172.21.15.44:0/482422667 shutdown_connections 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff346a00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff32c0a3da0 0x7ff32c0a6260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55c5aae17a30 tx=0x7ff334005850 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae27a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1ca90 0x55c5aae25130 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 --2- 172.21.15.44:0/482422667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae1c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.898+0000 7ff3511c4740 1 -- 172.21.15.44:0/482422667 >> 172.21.15.44:0/482422667 conn(0x55c5aae168a0 msgr2=0x55c5aae16c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.899+0000 7ff3511c4740 1 -- 172.21.15.44:0/482422667 shutdown_connections 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.899+0000 7ff3511c4740 1 -- 172.21.15.44:0/482422667 wait complete. 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.899+0000 7ff3511c4740 1 -- start start 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.899+0000 7ff3511c4740 1 Processor -- start 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae22770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae22cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5aae28400 con 0x55c5aae1ca90 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5aad3a860 con 0x55c5aae25670 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5aae28580 con 0x55c5aae1c170 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff3511c4740 10 client.0 _ll_register_callbacks cb 0x55c5aad08fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff347e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff346a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae22cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff347e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41880/0 (socket says 172.21.15.44:41880) 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff347e00640 1 -- 172.21.15.44:0/3373895016 learned_addr learned my addr 172.21.15.44:0/3373895016 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.900+0000 7ff347400640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae22770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff347e00640 1 -- 172.21.15.44:0/3373895016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 msgr2=0x55c5aae22770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff347e00640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae22770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff347e00640 1 -- 172.21.15.44:0/3373895016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 msgr2=0x55c5aae22cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff347e00640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae22cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff347e00640 1 -- 172.21.15.44:0/3373895016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5aad3e950 con 0x55c5aae25670 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff346a00640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff347e00640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae231f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff340098dc0 tx=0x7ff340005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1bbcf235b4b7c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff331600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff331600640 1 -- 172.21.15.44:0/3373895016 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340020070 con 0x55c5aae25670 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff331600640 1 -- 172.21.15.44:0/3373895016 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7ff340098a60 con 0x55c5aae25670 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff338004040 con 0x55c5aae25670 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff331600640 1 -- 172.21.15.44:0/3373895016 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff3400a4fb0 con 0x55c5aae25670 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff3511c4740 10 client.27467 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.901+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c5aad0c820 con 0x55c5aae25670 2024-04-20T11:30:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff331600640 1 -- 172.21.15.44:0/3373895016 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff338004040 con 0x55c5aae25670 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff331600640 1 -- 172.21.15.44:0/3373895016 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7ff3400982d0 con 0x55c5aae25670 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff3511c4740 10 client.27467 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff3511c4740 10 client.27467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff3511c4740 20 client.27467 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff3511c4740 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae35760 0x55c5aae37b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff3511c4740 4 client.27467 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.902+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5aad8ea70 con 0x55c5aae35760 2024-04-20T11:30:05.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.903+0000 7ff347400640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae35760 0x55c5aae37b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.903+0000 7ff347400640 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae35760 0x55c5aae37b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9656faec778df39e server_cookie=508111bdd164b419 in_seq=0 out_seq=0 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.903+0000 7ff331600640 10 client.27467 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff331600640 1 -- 172.21.15.44:0/3373895016 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5aad8ea70 con 0x55c5aae35760 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff331600640 10 client.27467 handle_command_reply: tid=0 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 client.27467 shutdown 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 msgr2=0x55c5aae231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae231f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff340098dc0 tx=0x7ff340005b90 comp rx=0 tx=0).stop 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 shutdown_connections 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae231f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae35760 0x55c5aae37b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 --2- 172.21.15.44:0/3373895016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae22770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff331600640 0 client.27467 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 >> 172.21.15.44:0/3373895016 conn(0x55c5aae168a0 msgr2=0x55c5aae185f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 shutdown_connections 2024-04-20T11:30:05.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.904+0000 7ff3511c4740 1 -- 172.21.15.44:0/3373895016 wait complete. 2024-04-20T11:30:05.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.905+0000 7ff3511c4740 20 client.27467 trim_cache size 0 max 16384 2024-04-20T11:30:05.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.919+0000 7ff3511c4740 1 Processor -- start 2024-04-20T11:30:05.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.919+0000 7ff3511c4740 1 -- start start 2024-04-20T11:30:05.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae24ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae1ca90 0x55c5aae312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae25670 0x55c5aae18390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff3511c4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481afcf0 con 0x55c5aae1c170 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff3511c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b1830 con 0x55c5aae1ca90 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff3511c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b03c0 con 0x55c5aae25670 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae24ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae24ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44140/0 (socket says 172.21.15.44:44140) 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 -- 172.21.15.44:0/130660559 learned_addr learned my addr 172.21.15.44:0/130660559 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff346a00640 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae1ca90 0x55c5aae312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347e00640 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae25670 0x55c5aae18390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 -- 172.21.15.44:0/130660559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae25670 msgr2=0x55c5aae18390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae25670 0x55c5aae18390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 -- 172.21.15.44:0/130660559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae1ca90 msgr2=0x55c5aae312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae1ca90 0x55c5aae312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.920+0000 7ff347400640 1 -- 172.21.15.44:0/130660559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5aae27dd0 con 0x55c5aae1c170 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.921+0000 7ff347400640 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae24ea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff3380053b0 tx=0x7ff3380053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882f7f6b2bf0ed2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.921+0000 7ff333e00640 1 -- 172.21.15.44:0/130660559 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33802cbf0 con 0x55c5aae1c170 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.921+0000 7ff333e00640 1 -- 172.21.15.44:0/130660559 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff33802cd90 con 0x55c5aae1c170 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.921+0000 7ff333e00640 1 -- 172.21.15.44:0/130660559 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33803f690 con 0x55c5aae1c170 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 -- 172.21.15.44:0/130660559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 msgr2=0x55c5aae24ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae24ea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff3380053b0 tx=0x7ff3380053e0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 -- 172.21.15.44:0/130660559 shutdown_connections 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae25670 0x55c5aae18390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae1ca90 0x55c5aae312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 --2- 172.21.15.44:0/130660559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1c170 0x55c5aae24ea0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 -- 172.21.15.44:0/130660559 >> 172.21.15.44:0/130660559 conn(0x55c5aae168a0 msgr2=0x55c5aae17c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 -- 172.21.15.44:0/130660559 shutdown_connections 2024-04-20T11:30:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 -- 172.21.15.44:0/130660559 wait complete. 2024-04-20T11:30:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 -- start start 2024-04-20T11:30:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.922+0000 7ff3511c4740 1 Processor -- start 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae29930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae29e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b1160 con 0x55c5aae1ca90 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481b0a90 con 0x55c5aae25670 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3481adce0 con 0x55c5aae1c170 2024-04-20T11:30:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff3511c4740 10 client.0 _ll_register_callbacks cb 0x7ff34009bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff346a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae29930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff346a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae29930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44148/0 (socket says 172.21.15.44:44148) 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff346a00640 1 -- 172.21.15.44:0/2129499052 learned_addr learned my addr 172.21.15.44:0/2129499052 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff347400640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.923+0000 7ff347e00640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae29e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff346a00640 1 -- 172.21.15.44:0/2129499052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 msgr2=0x55c5aae293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff346a00640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae293f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff346a00640 1 -- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 msgr2=0x55c5aae29e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff346a00640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae29e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff346a00640 1 -- 172.21.15.44:0/2129499052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5aaa933f0 con 0x55c5aae1ca90 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff347400640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff346a00640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae29930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff34810a900 tx=0x7ff334005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a401e3755ff3543d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff331600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff331600640 1 -- 172.21.15.44:0/2129499052 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334031060 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33802cd90 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.924+0000 7ff331600640 1 -- 172.21.15.44:0/2129499052 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7ff3340109c0 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff331600640 1 -- 172.21.15.44:0/2129499052 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff334010bc0 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 10 client.17349 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c5aab5d7d0 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff331600640 1 -- 172.21.15.44:0/2129499052 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff33802cd90 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff331600640 1 -- 172.21.15.44:0/2129499052 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7ff334022410 con 0x55c5aae1ca90 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 10 client.17349 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 10 client.17349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 20 client.17349 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae9c510 0x55c5aae9e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 4 client.17349 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.925+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5aad888e0 con 0x55c5aae9c510 2024-04-20T11:30:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.926+0000 7ff347400640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae9c510 0x55c5aae9e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.926+0000 7ff347400640 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae9c510 0x55c5aae9e930 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83e76b590cd5e592 server_cookie=e2695e46ddb70e41 in_seq=0 out_seq=0 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.926+0000 7ff331600640 10 client.17349 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.926+0000 7ff331600640 1 -- 172.21.15.44:0/2129499052 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5aad888e0 con 0x55c5aae9c510 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.926+0000 7ff331600640 10 client.17349 handle_command_reply: tid=0 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.926+0000 7ff3511c4740 1 client.17349 shutdown 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 msgr2=0x55c5aae29930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae29930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff34810a900 tx=0x7ff334005f90 comp rx=0 tx=0).stop 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 shutdown_connections 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5aae25670 0x55c5aae29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c5aae9c510 0x55c5aae9e930 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5aae1ca90 0x55c5aae29930 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 --2- 172.21.15.44:0/2129499052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5aae1c170 0x55c5aae293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff331600640 0 client.17349 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 >> 172.21.15.44:0/2129499052 conn(0x55c5aae168a0 msgr2=0x55c5aae184f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 shutdown_connections 2024-04-20T11:30:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 1 -- 172.21.15.44:0/2129499052 wait complete. 2024-04-20T11:30:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.927+0000 7ff3511c4740 20 client.17349 trim_cache size 0 max 16384 2024-04-20T11:30:05.931 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff32c0a3da0 msgr2=0x7ff32c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff32c0a3da0 0x7ff32c0a6260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55c5aae17a30 tx=0x7ff334005850 comp rx=0 tx=0).stop 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 msgr2=0x7ff34819acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 0x7ff34819acd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff340009fd0 tx=0x7ff34000b260 comp rx=0 tx=0).stop 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 shutdown_connections 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff34819a820 0x7ff34819acd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff348109350 0x7ff3481a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff32c0a3da0 0x7ff32c0a6260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 --2- 172.21.15.44:0/4230663918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff348104280 0x7ff3481a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 >> 172.21.15.44:0/4230663918 conn(0x7ff3480ffaa0 msgr2=0x7ff348101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 shutdown_connections 2024-04-20T11:30:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:05.928+0000 7ff34ce00640 1 -- 172.21.15.44:0/4230663918 wait complete. 2024-04-20T11:30:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:05 smithi078 ceph-mon[21464]: pgmap v229: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:30:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:05 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2352263219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:30:05.946 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:30:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:05 smithi165 ceph-mon[21975]: pgmap v229: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:30:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:05 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2352263219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:30:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 -- 172.21.15.44:0/1826869210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 msgr2=0x7f7148102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 --2- 172.21.15.44:0/1826869210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 0x7f7148102450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7138009be0 tx=0x7f713802bef0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 -- 172.21.15.44:0/1826869210 shutdown_connections 2024-04-20T11:30:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 --2- 172.21.15.44:0/1826869210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f714810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 --2- 172.21.15.44:0/1826869210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7148102a20 0x7f7148106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 --2- 172.21.15.44:0/1826869210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 0x7f7148102450 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 -- 172.21.15.44:0/1826869210 >> 172.21.15.44:0/1826869210 conn(0x7f71480fd930 msgr2=0x7f71480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.169+0000 7f714ca00640 1 -- 172.21.15.44:0/1826869210 shutdown_connections 2024-04-20T11:30:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.170+0000 7f714ca00640 1 -- 172.21.15.44:0/1826869210 wait complete. 2024-04-20T11:30:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 Processor -- start 2024-04-20T11:30:06.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 -- start start 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 0x7f71481a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f71481a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714819a810 0x7f714819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7148112330 con 0x7f7148102070 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71481121b0 con 0x7f7148107140 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.171+0000 7f714ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71481124b0 con 0x7f714819a810 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f71481a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f7146800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 0x7f71481a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f7147200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714819a810 0x7f714819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f71481a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41912/0 (socket says 172.21.15.44:41912) 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 -- 172.21.15.44:0/3262081726 learned_addr learned my addr 172.21.15.44:0/3262081726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 -- 172.21.15.44:0/3262081726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714819a810 msgr2=0x7f714819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714819a810 0x7f714819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 -- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 msgr2=0x7f71481a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 0x7f71481a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 -- 172.21.15.44:0/3262081726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714819b3a0 con 0x7f7148107140 2024-04-20T11:30:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.172+0000 7f713de00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f71481a0be0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f713000e9f0 tx=0x7f713000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.173+0000 7f713fe00640 1 -- 172.21.15.44:0/3262081726 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713000ccc0 con 0x7f7148107140 2024-04-20T11:30:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.173+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714819b690 con 0x7f7148107140 2024-04-20T11:30:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.173+0000 7f713fe00640 1 -- 172.21.15.44:0/3262081726 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7130005520 con 0x7f7148107140 2024-04-20T11:30:06.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.173+0000 7f713fe00640 1 -- 172.21.15.44:0/3262081726 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130010690 con 0x7f7148107140 2024-04-20T11:30:06.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.173+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481a7870 con 0x7f7148107140 2024-04-20T11:30:06.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.176+0000 7f7150038740 1 Processor -- start 2024-04-20T11:30:06.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.176+0000 7f7150038740 1 -- start start 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7150038740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 0x55c6d6670550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7150038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670a90 0x55c6d6679130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7150038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d667ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7150038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c6d65d5050 con 0x55c6d6670a90 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7150038740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6d657b490 con 0x55c6d6670170 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7150038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6d65d1010 con 0x55c6d6679670 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7146800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670a90 0x55c6d6679130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 0x55c6d6670550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f713fe00640 1 -- 172.21.15.44:0/3262081726 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7130004f70 con 0x7f7148107140 2024-04-20T11:30:06.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 0x55c6d6670550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41928/0 (socket says 172.21.15.44:41928) 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 -- 172.21.15.44:0/2084487846 learned_addr learned my addr 172.21.15.44:0/2084487846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 -- 172.21.15.44:0/2084487846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 msgr2=0x55c6d667ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d667ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 -- 172.21.15.44:0/2084487846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670a90 msgr2=0x55c6d6679130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.177+0000 7f7147200640 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670a90 0x55c6d6679130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.178+0000 7f7147200640 1 -- 172.21.15.44:0/2084487846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d6583070 con 0x55c6d6670170 2024-04-20T11:30:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.178+0000 7f7146800640 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670a90 0x55c6d6679130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.178+0000 7f7147200640 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 0x55c6d6670550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7140009be0 tx=0x7f7140011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92c9c571ae92567 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.178+0000 7f7137400640 1 -- 172.21.15.44:0/2084487846 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140004230 con 0x55c6d6670170 2024-04-20T11:30:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.178+0000 7f7137400640 1 -- 172.21.15.44:0/2084487846 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7140009e50 con 0x55c6d6670170 2024-04-20T11:30:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.178+0000 7f7137400640 1 -- 172.21.15.44:0/2084487846 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140019890 con 0x55c6d6670170 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 -- 172.21.15.44:0/2084487846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 msgr2=0x55c6d6670550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 0x55c6d6670550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7140009be0 tx=0x7f7140011810 comp rx=0 tx=0).stop 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f713fe00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f712c0a3da0 0x7f712c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 -- 172.21.15.44:0/2084487846 shutdown_connections 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d667ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670a90 0x55c6d6679130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 --2- 172.21.15.44:0/2084487846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670170 0x55c6d6670550 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 -- 172.21.15.44:0/2084487846 >> 172.21.15.44:0/2084487846 conn(0x55c6d666a8a0 msgr2=0x55c6d666ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f713fe00640 1 -- 172.21.15.44:0/3262081726 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f71300d0ae0 con 0x7f7148107140 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 -- 172.21.15.44:0/2084487846 shutdown_connections 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7146800640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f712c0a3da0 0x7f712c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.179+0000 7f7150038740 1 -- 172.21.15.44:0/2084487846 wait complete. 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 -- start start 2024-04-20T11:30:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 Processor -- start 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7146800640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f712c0a3da0 0x7f712c0a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7138006010 tx=0x7f7138009300 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d66766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6676c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6677170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c6d667c460 con 0x55c6d6679670 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.180+0000 7f7150038740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6d658e860 con 0x55c6d6670a90 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f7150038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6d667c5e0 con 0x55c6d6670170 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f7150038740 10 client.0 _ll_register_callbacks cb 0x55c6d655cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6676c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f7146800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d66766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f7147200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6677170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6676c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41932/0 (socket says 172.21.15.44:41932) 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 -- 172.21.15.44:0/4197963047 learned_addr learned my addr 172.21.15.44:0/4197963047 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 -- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 msgr2=0x55c6d66766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d66766f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 -- 172.21.15.44:0/4197963047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 msgr2=0x55c6d6677170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6677170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 -- 172.21.15.44:0/4197963047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d6592950 con 0x55c6d6670a90 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.181+0000 7f713de00640 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6676c30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f713009c760 tx=0x7f713009c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75254db2f38ccf26 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f712be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f712be00640 1 -- 172.21.15.44:0/4197963047 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71300bd0d0 con 0x55c6d6670a90 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f712be00640 1 -- 172.21.15.44:0/4197963047 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f71300ce490 con 0x55c6d6670a90 2024-04-20T11:30:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7140009e50 con 0x55c6d6670a90 2024-04-20T11:30:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f712be00640 1 -- 172.21.15.44:0/4197963047 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f71300ce690 con 0x55c6d6670a90 2024-04-20T11:30:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f7150038740 10 client.27485 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c6d6560820 con 0x55c6d6670a90 2024-04-20T11:30:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f712be00640 1 -- 172.21.15.44:0/4197963047 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7140009e50 con 0x55c6d6670a90 2024-04-20T11:30:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f712be00640 1 -- 172.21.15.44:0/4197963047 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f713009beb0 con 0x55c6d6670a90 2024-04-20T11:30:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f7150038740 10 client.27485 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.182+0000 7f7150038740 10 client.27485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.183+0000 7f7150038740 20 client.27485 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.183+0000 7f7150038740 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d6689660 0x55c6d668ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.183+0000 7f7150038740 4 client.27485 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.183+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d65e2a70 con 0x55c6d6689660 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.183+0000 7f7146800640 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d6689660 0x55c6d668ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.184+0000 7f7146800640 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d6689660 0x55c6d668ba80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75ec11f1e36dc728 server_cookie=cf4cdf2042694460 in_seq=0 out_seq=0 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.184+0000 7f712be00640 10 client.27485 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f712be00640 1 -- 172.21.15.44:0/4197963047 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6d65e2a70 con 0x55c6d6689660 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f712be00640 10 client.27485 handle_command_reply: tid=0 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 client.27485 shutdown 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 msgr2=0x55c6d6676c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6676c30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f713009c760 tx=0x7f713009c480 comp rx=0 tx=0).stop 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 shutdown_connections 2024-04-20T11:30:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d6689660 0x55c6d668ba80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6677170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6676c30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 --2- 172.21.15.44:0/4197963047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d66766f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 >> 172.21.15.44:0/4197963047 conn(0x55c6d666a8a0 msgr2=0x55c6d6669960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f712be00640 0 client.27485 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 shutdown_connections 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 1 -- 172.21.15.44:0/4197963047 wait complete. 2024-04-20T11:30:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.185+0000 7f7150038740 20 client.27485 trim_cache size 0 max 16384 2024-04-20T11:30:06.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 Processor -- start 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 -- start start 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d6678e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6685320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6691350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71481afe90 con 0x55c6d6679670 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b19d0 con 0x55c6d6670a90 2024-04-20T11:30:06.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.201+0000 7f7150038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b0560 con 0x55c6d6670170 2024-04-20T11:30:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d6678e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d6678e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46390/0 (socket says 172.21.15.44:46390) 2024-04-20T11:30:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 -- 172.21.15.44:0/2828351373 learned_addr learned my addr 172.21.15.44:0/2828351373 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7147200640 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6691350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f713de00640 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6685320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 -- 172.21.15.44:0/2828351373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 msgr2=0x55c6d6685320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6685320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 -- 172.21.15.44:0/2828351373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 msgr2=0x55c6d6691350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6691350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.202+0000 7f7146800640 1 -- 172.21.15.44:0/2828351373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d62e73f0 con 0x55c6d6670170 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.203+0000 7f7146800640 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d6678e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7138009370 tx=0x7f7138044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b45308f69da27aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.203+0000 7f7136a00640 1 -- 172.21.15.44:0/2828351373 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138033750 con 0x55c6d6670170 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.203+0000 7f7136a00640 1 -- 172.21.15.44:0/2828351373 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f713805b070 con 0x55c6d6670170 2024-04-20T11:30:06.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.203+0000 7f7136a00640 1 -- 172.21.15.44:0/2828351373 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138055830 con 0x55c6d6670170 2024-04-20T11:30:06.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.203+0000 7f7150038740 1 -- 172.21.15.44:0/2828351373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 msgr2=0x55c6d6678e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.203+0000 7f7150038740 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d6678e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7138009370 tx=0x7f7138044000 comp rx=0 tx=0).stop 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 -- 172.21.15.44:0/2828351373 shutdown_connections 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6679670 0x55c6d6691350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6685320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 --2- 172.21.15.44:0/2828351373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6670170 0x55c6d6678e20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 -- 172.21.15.44:0/2828351373 >> 172.21.15.44:0/2828351373 conn(0x55c6d666a8a0 msgr2=0x55c6d666c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 -- 172.21.15.44:0/2828351373 shutdown_connections 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 -- 172.21.15.44:0/2828351373 wait complete. 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 -- start start 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.204+0000 7f7150038740 1 Processor -- start 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670170 0x55c6d66735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6673b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d6674060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b1300 con 0x55c6d6670170 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b0c30 con 0x55c6d6670a90 2024-04-20T11:30:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71481ade80 con 0x55c6d6679670 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7150038740 10 client.0 _ll_register_callbacks cb 0x7f713009c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7146800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670170 0x55c6d66735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6673b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6673b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41956/0 (socket says 172.21.15.44:41956) 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 -- 172.21.15.44:0/305693814 learned_addr learned my addr 172.21.15.44:0/305693814 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f7147200640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d6674060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 -- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 msgr2=0x55c6d6674060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d6674060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 -- 172.21.15.44:0/305693814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670170 msgr2=0x55c6d66735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.205+0000 7f713de00640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670170 0x55c6d66735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f713de00640 1 -- 172.21.15.44:0/305693814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d63b17d0 con 0x55c6d6670a90 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f7146800640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670170 0x55c6d66735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f713de00640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6673b20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f713009c9f0 tx=0x7f713009c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4fdd8fed791718 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f712be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f712be00640 1 -- 172.21.15.44:0/305693814 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130087110 con 0x55c6d6670a90 2024-04-20T11:30:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f712be00640 1 -- 172.21.15.44:0/305693814 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f71300ce490 con 0x55c6d6670a90 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713805b070 con 0x55c6d6670a90 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f712be00640 1 -- 172.21.15.44:0/305693814 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f71300bd0d0 con 0x55c6d6670a90 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f7150038740 10 client.27491 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.206+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c6d66f8710 con 0x55c6d6670a90 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f712be00640 1 -- 172.21.15.44:0/305693814 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f713805b070 con 0x55c6d6670a90 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f712be00640 1 -- 172.21.15.44:0/305693814 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f713009b920 con 0x55c6d6670a90 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7150038740 10 client.27491 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7150038740 10 client.27491 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7150038740 20 client.27491 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7150038740 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d66f8dd0 0x55c6d66fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7150038740 4 client.27491 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6d65dc8e0 con 0x55c6d66f8dd0 2024-04-20T11:30:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.207+0000 7f7146800640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d66f8dd0 0x55c6d66fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7146800640 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d66f8dd0 0x55c6d66fb1c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=915a2b73284f8de server_cookie=1e2c218ecdee40f0 in_seq=0 out_seq=0 2024-04-20T11:30:06.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f712be00640 10 client.27491 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f712be00640 1 -- 172.21.15.44:0/305693814 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6d65dc8e0 con 0x55c6d66f8dd0 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f712be00640 10 client.27491 handle_command_reply: tid=0 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 client.27491 shutdown 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 msgr2=0x55c6d6673b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6673b20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f713009c9f0 tx=0x7f713009c5a0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 shutdown_connections 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c6d66f8dd0 0x55c6d66fb1c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6d6679670 0x55c6d6674060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6d6670a90 0x55c6d6673b20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 --2- 172.21.15.44:0/305693814 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6d6670170 0x55c6d66735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 >> 172.21.15.44:0/305693814 conn(0x55c6d666a8a0 msgr2=0x55c6d6676690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f712be00640 0 client.27491 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 shutdown_connections 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 1 -- 172.21.15.44:0/305693814 wait complete. 2024-04-20T11:30:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.208+0000 7f7150038740 20 client.27491 trim_cache size 0 max 16384 2024-04-20T11:30:06.213 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f712c0a3da0 msgr2=0x7f712c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f712c0a3da0 0x7f712c0a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7138006010 tx=0x7f7138009300 comp rx=0 tx=0).stop 2024-04-20T11:30:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 msgr2=0x7f71481a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f71481a0be0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f713000e9f0 tx=0x7f713000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 shutdown_connections 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f714819a810 0x7f714819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7148107140 0x7f71481a0be0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f712c0a3da0 0x7f712c0a6260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 --2- 172.21.15.44:0/3262081726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7148102070 0x7f71481a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 >> 172.21.15.44:0/3262081726 conn(0x7f71480fd930 msgr2=0x7f71480ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 shutdown_connections 2024-04-20T11:30:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.209+0000 7f714ca00640 1 -- 172.21.15.44:0/3262081726 wait complete. 2024-04-20T11:30:06.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:30:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 -- 172.21.15.44:0/1533694811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168102070 msgr2=0x7f4168102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 --2- 172.21.15.44:0/1533694811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168102070 0x7f4168102450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4158009be0 tx=0x7f415802bef0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 -- 172.21.15.44:0/1533694811 shutdown_connections 2024-04-20T11:30:06.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 --2- 172.21.15.44:0/1533694811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168107140 0x7f416810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 --2- 172.21.15.44:0/1533694811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168102a20 0x7f4168106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 --2- 172.21.15.44:0/1533694811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168102070 0x7f4168102450 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 -- 172.21.15.44:0/1533694811 >> 172.21.15.44:0/1533694811 conn(0x7f41680fd930 msgr2=0x7f41680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 -- 172.21.15.44:0/1533694811 shutdown_connections 2024-04-20T11:30:06.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.443+0000 7f416d200640 1 -- 172.21.15.44:0/1533694811 wait complete. 2024-04-20T11:30:06.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.444+0000 7f416d200640 1 Processor -- start 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 -- start start 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168102070 0x7f416819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168107140 0x7f416819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 0x7f4168198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4168112330 con 0x7f4168198650 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41681121b0 con 0x7f4168102070 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f416d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41681124b0 con 0x7f4168107140 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f4167e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 0x7f4168198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f4167e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 0x7f4168198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44234/0 (socket says 172.21.15.44:44234) 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f415ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168107140 0x7f416819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f4167e00640 1 -- 172.21.15.44:0/3227339468 learned_addr learned my addr 172.21.15.44:0/3227339468 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.445+0000 7f4167400640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168102070 0x7f416819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167400640 1 -- 172.21.15.44:0/3227339468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168107140 msgr2=0x7f416819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167400640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168107140 0x7f416819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167400640 1 -- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 msgr2=0x7f4168198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167400640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 0x7f4168198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167400640 1 -- 172.21.15.44:0/3227339468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4168199170 con 0x7f4168102070 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f415ea00640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168107140 0x7f416819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167e00640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 0x7f4168198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4167400640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168102070 0x7f416819e450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4158000f80 tx=0x7f4158033720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4164c00640 1 -- 172.21.15.44:0/3227339468 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158043020 con 0x7f4168102070 2024-04-20T11:30:06.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4168199400 con 0x7f4168102070 2024-04-20T11:30:06.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4164c00640 1 -- 172.21.15.44:0/3227339468 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4158033910 con 0x7f4168102070 2024-04-20T11:30:06.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.446+0000 7f4164c00640 1 -- 172.21.15.44:0/3227339468 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415803e6b0 con 0x7f4168102070 2024-04-20T11:30:06.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.447+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681a5600 con 0x7f4168102070 2024-04-20T11:30:06.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 Processor -- start 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 -- start start 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f4164c00640 1 -- 172.21.15.44:0/3227339468 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4158046020 con 0x7f4168102070 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d685bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6869540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638d67c1050 con 0x5638d685bb50 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5638d6767490 con 0x5638d685c500 2024-04-20T11:30:06.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.450+0000 7f417142f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638d67bd010 con 0x5638d6869a80 2024-04-20T11:30:06.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f4167e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d685bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f4167e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d685bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44244/0 (socket says 172.21.15.44:44244) 2024-04-20T11:30:06.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f4167e00640 1 -- 172.21.15.44:0/3446578146 learned_addr learned my addr 172.21.15.44:0/3446578146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f4167400640 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6869540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 -- 172.21.15.44:0/3446578146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 msgr2=0x5638d6869540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6869540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 -- 172.21.15.44:0/3446578146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 msgr2=0x5638d685bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d685bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 -- 172.21.15.44:0/3446578146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638d676f070 con 0x5638d6869a80 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f4167e00640 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d685bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.451+0000 7f415ea00640 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686be70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f41680fdd30 tx=0x7f41500118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfdc140db40de6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f415cc00640 1 -- 172.21.15.44:0/3446578146 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150004030 con 0x5638d6869a80 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f415cc00640 1 -- 172.21.15.44:0/3446578146 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41500041d0 con 0x5638d6869a80 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f4164c00640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f414c0a3d20 0x7f414c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f415cc00640 1 -- 172.21.15.44:0/3446578146 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415000f8d0 con 0x5638d6869a80 2024-04-20T11:30:06.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f4164c00640 1 -- 172.21.15.44:0/3227339468 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f41580f2cd0 con 0x7f4168102070 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f4167e00640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f414c0a3d20 0x7f414c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f417142f740 1 -- 172.21.15.44:0/3446578146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 msgr2=0x5638d686be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.452+0000 7f417142f740 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686be70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f41680fdd30 tx=0x7f41500118d0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 -- 172.21.15.44:0/3446578146 shutdown_connections 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686be70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6869540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 --2- 172.21.15.44:0/3446578146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d685bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 -- 172.21.15.44:0/3446578146 >> 172.21.15.44:0/3446578146 conn(0x5638d68568a0 msgr2=0x5638d6856c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 -- 172.21.15.44:0/3446578146 shutdown_connections 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 -- 172.21.15.44:0/3446578146 wait complete. 2024-04-20T11:30:06.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f4167e00640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f414c0a3d20 0x7f414c0a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4160012880 tx=0x7f4160005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.453+0000 7f417142f740 1 -- start start 2024-04-20T11:30:06.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 Processor -- start 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d685bb50 0x5638d6866ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6867000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d6869a80 0x5638d6867540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638d686c840 con 0x5638d6869a80 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5638d677a860 con 0x5638d685c500 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638d686c9c0 con 0x5638d685bb50 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f417142f740 10 client.0 _ll_register_callbacks cb 0x5638d6748fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f4167e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d6869a80 0x5638d6867540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f415ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6867000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.454+0000 7f4167400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d685bb50 0x5638d6866ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6867000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41988/0 (socket says 172.21.15.44:41988) 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 -- 172.21.15.44:0/2602480265 learned_addr learned my addr 172.21.15.44:0/2602480265 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f4167e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d6869a80 0x5638d6867540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44256/0 (socket says 172.21.15.44:44256) 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 -- 172.21.15.44:0/2602480265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d685bb50 msgr2=0x5638d6866ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d685bb50 0x5638d6866ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 -- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d6869a80 msgr2=0x5638d6867540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d6869a80 0x5638d6867540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 -- 172.21.15.44:0/2602480265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638d677e950 con 0x5638d685c500 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f4167400640 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d685bb50 0x5638d6866ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f415ea00640 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6867000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41500056b0 tx=0x7f41500056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f65285bb6e4d6980 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.455+0000 7f4155600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f4155600640 1 -- 172.21.15.44:0/2602480265 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150028020 con 0x5638d685c500 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f4155600640 1 -- 172.21.15.44:0/2602480265 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f4150016040 con 0x5638d685c500 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41500041d0 con 0x5638d685c500 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f4155600640 1 -- 172.21.15.44:0/2602480265 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4150033050 con 0x5638d685c500 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f417142f740 10 client.27503 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5638d674c820 con 0x5638d685c500 2024-04-20T11:30:06.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.456+0000 7f4155600640 1 -- 172.21.15.44:0/2602480265 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41500041d0 con 0x5638d685c500 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f4155600640 1 -- 172.21.15.44:0/2602480265 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f4150004920 con 0x5638d685c500 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f417142f740 10 client.27503 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f417142f740 10 client.27503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f417142f740 20 client.27503 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f417142f740 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d6879990 0x5638d687bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f417142f740 4 client.27503 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638d67cea70 con 0x5638d6879990 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.457+0000 7f4167400640 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d6879990 0x5638d687bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.458+0000 7f4167400640 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d6879990 0x5638d687bd80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=698dc6e10fc2548e server_cookie=28bb590ba4bb188 in_seq=0 out_seq=0 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.458+0000 7f4155600640 10 client.27503 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.458+0000 7f4155600640 1 -- 172.21.15.44:0/2602480265 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638d67cea70 con 0x5638d6879990 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.458+0000 7f4155600640 10 client.27503 handle_command_reply: tid=0 2024-04-20T11:30:06.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.458+0000 7f417142f740 1 client.27503 shutdown 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 msgr2=0x5638d6867000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6867000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41500056b0 tx=0x7f41500056e0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 shutdown_connections 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d6879990 0x5638d687bd80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d6869a80 0x5638d6867540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d6867000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 --2- 172.21.15.44:0/2602480265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d685bb50 0x5638d6866ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 >> 172.21.15.44:0/2602480265 conn(0x5638d68568a0 msgr2=0x5638d68585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f4155600640 0 client.27503 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 shutdown_connections 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 1 -- 172.21.15.44:0/2602480265 wait complete. 2024-04-20T11:30:06.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.459+0000 7f417142f740 20 client.27503 trim_cache size 0 max 16384 2024-04-20T11:30:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 Processor -- start 2024-04-20T11:30:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 -- start start 2024-04-20T11:30:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685bb50 0x5638d68691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 0x5638d6875740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d6867310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41681ab950 con 0x5638d685c500 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41681ad490 con 0x5638d685bb50 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.474+0000 7f417142f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41681ac020 con 0x5638d6869a80 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f415ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 0x5638d6875740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f415ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 0x5638d6875740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44260/0 (socket says 172.21.15.44:44260) 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685bb50 0x5638d68691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f415ea00640 1 -- 172.21.15.44:0/2603247351 learned_addr learned my addr 172.21.15.44:0/2603247351 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167e00640 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d6867310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 -- 172.21.15.44:0/2603247351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 msgr2=0x5638d6867310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d6867310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 -- 172.21.15.44:0/2603247351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 msgr2=0x5638d6875740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 0x5638d6875740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 -- 172.21.15.44:0/2603247351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638d6865a70 con 0x5638d685bb50 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f415ea00640 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 0x5638d6875740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.475+0000 7f4167400640 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685bb50 0x5638d68691f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f41580f0540 tx=0x7f41580f6820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d492c7470e6a5a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f4157e00640 1 -- 172.21.15.44:0/2603247351 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580bb060 con 0x5638d685bb50 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f4157e00640 1 -- 172.21.15.44:0/2603247351 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41580f4040 con 0x5638d685bb50 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f4157e00640 1 -- 172.21.15.44:0/2603247351 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580be720 con 0x5638d685bb50 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 -- 172.21.15.44:0/2603247351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685bb50 msgr2=0x5638d68691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685bb50 0x5638d68691f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f41580f0540 tx=0x7f41580f6820 comp rx=0 tx=0).stop 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 -- 172.21.15.44:0/2603247351 shutdown_connections 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d6867310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685c500 0x5638d6875740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 --2- 172.21.15.44:0/2603247351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685bb50 0x5638d68691f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.476+0000 7f417142f740 1 -- 172.21.15.44:0/2603247351 >> 172.21.15.44:0/2603247351 conn(0x5638d68568a0 msgr2=0x5638d6856c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.477+0000 7f417142f740 1 -- 172.21.15.44:0/2603247351 shutdown_connections 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.477+0000 7f417142f740 1 -- 172.21.15.44:0/2603247351 wait complete. 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.477+0000 7f417142f740 1 -- start start 2024-04-20T11:30:06.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.477+0000 7f417142f740 1 Processor -- start 2024-04-20T11:30:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d6871de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d686d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41681acdc0 con 0x5638d685bb50 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41681ac6f0 con 0x5638d685c500 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41681a9940 con 0x5638d6869a80 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f417142f740 10 client.0 _ll_register_callbacks cb 0x7f4150009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d6871de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d6871de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44268/0 (socket says 172.21.15.44:44268) 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 -- 172.21.15.44:0/153849859 learned_addr learned my addr 172.21.15.44:0/153849859 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167e00640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f415ea00640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d686d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 -- 172.21.15.44:0/153849859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 msgr2=0x5638d686d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 -- 172.21.15.44:0/153849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 msgr2=0x5638d686d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d686d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.478+0000 7f4167400640 1 -- 172.21.15.44:0/153849859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638d64d33f0 con 0x5638d685bb50 2024-04-20T11:30:06.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4167400640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d6871de0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f41580beac0 tx=0x7f4158009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab5e92b0ee93423 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4155600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4155600640 1 -- 172.21.15.44:0/153849859 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580d7070 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4155600640 1 -- 172.21.15.44:0/153849859 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f41580bce60 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41580f4040 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4155600640 1 -- 172.21.15.44:0/153849859 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f41580be900 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f417142f740 10 client.17385 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5638d659d7d0 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4155600640 1 -- 172.21.15.44:0/153849859 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41580f4040 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.479+0000 7f4155600640 1 -- 172.21.15.44:0/153849859 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f41580bb430 con 0x5638d685bb50 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f417142f740 10 client.17385 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f417142f740 10 client.17385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f417142f740 20 client.17385 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f417142f740 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d68dff50 0x5638d68e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f417142f740 4 client.17385 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638d67c88e0 con 0x5638d68dff50 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f415ea00640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d68dff50 0x5638d68e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f415ea00640 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d68dff50 0x5638d68e2370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce77159b9e7e35b8 server_cookie=b2fdc33e1845eeb6 in_seq=0 out_seq=0 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.480+0000 7f4155600640 10 client.17385 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f4155600640 1 -- 172.21.15.44:0/153849859 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638d67c88e0 con 0x5638d68dff50 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f4155600640 10 client.17385 handle_command_reply: tid=0 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 client.17385 shutdown 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 msgr2=0x5638d6871de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d6871de0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f41580beac0 tx=0x7f4158009500 comp rx=0 tx=0).stop 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 shutdown_connections 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5638d68dff50 0x5638d68e2370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638d6869a80 0x5638d686d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638d685c500 0x5638d686d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 --2- 172.21.15.44:0/153849859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638d685bb50 0x5638d6871de0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f4155600640 0 client.17385 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:06.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 >> 172.21.15.44:0/153849859 conn(0x5638d68568a0 msgr2=0x5638d68672f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 shutdown_connections 2024-04-20T11:30:06.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 1 -- 172.21.15.44:0/153849859 wait complete. 2024-04-20T11:30:06.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.481+0000 7f417142f740 20 client.17385 trim_cache size 0 max 16384 2024-04-20T11:30:06.486 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f414c0a3d20 msgr2=0x7f414c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f414c0a3d20 0x7f414c0a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4160012880 tx=0x7f4160005f90 comp rx=0 tx=0).stop 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168102070 msgr2=0x7f416819e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168102070 0x7f416819e450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4158000f80 tx=0x7f4158033720 comp rx=0 tx=0).stop 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 shutdown_connections 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f414c0a3d20 0x7f414c0a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4168198650 0x7f4168198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4168107140 0x7f416819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 --2- 172.21.15.44:0/3227339468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4168102070 0x7f416819e450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 >> 172.21.15.44:0/3227339468 conn(0x7f41680fd930 msgr2=0x7f41680ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 shutdown_connections 2024-04-20T11:30:06.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.482+0000 7f416d200640 1 -- 172.21.15.44:0/3227339468 wait complete. 2024-04-20T11:30:06.501 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:30:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 -- 172.21.15.44:0/1102942962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85001038e0 msgr2=0x7f850010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 --2- 172.21.15.44:0/1102942962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85001038e0 0x7f850010bdc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e802bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 -- 172.21.15.44:0/1102942962 shutdown_connections 2024-04-20T11:30:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 --2- 172.21.15.44:0/1102942962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f850010c430 0x7f850010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 --2- 172.21.15.44:0/1102942962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85001038e0 0x7f850010bdc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 --2- 172.21.15.44:0/1102942962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500102fc0 0x7f85001033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 -- 172.21.15.44:0/1102942962 >> 172.21.15.44:0/1102942962 conn(0x7f85000fc7c0 msgr2=0x7f85000febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 -- 172.21.15.44:0/1102942962 shutdown_connections 2024-04-20T11:30:06.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.716+0000 7f84ffe00640 1 -- 172.21.15.44:0/1102942962 wait complete. 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.717+0000 7f84ffe00640 1 Processor -- start 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.717+0000 7f84ffe00640 1 -- start start 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 0x7f8500100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001038e0 0x7f8500101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850010c430 0x7f8500101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ffe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85001122b0 con 0x7f8500102fc0 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ffe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8500112130 con 0x7f85001038e0 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ffe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8500112430 con 0x7f850010c430 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84fea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 0x7f8500100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001038e0 0x7f8500101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84fea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 0x7f8500100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44286/0 (socket says 172.21.15.44:44286) 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84fea00640 1 -- 172.21.15.44:0/2881676320 learned_addr learned my addr 172.21.15.44:0/2881676320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.718+0000 7f84ff400640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850010c430 0x7f8500101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84fea00640 1 -- 172.21.15.44:0/2881676320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850010c430 msgr2=0x7f8500101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84fea00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850010c430 0x7f8500101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84fea00640 1 -- 172.21.15.44:0/2881676320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001038e0 msgr2=0x7f8500101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84fea00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001038e0 0x7f8500101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84fea00640 1 -- 172.21.15.44:0/2881676320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a6cf0 con 0x7f8500102fc0 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84ff400640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850010c430 0x7f8500101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84fea00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 0x7f8500100c10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f84f000e9f0 tx=0x7f84f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84f7400640 1 -- 172.21.15.44:0/2881676320 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f000cd80 con 0x7f8500102fc0 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84f7400640 1 -- 172.21.15.44:0/2881676320 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f0005520 con 0x7f8500102fc0 2024-04-20T11:30:06.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a6fe0 con 0x7f8500102fc0 2024-04-20T11:30:06.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84f7400640 1 -- 172.21.15.44:0/2881676320 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0010690 con 0x7f8500102fc0 2024-04-20T11:30:06.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.719+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a7450 con 0x7f8500102fc0 2024-04-20T11:30:06.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 Processor -- start 2024-04-20T11:30:06.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 -- start start 2024-04-20T11:30:06.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f84f7400640 1 -- 172.21.15.44:0/2881676320 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f84f00046c0 con 0x7f8500102fc0 2024-04-20T11:30:06.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 0x5647e0cf8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d03a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5647e0c5d050 con 0x5647e0d01670 2024-04-20T11:30:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647e0c03490 con 0x5647e0cf8170 2024-04-20T11:30:06.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f85085fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647e0c59010 con 0x5647e0cf8a90 2024-04-20T11:30:06.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f84ff400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 0x5647e0cf8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d03a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f84ff400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 0x5647e0cf8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42038/0 (socket says 172.21.15.44:42038) 2024-04-20T11:30:06.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f84ff400640 1 -- 172.21.15.44:0/3353992376 learned_addr learned my addr 172.21.15.44:0/3353992376 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.723+0000 7f84fea00640 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84ff400640 1 -- 172.21.15.44:0/3353992376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 msgr2=0x5647e0d01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84ff400640 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d01130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84ff400640 1 -- 172.21.15.44:0/3353992376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 msgr2=0x5647e0d03a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84ff400640 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d03a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84ff400640 1 -- 172.21.15.44:0/3353992376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e0c0b070 con 0x5647e0cf8170 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84fea00640 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84ff400640 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 0x5647e0cf8550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009fd0 tx=0x7f84f8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc874020b4ec3b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84efe00640 1 -- 172.21.15.44:0/3353992376 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8011700 con 0x5647e0cf8170 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84efe00640 1 -- 172.21.15.44:0/3353992376 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f8011da0 con 0x5647e0cf8170 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.724+0000 7f84efe00640 1 -- 172.21.15.44:0/3353992376 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f800e3c0 con 0x5647e0cf8170 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.725+0000 7f84f7400640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84e40a3d20 0x7f84e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.725+0000 7f84f7400640 1 -- 172.21.15.44:0/2881676320 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f84f00d0cc0 con 0x7f8500102fc0 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.725+0000 7f84fea00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84e40a3d20 0x7f84e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.725+0000 7f85085fd740 1 -- 172.21.15.44:0/3353992376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 msgr2=0x5647e0cf8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.725+0000 7f85085fd740 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 0x5647e0cf8550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009fd0 tx=0x7f84f8010860 comp rx=0 tx=0).stop 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 -- 172.21.15.44:0/3353992376 shutdown_connections 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d03a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 --2- 172.21.15.44:0/3353992376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8170 0x5647e0cf8550 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 -- 172.21.15.44:0/3353992376 >> 172.21.15.44:0/3353992376 conn(0x5647e0cf28a0 msgr2=0x5647e0cf2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 -- 172.21.15.44:0/3353992376 shutdown_connections 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f84fea00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84e40a3d20 0x7f84e40a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x5647e0cf7c40 tx=0x7f84f0004c10 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 -- 172.21.15.44:0/3353992376 wait complete. 2024-04-20T11:30:06.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 -- start start 2024-04-20T11:30:06.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.726+0000 7f85085fd740 1 Processor -- start 2024-04-20T11:30:06.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0cfe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0cfebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0cff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5647e0d04460 con 0x5647e0d01670 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647e0c16860 con 0x5647e0cf8a90 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647e0d045e0 con 0x5647e0cf8170 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f85085fd740 10 client.0 _ll_register_callbacks cb 0x5647e0be4fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0cfebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0cfebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42044/0 (socket says 172.21.15.44:42044) 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f84f7e00640 1 -- 172.21.15.44:0/850305568 learned_addr learned my addr 172.21.15.44:0/850305568 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f84ff400640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0cff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.727+0000 7f84ff400640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0cfe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84f7e00640 1 -- 172.21.15.44:0/850305568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 msgr2=0x5647e0cfe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84f7e00640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0cfe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84f7e00640 1 -- 172.21.15.44:0/850305568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 msgr2=0x5647e0cff110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84f7e00640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0cff110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84f7e00640 1 -- 172.21.15.44:0/850305568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e0c1a950 con 0x5647e0cf8a90 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84ff400640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0cff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84f7e00640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0cfebd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005ec0 tx=0x7f84e802cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ea7b12b4c40a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84ecc00640 1 -- 172.21.15.44:0/850305568 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8043070 con 0x5647e0cf8a90 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f8011da0 con 0x5647e0cf8a90 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.728+0000 7f84ecc00640 1 -- 172.21.15.44:0/850305568 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f84e8004250 con 0x5647e0cf8a90 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f84ecc00640 1 -- 172.21.15.44:0/850305568 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f84e804f050 con 0x5647e0cf8a90 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 10 client.27521 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5647e0be8820 con 0x5647e0cf8a90 2024-04-20T11:30:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f84ecc00640 1 -- 172.21.15.44:0/850305568 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f8011da0 con 0x5647e0cf8a90 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f84ecc00640 1 -- 172.21.15.44:0/850305568 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f84e8004b20 con 0x5647e0cf8a90 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 10 client.27521 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 10 client.27521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 20 client.27521 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d115e0 0x5647e0d13a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 4 client.27521 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.729+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647e0c6aa70 con 0x5647e0d115e0 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.730+0000 7f84ff400640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d115e0 0x5647e0d13a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.730+0000 7f84ff400640 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d115e0 0x5647e0d13a00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a5336555119af9 server_cookie=6d7f6b305ab08615 in_seq=0 out_seq=0 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.730+0000 7f84ecc00640 10 client.27521 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.731+0000 7f84ecc00640 1 -- 172.21.15.44:0/850305568 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647e0c6aa70 con 0x5647e0d115e0 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.731+0000 7f84ecc00640 10 client.27521 handle_command_reply: tid=0 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.731+0000 7f85085fd740 1 client.27521 shutdown 2024-04-20T11:30:06.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 msgr2=0x5647e0cfebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0cfebd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005ec0 tx=0x7f84e802cd20 comp rx=0 tx=0).stop 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 shutdown_connections 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d115e0 0x5647e0d13a00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0cff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0cfebd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f84ecc00640 0 client.27521 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 --2- 172.21.15.44:0/850305568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0cfe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 >> 172.21.15.44:0/850305568 conn(0x5647e0cf28a0 msgr2=0x5647e0cf45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 shutdown_connections 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 1 -- 172.21.15.44:0/850305568 wait complete. 2024-04-20T11:30:06.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.732+0000 7f85085fd740 20 client.27521 trim_cache size 0 max 16384 2024-04-20T11:30:06.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 Processor -- start 2024-04-20T11:30:06.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 -- start start 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 0x5647e0d00dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d0d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0d01670 0x5647e0cf1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85001afcd0 con 0x5647e0cf8170 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b1810 con 0x5647e0d01670 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.747+0000 7f85085fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b03a0 con 0x5647e0cf8a90 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 0x5647e0d00dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 0x5647e0d00dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44312/0 (socket says 172.21.15.44:44312) 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 -- 172.21.15.44:0/347162415 learned_addr learned my addr 172.21.15.44:0/347162415 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84f7e00640 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d0d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0d01670 0x5647e0cf1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 -- 172.21.15.44:0/347162415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 msgr2=0x5647e0d0d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d0d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 -- 172.21.15.44:0/347162415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0d01670 msgr2=0x5647e0cf1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0d01670 0x5647e0cf1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 -- 172.21.15.44:0/347162415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e0cfd640 con 0x5647e0cf8170 2024-04-20T11:30:06.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84f7e00640 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d0d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:06.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.748+0000 7f84ff400640 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 0x5647e0d00dc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f84f801bc20 tx=0x7f84f801bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5ca22323c1d831 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f84ef400640 1 -- 172.21.15.44:0/347162415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80049e0 con 0x5647e0cf8170 2024-04-20T11:30:06.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f84ef400640 1 -- 172.21.15.44:0/347162415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f801b930 con 0x5647e0cf8170 2024-04-20T11:30:06.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f84ef400640 1 -- 172.21.15.44:0/347162415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f802b5e0 con 0x5647e0cf8170 2024-04-20T11:30:06.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 -- 172.21.15.44:0/347162415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 msgr2=0x5647e0d00dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 0x5647e0d00dc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f84f801bc20 tx=0x7f84f801bb10 comp rx=0 tx=0).stop 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 -- 172.21.15.44:0/347162415 shutdown_connections 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0d01670 0x5647e0cf1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8a90 0x5647e0d0d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 --2- 172.21.15.44:0/347162415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0cf8170 0x5647e0d00dc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 -- 172.21.15.44:0/347162415 >> 172.21.15.44:0/347162415 conn(0x5647e0cf28a0 msgr2=0x5647e0cf2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 -- 172.21.15.44:0/347162415 shutdown_connections 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.749+0000 7f85085fd740 1 -- 172.21.15.44:0/347162415 wait complete. 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.750+0000 7f85085fd740 1 -- start start 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.750+0000 7f85085fd740 1 Processor -- start 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0d05870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0d05db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b1140 con 0x5647e0d01670 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b0a70 con 0x5647e0cf8a90 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f85001adcc0 con 0x5647e0cf8170 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f85085fd740 10 client.0 _ll_register_callbacks cb 0x7f84e802c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0d05db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84ff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0d05870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0d05db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42066/0 (socket says 172.21.15.44:42066) 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 -- 172.21.15.44:0/1850285497 learned_addr learned my addr 172.21.15.44:0/1850285497 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84ff400640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 -- 172.21.15.44:0/1850285497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 msgr2=0x5647e0d05870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0d05870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 -- 172.21.15.44:0/1850285497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 msgr2=0x5647e0d76b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d76b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84f7e00640 1 -- 172.21.15.44:0/1850285497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e0cf88c0 con 0x5647e0cf8a90 2024-04-20T11:30:06.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.751+0000 7f84ff400640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f84f7e00640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0d05db0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f84e8001eb0 tx=0x7f84e80097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ce58ea1d26bbbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f84ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f84ecc00640 1 -- 172.21.15.44:0/1850285497 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e804c070 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f84ecc00640 1 -- 172.21.15.44:0/1850285497 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8455+0+0 (secure 0 0 0) 0x7f84e80057f0 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f801b930 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f84ecc00640 1 -- 172.21.15.44:0/1850285497 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f84e8043040 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f85085fd740 10 client.27527 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.752+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5647e096f3f0 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f84ecc00640 1 -- 172.21.15.44:0/1850285497 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f801b930 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f84ecc00640 1 -- 172.21.15.44:0/1850285497 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f84e8004960 con 0x5647e0cf8a90 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f85085fd740 10 client.27527 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f85085fd740 10 client.27527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:06.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f85085fd740 20 client.27527 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:06.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f85085fd740 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d78620 0x5647e0d7aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:06.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f85085fd740 4 client.27527 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:06.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647e0c648e0 con 0x5647e0d78620 2024-04-20T11:30:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.753+0000 7f84ff400640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d78620 0x5647e0d7aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f84ff400640 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d78620 0x5647e0d7aa40 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad7b9532632b06b server_cookie=4139d2a7f183e46b in_seq=0 out_seq=0 2024-04-20T11:30:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f84ecc00640 10 client.27527 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f84ecc00640 1 -- 172.21.15.44:0/1850285497 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647e0c648e0 con 0x5647e0d78620 2024-04-20T11:30:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f84ecc00640 10 client.27527 handle_command_reply: tid=0 2024-04-20T11:30:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 client.27527 shutdown 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 msgr2=0x5647e0d05db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0d05db0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f84e8001eb0 tx=0x7f84e80097c0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 shutdown_connections 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5647e0d78620 0x5647e0d7aa40 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5647e0d01670 0x5647e0d76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647e0cf8a90 0x5647e0d05db0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 --2- 172.21.15.44:0/1850285497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647e0cf8170 0x5647e0d05870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 >> 172.21.15.44:0/1850285497 conn(0x5647e0cf28a0 msgr2=0x5647e0cfeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f84ecc00640 0 client.27527 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 shutdown_connections 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.754+0000 7f85085fd740 1 -- 172.21.15.44:0/1850285497 wait complete. 2024-04-20T11:30:06.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f85085fd740 20 client.27527 trim_cache size 0 max 16384 2024-04-20T11:30:06.760 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:06.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84e40a3d20 msgr2=0x7f84e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84e40a3d20 0x7f84e40a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x5647e0cf7c40 tx=0x7f84f0004c10 comp rx=0 tx=0).stop 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 msgr2=0x7f8500100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 0x7f8500100c10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f84f000e9f0 tx=0x7f84f000edf0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 shutdown_connections 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850010c430 0x7f8500101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001038e0 0x7f8500101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84e40a3d20 0x7f84e40a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 --2- 172.21.15.44:0/2881676320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8500102fc0 0x7f8500100c10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 >> 172.21.15.44:0/2881676320 conn(0x7f85000fc7c0 msgr2=0x7f85000fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.755+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 shutdown_connections 2024-04-20T11:30:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:06.756+0000 7f84ffe00640 1 -- 172.21.15.44:0/2881676320 wait complete. 2024-04-20T11:30:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:07 smithi165 ceph-mon[21975]: pgmap v230: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:30:07.775 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:30:07.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:30:07.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:30:07.856 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:07 smithi044 ceph-mon[16855]: pgmap v230: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:30:07.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:07 smithi078 ceph-mon[21464]: pgmap v230: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 -- 172.21.15.44:0/2231088948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0100ba0 msgr2=0x7f16e0101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2231088948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0100ba0 0x7f16e0101020 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009900 tx=0x7f16c802bc40 comp rx=0 tx=0).stop 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 -- 172.21.15.44:0/2231088948 shutdown_connections 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2231088948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0101560 0x7f16e010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2231088948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0100ba0 0x7f16e0101020 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2231088948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 -- 172.21.15.44:0/2231088948 >> 172.21.15.44:0/2231088948 conn(0x7f16e0075c60 msgr2=0x7f16e0076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.995+0000 7f16dfe00640 1 -- 172.21.15.44:0/2231088948 shutdown_connections 2024-04-20T11:30:07.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.996+0000 7f16dfe00640 1 -- 172.21.15.44:0/2231088948 wait complete. 2024-04-20T11:30:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 Processor -- start 2024-04-20T11:30:07.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 -- start start 2024-04-20T11:30:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0100ba0 0x7f16e01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0101560 0x7f16e01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:07.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01122d0 con 0x7f16e0101560 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0112150 con 0x7f16e0106590 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16dfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0112450 con 0x7f16e0100ba0 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16de000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0101560 0x7f16e01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42094/0 (socket says 172.21.15.44:42094) 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.997+0000 7f16df400640 1 -- 172.21.15.44:0/2335730337 learned_addr learned my addr 172.21.15.44:0/2335730337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16dea00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0100ba0 0x7f16e01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16df400640 1 -- 172.21.15.44:0/2335730337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0100ba0 msgr2=0x7f16e01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16df400640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0100ba0 0x7f16e01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16df400640 1 -- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0101560 msgr2=0x7f16e01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16df400640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0101560 0x7f16e01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16df400640 1 -- 172.21.15.44:0/2335730337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e019b030 con 0x7f16e0106590 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16dea00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0100ba0 0x7f16e01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16de000640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0101560 0x7f16e01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.998+0000 7f16df400640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e019a770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f16d800eaa0 tx=0x7f16d800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.999+0000 7f16d7400640 1 -- 172.21.15.44:0/2335730337 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800cdb0 con 0x7f16e0106590 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.999+0000 7f16d7400640 1 -- 172.21.15.44:0/2335730337 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16d8004b20 con 0x7f16e0106590 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.999+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e019b320 con 0x7f16e0106590 2024-04-20T11:30:08.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.999+0000 7f16d7400640 1 -- 172.21.15.44:0/2335730337 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8011760 con 0x7f16e0106590 2024-04-20T11:30:08.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:07.999+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01a7630 con 0x7f16e0106590 2024-04-20T11:30:08.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.002+0000 7f16e8821740 1 Processor -- start 2024-04-20T11:30:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.002+0000 7f16d7400640 1 -- 172.21.15.44:0/2335730337 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f16d8004cd0 con 0x7f16e0106590 2024-04-20T11:30:08.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.002+0000 7f16e8821740 1 -- start start 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16e8821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645af0f0 0x555a645af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16e8821740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 0x555a645b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16e8821740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645b85f0 0x555a645ba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16e8821740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a64513d20 con 0x555a645afa10 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16e8821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a644ba3e0 con 0x555a645af0f0 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16e8821740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555a6450fce0 con 0x555a645b85f0 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16dea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 0x555a645b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16de000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645af0f0 0x555a645af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16dea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 0x555a645b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44346/0 (socket says 172.21.15.44:44346) 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16dea00640 1 -- 172.21.15.44:0/2991066744 learned_addr learned my addr 172.21.15.44:0/2991066744 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.003+0000 7f16de000640 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645b85f0 0x555a645ba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16dea00640 1 -- 172.21.15.44:0/2991066744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645b85f0 msgr2=0x555a645ba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16dea00640 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645b85f0 0x555a645ba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16dea00640 1 -- 172.21.15.44:0/2991066744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645af0f0 msgr2=0x555a645af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16dea00640 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645af0f0 0x555a645af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16dea00640 1 -- 172.21.15.44:0/2991066744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a644d9980 con 0x555a645afa10 2024-04-20T11:30:08.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16dea00640 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 0x555a645b80b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009d50 tx=0x7f16d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6589257d07d57d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16d7400640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16c40a3da0 0x7f16c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16cfe00640 1 -- 172.21.15.44:0/2991066744 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0012650 con 0x555a645afa10 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16d7400640 1 -- 172.21.15.44:0/2335730337 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8499+0+0 (secure 0 0 0) 0x7f16d80d0a30 con 0x7f16e0106590 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.004+0000 7f16cfe00640 1 -- 172.21.15.44:0/2991066744 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16d0009e40 con 0x555a645afa10 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16cfe00640 1 -- 172.21.15.44:0/2991066744 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d000fd80 con 0x555a645afa10 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16de000640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16c40a3da0 0x7f16c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 -- 172.21.15.44:0/2991066744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 msgr2=0x555a645b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 0x555a645b80b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009d50 tx=0x7f16d00117b0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16de000640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16c40a3da0 0x7f16c40a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f16c8006010 tx=0x7f16c8005b70 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 -- 172.21.15.44:0/2991066744 shutdown_connections 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645b85f0 0x555a645ba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645afa10 0x555a645b80b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 --2- 172.21.15.44:0/2991066744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645af0f0 0x555a645af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.005+0000 7f16e8821740 1 -- 172.21.15.44:0/2991066744 >> 172.21.15.44:0/2991066744 conn(0x555a645a9820 msgr2=0x555a645a9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.006+0000 7f16e8821740 1 -- 172.21.15.44:0/2991066744 shutdown_connections 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.006+0000 7f16e8821740 1 -- 172.21.15.44:0/2991066744 wait complete. 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.006+0000 7f16e8821740 1 -- start start 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.006+0000 7f16e8821740 1 Processor -- start 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555a645bb3e0 con 0x555a645b85f0 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555a644cd7a0 con 0x555a645afa10 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555a645bb560 con 0x555a645af0f0 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16e8821740 10 client.0 _ll_register_callbacks cb 0x555a644cfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16de000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44354/0 (socket says 172.21.15.44:44354) 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16df400640 1 -- 172.21.15.44:0/2782619813 learned_addr learned my addr 172.21.15.44:0/2782619813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16de000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42106/0 (socket says 172.21.15.44:42106) 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16dea00640 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16df400640 1 -- 172.21.15.44:0/2782619813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 msgr2=0x555a645b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.007+0000 7f16df400640 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16df400640 1 -- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 msgr2=0x555a645b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16df400640 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16df400640 1 -- 172.21.15.44:0/2782619813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a642ee7e0 con 0x555a645b85f0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16df400640 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645b6050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009270 tx=0x7f16d809d1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722ca0aed721bcf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16ccc00640 1 -- 172.21.15.44:0/2782619813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d801f040 con 0x555a645b85f0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16ccc00640 1 -- 172.21.15.44:0/2782619813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8499+0+0 (secure 0 0 0) 0x7f16d80872c0 con 0x555a645b85f0 2024-04-20T11:30:08.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a64521740 con 0x555a645b85f0 2024-04-20T11:30:08.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16ccc00640 1 -- 172.21.15.44:0/2782619813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f16d80bbb40 con 0x555a645b85f0 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16e8821740 10 client.17415 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.008+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555a6434ef70 con 0x555a645b85f0 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16ccc00640 1 -- 172.21.15.44:0/2782619813 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x555a64521740 con 0x555a645b85f0 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16ccc00640 1 -- 172.21.15.44:0/2782619813 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f16d800f040 con 0x555a645b85f0 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16e8821740 10 client.17415 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16e8821740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16e8821740 20 client.17415 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16e8821740 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a645c89d0 0x555a645cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16e8821740 4 client.17415 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.009+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a6451b5b0 con 0x555a645c89d0 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.010+0000 7f16dea00640 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a645c89d0 0x555a645cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.010+0000 7f16dea00640 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a645c89d0 0x555a645cade0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864c4ec5de602ae3 server_cookie=b025b21db50d56e6 in_seq=0 out_seq=0 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.010+0000 7f16ccc00640 10 client.17415 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16ccc00640 1 -- 172.21.15.44:0/2782619813 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a6451b5b0 con 0x555a645c89d0 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16ccc00640 10 client.17415 handle_command_reply: tid=0 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 client.17415 shutdown 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 msgr2=0x555a645b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645b6050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009270 tx=0x7f16d809d1c0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 shutdown_connections 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a645c89d0 0x555a645cade0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645b6050 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 --2- 172.21.15.44:0/2782619813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16ccc00640 0 client.17415 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.011+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 >> 172.21.15.44:0/2782619813 conn(0x555a645a9820 msgr2=0x555a645ab570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.012+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 shutdown_connections 2024-04-20T11:30:08.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.012+0000 7f16e8821740 1 -- 172.21.15.44:0/2782619813 wait complete. 2024-04-20T11:30:08.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.012+0000 7f16e8821740 20 client.17415 trim_cache size 0 max 16384 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 Processor -- start 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 -- start start 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01afd00 con 0x555a645b85f0 2024-04-20T11:30:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01b1840 con 0x555a645afa10 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16e8821740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01b03d0 con 0x555a645af0f0 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16dea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.032+0000 7f16df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46556/0 (socket says 172.21.15.44:46556) 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 -- 172.21.15.44:0/2322143101 learned_addr learned my addr 172.21.15.44:0/2322143101 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16de000640 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 -- 172.21.15.44:0/2322143101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 msgr2=0x555a645c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 -- 172.21.15.44:0/2322143101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 msgr2=0x555a645d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 -- 172.21.15.44:0/2322143101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a641e6170 con 0x555a645af0f0 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16dea00640 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b7d00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009c70 tx=0x7f16d000f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c973d057ff1aa091 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.033+0000 7f16cf400640 1 -- 172.21.15.44:0/2322143101 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0009530 con 0x555a645af0f0 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16cf400640 1 -- 172.21.15.44:0/2322143101 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16d0028070 con 0x555a645af0f0 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16cf400640 1 -- 172.21.15.44:0/2322143101 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d00292c0 con 0x555a645af0f0 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 -- 172.21.15.44:0/2322143101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 msgr2=0x555a645b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b7d00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009c70 tx=0x7f16d000f980 comp rx=0 tx=0).stop 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 -- 172.21.15.44:0/2322143101 shutdown_connections 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645b85f0 0x555a645d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645afa10 0x555a645c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 --2- 172.21.15.44:0/2322143101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645af0f0 0x555a645b7d00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 -- 172.21.15.44:0/2322143101 >> 172.21.15.44:0/2322143101 conn(0x555a645a9820 msgr2=0x555a645a8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 -- 172.21.15.44:0/2322143101 shutdown_connections 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.034+0000 7f16e8821740 1 -- 172.21.15.44:0/2322143101 wait complete. 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.035+0000 7f16e8821740 1 -- start start 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.035+0000 7f16e8821740 1 Processor -- start 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.035+0000 7f16e8821740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645af0f0 0x555a645b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.035+0000 7f16e8821740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645afa10 0x555a645b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.035+0000 7f16e8821740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645b85f0 0x555a6462df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16e8821740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01b1170 con 0x555a645af0f0 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16e8821740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01b0aa0 con 0x555a645b85f0 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16e8821740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01adcf0 con 0x555a645afa10 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16e8821740 10 client.0 _ll_register_callbacks cb 0x7f16d80058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16de000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645afa10 0x555a645b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16dea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645af0f0 0x555a645b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16de000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645afa10 0x555a645b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46566/0 (socket says 172.21.15.44:46566) 2024-04-20T11:30:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16de000640 1 -- 172.21.15.44:0/3131085638 learned_addr learned my addr 172.21.15.44:0/3131085638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16df400640 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645b85f0 0x555a6462df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16dea00640 1 -- 172.21.15.44:0/3131085638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645afa10 msgr2=0x555a645b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16dea00640 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645afa10 0x555a645b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16dea00640 1 -- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645b85f0 msgr2=0x555a6462df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16dea00640 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645b85f0 0x555a6462df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.036+0000 7f16dea00640 1 -- 172.21.15.44:0/3131085638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6462e460 con 0x555a645af0f0 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16dea00640 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645af0f0 0x555a645b2620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009350 tx=0x7f16d0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f93972f672dbac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:08.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16ccc00640 1 -- 172.21.15.44:0/3131085638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d000f560 con 0x555a645af0f0 2024-04-20T11:30:08.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d0028070 con 0x555a645af0f0 2024-04-20T11:30:08.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16ccc00640 1 -- 172.21.15.44:0/3131085638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8499+0+0 (secure 0 0 0) 0x7f16d000f700 con 0x555a645af0f0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16ccc00640 1 -- 172.21.15.44:0/3131085638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f16d000f9f0 con 0x555a645af0f0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16e8821740 10 client.17421 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555a6462f5e0 con 0x555a645af0f0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.037+0000 7f16ccc00640 1 -- 172.21.15.44:0/3131085638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16d0028070 con 0x555a645af0f0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16ccc00640 1 -- 172.21.15.44:0/3131085638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f16d0003680 con 0x555a645af0f0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16e8821740 10 client.17421 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16e8821740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16e8821740 20 client.17421 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16e8821740 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a6462fbf0 0x555a64632010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16e8821740 4 client.17421 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a64497b10 con 0x555a6462fbf0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.038+0000 7f16de000640 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a6462fbf0 0x555a64632010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.039+0000 7f16de000640 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a6462fbf0 0x555a64632010 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2308fe113a9ccb26 server_cookie=1bc591c620b5986 in_seq=0 out_seq=0 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.039+0000 7f16ccc00640 10 client.17421 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16ccc00640 1 -- 172.21.15.44:0/3131085638 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a64497b10 con 0x555a6462fbf0 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16ccc00640 10 client.17421 handle_command_reply: tid=0 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 client.17421 shutdown 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645af0f0 msgr2=0x555a645b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645af0f0 0x555a645b2620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009350 tx=0x7f16d0009620 comp rx=0 tx=0).stop 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 shutdown_connections 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x555a6462fbf0 0x555a64632010 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a645b85f0 0x555a6462df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a645afa10 0x555a645b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 --2- 172.21.15.44:0/3131085638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555a645af0f0 0x555a645b2620 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16ccc00640 0 client.17421 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.040+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 >> 172.21.15.44:0/3131085638 conn(0x555a645a9820 msgr2=0x555a645b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 shutdown_connections 2024-04-20T11:30:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16e8821740 1 -- 172.21.15.44:0/3131085638 wait complete. 2024-04-20T11:30:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16e8821740 20 client.17421 trim_cache size 0 max 16384 2024-04-20T11:30:08.044 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:30:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:30:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "6bb6bf9d-0cc4-4f7d-895b-f33a17fc27ee", 2024-04-20T11:30:08.044 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:30:08.044 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16c40a3da0 msgr2=0x7f16c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16c40a3da0 0x7f16c40a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f16c8006010 tx=0x7f16c8005b70 comp rx=0 tx=0).stop 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 msgr2=0x7f16e019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.041+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e019a770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f16d800eaa0 tx=0x7f16d800ef60 comp rx=0 tx=0).stop 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 shutdown_connections 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e0106590 0x7f16e019a770 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16c40a3da0 0x7f16c40a6260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e0101560 0x7f16e01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 --2- 172.21.15.44:0/2335730337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e0100ba0 0x7f16e01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 >> 172.21.15.44:0/2335730337 conn(0x7f16e0075c60 msgr2=0x7f16e00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 shutdown_connections 2024-04-20T11:30:08.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.042+0000 7f16dfe00640 1 -- 172.21.15.44:0/2335730337 wait complete. 2024-04-20T11:30:08.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:30:08.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 -- 172.21.15.44:0/451966512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 msgr2=0x7fd5f8111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 --2- 172.21.15.44:0/451966512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f8111050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009ca0 tx=0x7fd5f002c0f0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 -- 172.21.15.44:0/451966512 shutdown_connections 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 --2- 172.21.15.44:0/451966512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f8111050 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 --2- 172.21.15.44:0/451966512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5f8104d60 0x7fd5f8108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 --2- 172.21.15.44:0/451966512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f81043b0 0x7fd5f8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 -- 172.21.15.44:0/451966512 >> 172.21.15.44:0/451966512 conn(0x7fd5f80ffbd0 msgr2=0x7fd5f8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 -- 172.21.15.44:0/451966512 shutdown_connections 2024-04-20T11:30:08.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.279+0000 7fd5fe400640 1 -- 172.21.15.44:0/451966512 wait complete. 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.280+0000 7fd5fe400640 1 Processor -- start 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.280+0000 7fd5fe400640 1 -- start start 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5fe400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5f81043b0 0x7fd5f81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5fe400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f81a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5fe400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f819a8f0 0x7fd5f819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5fe400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8114650 con 0x7fd5f8109480 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5fe400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81144d0 con 0x7fd5f819a8f0 2024-04-20T11:30:08.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5fe400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81147d0 con 0x7fd5f81043b0 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f81a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f81a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44396/0 (socket says 172.21.15.44:44396) 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 -- 172.21.15.44:0/2454785855 learned_addr learned my addr 172.21.15.44:0/2454785855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5f7400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5f81043b0 0x7fd5f81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f819a8f0 0x7fd5f819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 -- 172.21.15.44:0/2454785855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5f81043b0 msgr2=0x7fd5f81a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5f81043b0 0x7fd5f81a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 -- 172.21.15.44:0/2454785855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f819a8f0 msgr2=0x7fd5f819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f819a8f0 0x7fd5f819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5eea00640 1 -- 172.21.15.44:0/2454785855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f819b410 con 0x7fd5f8109480 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.281+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f819a8f0 0x7fd5f819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.282+0000 7fd5eea00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f81a0c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.282+0000 7fd5f4c00640 1 -- 172.21.15.44:0/2454785855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000cd80 con 0x7fd5f8109480 2024-04-20T11:30:08.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.282+0000 7fd5f4c00640 1 -- 172.21.15.44:0/2454785855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5e0005520 con 0x7fd5f8109480 2024-04-20T11:30:08.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.282+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f819b700 con 0x7fd5f8109480 2024-04-20T11:30:08.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.282+0000 7fd5f4c00640 1 -- 172.21.15.44:0/2454785855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0010690 con 0x7fd5f8109480 2024-04-20T11:30:08.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.282+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81a7950 con 0x7fd5f8109480 2024-04-20T11:30:08.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd5f4c00640 1 -- 172.21.15.44:0/2454785855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd5e00046c0 con 0x7fd5f8109480 2024-04-20T11:30:08.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 Processor -- start 2024-04-20T11:30:08.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 -- start start 2024-04-20T11:30:08.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97481a0 0x564ac9748580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac9751160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97516a0 0x564ac9753a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564ac96ae150 con 0x564ac97516a0 2024-04-20T11:30:08.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ac9654370 con 0x564ac97481a0 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.286+0000 7fd60122a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564ac96aa110 con 0x564ac9748ac0 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac9751160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac9751160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46604/0 (socket says 172.21.15.44:46604) 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 -- 172.21.15.44:0/3251161628 learned_addr learned my addr 172.21.15.44:0/3251161628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5eea00640 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97516a0 0x564ac9753a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97481a0 0x564ac9748580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 -- 172.21.15.44:0/3251161628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97481a0 msgr2=0x564ac9748580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97481a0 0x564ac9748580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 -- 172.21.15.44:0/3251161628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97516a0 msgr2=0x564ac9753a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97516a0 0x564ac9753a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7400640 1 -- 172.21.15.44:0/3251161628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac948a7d0 con 0x564ac9748ac0 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97481a0 0x564ac9748580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f4c00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5dc0a3d20 0x7fd5dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.287+0000 7fd5f4c00640 1 -- 172.21.15.44:0/2454785855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8499+0+0 (secure 0 0 0) 0x7fd5e00d0ba0 con 0x7fd5f8109480 2024-04-20T11:30:08.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.288+0000 7fd5f7400640 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac9751160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009c90 tx=0x7fd5e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed505230b382c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.288+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5dc0a3d20 0x7fd5dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.288+0000 7fd5ecc00640 1 -- 172.21.15.44:0/3251161628 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8005220 con 0x564ac9748ac0 2024-04-20T11:30:08.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.288+0000 7fd5ecc00640 1 -- 172.21.15.44:0/3251161628 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5e8004ca0 con 0x564ac9748ac0 2024-04-20T11:30:08.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.288+0000 7fd5ecc00640 1 -- 172.21.15.44:0/3251161628 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8019950 con 0x564ac9748ac0 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 -- 172.21.15.44:0/3251161628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 msgr2=0x564ac9751160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac9751160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009c90 tx=0x7fd5e8011810 comp rx=0 tx=0).stop 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5dc0a3d20 0x7fd5dc0a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0006010 tx=0x7fd5f0004a40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 -- 172.21.15.44:0/3251161628 shutdown_connections 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97516a0 0x564ac9753a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac9751160 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 --2- 172.21.15.44:0/3251161628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97481a0 0x564ac9748580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 -- 172.21.15.44:0/3251161628 >> 172.21.15.44:0/3251161628 conn(0x564ac97428d0 msgr2=0x564ac9742cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 -- 172.21.15.44:0/3251161628 shutdown_connections 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 -- 172.21.15.44:0/3251161628 wait complete. 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.289+0000 7fd60122a740 1 -- start start 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 Processor -- start 2024-04-20T11:30:08.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564ac9754490 con 0x564ac97481a0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ac9667740 con 0x564ac97516a0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.290+0000 7fd60122a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564ac9754610 con 0x564ac9748ac0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd60122a740 10 client.0 _ll_register_callbacks cb 0x564ac9662b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46620/0 (socket says 172.21.15.44:46620) 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5eea00640 1 -- 172.21.15.44:0/785560034 learned_addr learned my addr 172.21.15.44:0/785560034 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7e00640 1 -- 172.21.15.44:0/785560034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 msgr2=0x564ac974ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7e00640 1 -- 172.21.15.44:0/785560034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 msgr2=0x564ac974e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.291+0000 7fd5f7e00640 1 -- 172.21.15.44:0/785560034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac93c03f0 con 0x564ac97516a0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974f100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009bf0 tx=0x7fd5f00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1269662d41e9276 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd5e5600640 1 -- 172.21.15.44:0/785560034 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0039040 con 0x564ac97516a0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e8004ca0 con 0x564ac97516a0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd5e5600640 1 -- 172.21.15.44:0/785560034 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8499+0+0 (secure 0 0 0) 0x7fd5f0040660 con 0x564ac97516a0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd5e5600640 1 -- 172.21.15.44:0/785560034 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd5f0059050 con 0x564ac97516a0 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd60122a740 10 client.27557 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:08.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.292+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564ac93800f0 con 0x564ac97516a0 2024-04-20T11:30:08.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd5e5600640 1 -- 172.21.15.44:0/785560034 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5e8004ca0 con 0x564ac97516a0 2024-04-20T11:30:08.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd5e5600640 1 -- 172.21.15.44:0/785560034 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd5f004b9e0 con 0x564ac97516a0 2024-04-20T11:30:08.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd60122a740 10 client.27557 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:08.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd60122a740 10 client.27557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:08.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd60122a740 20 client.27557 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd60122a740 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97619c0 0x564ac9763dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd60122a740 4 client.27557 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.293+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ac96bbb70 con 0x564ac97619c0 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.294+0000 7fd5f7400640 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97619c0 0x564ac9763dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.294+0000 7fd5f7400640 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97619c0 0x564ac9763dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0ee7f59103b8a9 server_cookie=6935ef9a0334c4ae in_seq=0 out_seq=0 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.294+0000 7fd5e5600640 10 client.27557 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd5e5600640 1 -- 172.21.15.44:0/785560034 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ac96bbb70 con 0x564ac97619c0 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd5e5600640 10 client.27557 handle_command_reply: tid=0 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 client.27557 shutdown 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 msgr2=0x564ac974f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974f100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009bf0 tx=0x7fd5f00095c0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 shutdown_connections 2024-04-20T11:30:08.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97619c0 0x564ac9763dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974f100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd5e5600640 0 client.27557 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 --2- 172.21.15.44:0/785560034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 >> 172.21.15.44:0/785560034 conn(0x564ac97428d0 msgr2=0x564ac9744620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.295+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 shutdown_connections 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.296+0000 7fd60122a740 1 -- 172.21.15.44:0/785560034 wait complete. 2024-04-20T11:30:08.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.296+0000 7fd60122a740 20 client.27557 trim_cache size 0 max 16384 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 Processor -- start 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 -- start start 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac97481a0 0x564ac9750db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 0x564ac975d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac97443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81aff90 con 0x564ac9748ac0 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81b1ad0 con 0x564ac97516a0 2024-04-20T11:30:08.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.315+0000 7fd60122a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81b0660 con 0x564ac97481a0 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 0x564ac975d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 0x564ac975d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43744/0 (socket says 172.21.15.44:43744) 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 -- 172.21.15.44:0/129878363 learned_addr learned my addr 172.21.15.44:0/129878363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5f7400640 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac97481a0 0x564ac9750db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac97443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 -- 172.21.15.44:0/129878363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac97481a0 msgr2=0x564ac9750db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac97481a0 0x564ac9750db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 -- 172.21.15.44:0/129878363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 msgr2=0x564ac97443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac97443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 -- 172.21.15.44:0/129878363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac97488f0 con 0x564ac9748ac0 2024-04-20T11:30:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5eea00640 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 0x564ac975d350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00ce3b0 tx=0x7fd5e00d46f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121a626c3bfe4d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.316+0000 7fd5e7e00640 1 -- 172.21.15.44:0/129878363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0098df0 con 0x564ac9748ac0 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd5e7e00640 1 -- 172.21.15.44:0/129878363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5e009d250 con 0x564ac9748ac0 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd5e7e00640 1 -- 172.21.15.44:0/129878363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e009c4b0 con 0x564ac9748ac0 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 -- 172.21.15.44:0/129878363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 msgr2=0x564ac975d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 0x564ac975d350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00ce3b0 tx=0x7fd5e00d46f0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 -- 172.21.15.44:0/129878363 shutdown_connections 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac97443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac9748ac0 0x564ac975d350 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 --2- 172.21.15.44:0/129878363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac97481a0 0x564ac9750db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 -- 172.21.15.44:0/129878363 >> 172.21.15.44:0/129878363 conn(0x564ac97428d0 msgr2=0x564ac9743ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 -- 172.21.15.44:0/129878363 shutdown_connections 2024-04-20T11:30:08.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.317+0000 7fd60122a740 1 -- 172.21.15.44:0/129878363 wait complete. 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.318+0000 7fd60122a740 1 -- start start 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.318+0000 7fd60122a740 1 Processor -- start 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.318+0000 7fd60122a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.318+0000 7fd60122a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.318+0000 7fd60122a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd60122a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81b1400 con 0x564ac97481a0 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd60122a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81b0d30 con 0x564ac97516a0 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd60122a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81adf80 con 0x564ac9748ac0 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd60122a740 10 client.0 _ll_register_callbacks cb 0x7fd5e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43750/0 (socket says 172.21.15.44:43750) 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7400640 1 -- 172.21.15.44:0/3235565586 learned_addr learned my addr 172.21.15.44:0/3235565586 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7e00640 1 -- 172.21.15.44:0/3235565586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 msgr2=0x564ac974b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7e00640 1 -- 172.21.15.44:0/3235565586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 msgr2=0x564ac974b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7e00640 1 -- 172.21.15.44:0/3235565586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac97c6f10 con 0x564ac97516a0 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.319+0000 7fd5f7400640 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5f7e00640 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974bd00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0004af0 tx=0x7fd5f00346c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a279c61e47975285 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5e5600640 1 -- 172.21.15.44:0/3235565586 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0034d40 con 0x564ac97516a0 2024-04-20T11:30:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5e5600640 1 -- 172.21.15.44:0/3235565586 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8499+0+0 (secure 0 0 0) 0x7fd5f004b450 con 0x564ac97516a0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e009d250 con 0x564ac97516a0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5e5600640 1 -- 172.21.15.44:0/3235565586 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd5f004b650 con 0x564ac97516a0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd60122a740 10 client.27563 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564ac97c8100 con 0x564ac97516a0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5e5600640 1 -- 172.21.15.44:0/3235565586 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd5e009d250 con 0x564ac97516a0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.320+0000 7fd5e5600640 1 -- 172.21.15.44:0/3235565586 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd5f0040920 con 0x564ac97516a0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd60122a740 10 client.27563 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd60122a740 10 client.27563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd60122a740 20 client.27563 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd60122a740 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97c87d0 0x564ac97cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd60122a740 4 client.27563 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ac96b59e0 con 0x564ac97c87d0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd5f7400640 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97c87d0 0x564ac97cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd5f7400640 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97c87d0 0x564ac97cabf0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0058358540b497 server_cookie=f4142e8661a812e1 in_seq=0 out_seq=0 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.321+0000 7fd5e5600640 10 client.27563 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd5e5600640 1 -- 172.21.15.44:0/3235565586 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564ac96b59e0 con 0x564ac97c87d0 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd5e5600640 10 client.27563 handle_command_reply: tid=0 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 client.27563 shutdown 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 msgr2=0x564ac974bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974bd00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0004af0 tx=0x7fd5f00346c0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 shutdown_connections 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x564ac97c87d0 0x564ac97cabf0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ac97516a0 0x564ac974bd00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd5e5600640 0 client.27563 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ac9748ac0 0x564ac974b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 --2- 172.21.15.44:0/3235565586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564ac97481a0 0x564ac974b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 >> 172.21.15.44:0/3235565586 conn(0x564ac97428d0 msgr2=0x564ac974eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 shutdown_connections 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 1 -- 172.21.15.44:0/3235565586 wait complete. 2024-04-20T11:30:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.322+0000 7fd60122a740 20 client.27563 trim_cache size 0 max 16384 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "6bb6bf9d-0cc4-4f7d-895b-f33a17fc27ee": { 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "6bb6bf9d-0cc4-4f7d-895b-f33a17fc27ee", 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:30:08.327 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5dc0a3d20 msgr2=0x7fd5dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5dc0a3d20 0x7fd5dc0a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0006010 tx=0x7fd5f0004a40 comp rx=0 tx=0).stop 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 msgr2=0x7fd5f81a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f81a0c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e000ee80 comp rx=0 tx=0).stop 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 shutdown_connections 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5f819a8f0 0x7fd5f819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd5dc0a3d20 0x7fd5dc0a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f8109480 0x7fd5f81a0c80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 --2- 172.21.15.44:0/2454785855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5f81043b0 0x7fd5f81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 >> 172.21.15.44:0/2454785855 conn(0x7fd5f80ffbd0 msgr2=0x7fd5f81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 shutdown_connections 2024-04-20T11:30:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:08.323+0000 7fd5fe400640 1 -- 172.21.15.44:0/2454785855 wait complete. 2024-04-20T11:30:08.342 INFO:tasks.cephfs.filesystem:scrub status for tag:6bb6bf9d-0cc4-4f7d-895b-f33a17fc27ee - {'path': '/', 'tag': '6bb6bf9d-0cc4-4f7d-895b-f33a17fc27ee', 'options': 'recursive,force'} 2024-04-20T11:30:08.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:08 smithi078 ceph-mon[21464]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T11:30:08.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:08 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:30:08.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:08 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:08.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:08 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:30:08.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:08 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:08 smithi044 ceph-mon[16855]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T11:30:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:08 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:30:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:08 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:08 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:30:08.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:08 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:08.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:08 smithi165 ceph-mon[21975]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T11:30:08.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:08 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:30:08.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:08 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:08.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:08 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:30:08.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:08 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:09.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:09 smithi078 ceph-mon[21464]: pgmap v232: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:30:09.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:09 smithi044 ceph-mon[16855]: pgmap v232: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:30:09.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:09 smithi165 ceph-mon[21975]: pgmap v232: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:30:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:11 smithi078 ceph-mon[21464]: pgmap v233: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:30:11.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:11 smithi044 ceph-mon[16855]: pgmap v233: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:30:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:11 smithi165 ceph-mon[21975]: pgmap v233: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:30:12.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:12 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:30:12.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:12 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:30:12.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:12 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:30:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:13 smithi078 ceph-mon[21464]: pgmap v234: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:13 smithi078 ceph-mon[21464]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T11:30:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:13 smithi044 ceph-mon[16855]: pgmap v234: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:13 smithi044 ceph-mon[16855]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T11:30:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:13 smithi165 ceph-mon[21975]: pgmap v234: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:13 smithi165 ceph-mon[21975]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T11:30:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:15 smithi078 ceph-mon[21464]: pgmap v236: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-20T11:30:15.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:15 smithi044 ceph-mon[16855]: pgmap v236: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-20T11:30:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:15 smithi165 ceph-mon[21975]: pgmap v236: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-20T11:30:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:17 smithi078 ceph-mon[21464]: pgmap v237: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 117 B/s wr, 6 op/s 2024-04-20T11:30:17.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:17 smithi044 ceph-mon[16855]: pgmap v237: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 117 B/s wr, 6 op/s 2024-04-20T11:30:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:17 smithi165 ceph-mon[21975]: pgmap v237: 97 pgs: 97 active+clean; 267 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 117 B/s wr, 6 op/s 2024-04-20T11:30:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:19 smithi078 ceph-mon[21464]: pgmap v238: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:19 smithi044 ceph-mon[16855]: pgmap v238: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:19 smithi165 ceph-mon[21975]: pgmap v238: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:21 smithi078 ceph-mon[21464]: pgmap v239: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:21.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:21 smithi044 ceph-mon[16855]: pgmap v239: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:21 smithi165 ceph-mon[21975]: pgmap v239: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:30:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:23 smithi078 ceph-mon[21464]: pgmap v240: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:30:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:23 smithi044 ceph-mon[16855]: pgmap v240: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:30:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:23 smithi165 ceph-mon[21975]: pgmap v240: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:30:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:25 smithi078 ceph-mon[21464]: pgmap v241: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:30:25.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:25 smithi044 ceph-mon[16855]: pgmap v241: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:30:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:25 smithi165 ceph-mon[21975]: pgmap v241: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:30:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:27 smithi078 ceph-mon[21464]: pgmap v242: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:30:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:27 smithi044 ceph-mon[16855]: pgmap v242: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:30:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:27 smithi165 ceph-mon[21975]: pgmap v242: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:30:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:29 smithi078 ceph-mon[21464]: pgmap v243: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:30:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:29 smithi044 ceph-mon[16855]: pgmap v243: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:30:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:29 smithi165 ceph-mon[21975]: pgmap v243: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:30:31.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:31 smithi044 ceph-mon[16855]: pgmap v244: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:31 smithi165 ceph-mon[21975]: pgmap v244: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:32.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:31 smithi078 ceph-mon[21464]: pgmap v244: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:33 smithi165 ceph-mon[21975]: pgmap v245: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:34.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:33 smithi078 ceph-mon[21464]: pgmap v245: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:34.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:33 smithi044 ceph-mon[16855]: pgmap v245: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:35 smithi165 ceph-mon[21975]: pgmap v246: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:35.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:35 smithi044 ceph-mon[16855]: pgmap v246: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:36.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:35 smithi078 ceph-mon[21464]: pgmap v246: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:37 smithi165 ceph-mon[21975]: pgmap v247: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:37.989 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:37 smithi078 ceph-mon[21464]: pgmap v247: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:38.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:37 smithi044 ceph-mon[16855]: pgmap v247: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:38.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:30:38.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 -- 172.21.15.44:0/2764137507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc104d60 msgr2=0x7fcefc108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 --2- 172.21.15.44:0/2764137507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc104d60 0x7fcefc108da0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009900 tx=0x7fcee802bc40 comp rx=0 tx=0).stop 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 -- 172.21.15.44:0/2764137507 shutdown_connections 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 --2- 172.21.15.44:0/2764137507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcefc109480 0x7fcefc111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 --2- 172.21.15.44:0/2764137507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc104d60 0x7fcefc108da0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 --2- 172.21.15.44:0/2764137507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 0x7fcefc104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 -- 172.21.15.44:0/2764137507 >> 172.21.15.44:0/2764137507 conn(0x7fcefc0ffbd0 msgr2=0x7fcefc101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 -- 172.21.15.44:0/2764137507 shutdown_connections 2024-04-20T11:30:38.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.572+0000 7fcf03800640 1 -- 172.21.15.44:0/2764137507 wait complete. 2024-04-20T11:30:38.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.573+0000 7fcf03800640 1 Processor -- start 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.573+0000 7fcf03800640 1 -- start start 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf03800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 0x7fcefc1a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf03800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcefc109480 0x7fcefc1a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf03800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 0x7fcefc19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf03800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc114650 con 0x7fcefc109480 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf03800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1144d0 con 0x7fcefc19aa70 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf03800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1147d0 con 0x7fcefc1043b0 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf01800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 0x7fcefc19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcefc109480 0x7fcefc1a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf00e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 0x7fcefc1a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf01800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 0x7fcefc19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59200/0 (socket says 172.21.15.44:59200) 2024-04-20T11:30:38.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.574+0000 7fcf01800640 1 -- 172.21.15.44:0/3319343201 learned_addr learned my addr 172.21.15.44:0/3319343201 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:38.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf00e00640 1 -- 172.21.15.44:0/3319343201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 msgr2=0x7fcefc19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf00e00640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 0x7fcefc19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf00e00640 1 -- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcefc109480 msgr2=0x7fcefc1a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf00e00640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcefc109480 0x7fcefc1a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf00e00640 1 -- 172.21.15.44:0/3319343201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc19b590 con 0x7fcefc1043b0 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf01800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 0x7fcefc19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf00e00640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 0x7fcefc1a0870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcef000e9f0 tx=0x7fcef000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcef9600640 1 -- 172.21.15.44:0/3319343201 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef000ccc0 con 0x7fcefc1043b0 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc19b880 con 0x7fcefc1043b0 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcef9600640 1 -- 172.21.15.44:0/3319343201 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcef0005020 con 0x7fcefc1043b0 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcef9600640 1 -- 172.21.15.44:0/3319343201 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0004b40 con 0x7fcefc1043b0 2024-04-20T11:30:38.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.575+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1a7a30 con 0x7fcefc1043b0 2024-04-20T11:30:38.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.578+0000 7fcf0642f740 1 Processor -- start 2024-04-20T11:30:38.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 -- start start 2024-04-20T11:30:38.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcef9600640 1 -- 172.21.15.44:0/3319343201 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcef0004ce0 con 0x7fcefc1043b0 2024-04-20T11:30:38.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9224580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b922fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5600b918a150 con 0x5600b922d6a0 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600b9130370 con 0x5600b9224ac0 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf0642f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5600b9186110 con 0x5600b92241a0 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf00e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf00e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59202/0 (socket says 172.21.15.44:59202) 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b922fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf00e00640 1 -- 172.21.15.44:0/2329670889 learned_addr learned my addr 172.21.15.44:0/2329670889 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.579+0000 7fcf01800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9224580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fcf00e00640 1 -- 172.21.15.44:0/2329670889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 msgr2=0x5600b9224580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fcf00e00640 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9224580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fcf00e00640 1 -- 172.21.15.44:0/2329670889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 msgr2=0x5600b922fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fcf00e00640 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b922fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fcf00e00640 1 -- 172.21.15.44:0/2329670889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b8f667d0 con 0x5600b9224ac0 2024-04-20T11:30:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fcf00e00640 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922d160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcef00cde30 tx=0x7fcef00d6bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad21c69cd4687f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fceee000640 1 -- 172.21.15.44:0/2329670889 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00df070 con 0x5600b9224ac0 2024-04-20T11:30:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fceee000640 1 -- 172.21.15.44:0/2329670889 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcef00d6e40 con 0x5600b9224ac0 2024-04-20T11:30:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.580+0000 7fceee000640 1 -- 172.21.15.44:0/2329670889 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00db420 con 0x5600b9224ac0 2024-04-20T11:30:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcef9600640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcee40a3d20 0x7fcee40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 -- 172.21.15.44:0/2329670889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 msgr2=0x5600b922d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922d160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcef00cde30 tx=0x7fcef00d6bf0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcef9600640 1 -- 172.21.15.44:0/3319343201 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7fcef00d07c0 con 0x7fcefc1043b0 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 -- 172.21.15.44:0/2329670889 shutdown_connections 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b922fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf01800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcee40a3d20 0x7fcee40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922d160 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2329670889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9224580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 -- 172.21.15.44:0/2329670889 >> 172.21.15.44:0/2329670889 conn(0x5600b921e8d0 msgr2=0x5600b921ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 -- 172.21.15.44:0/2329670889 shutdown_connections 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.581+0000 7fcf0642f740 1 -- 172.21.15.44:0/2329670889 wait complete. 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.582+0000 7fcf0642f740 1 -- start start 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.582+0000 7fcf01800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcee40a3d20 0x7fcee40a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcef40049d0 tx=0x7fcef4009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.582+0000 7fcf0642f740 1 Processor -- start 2024-04-20T11:30:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.582+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b92241a0 0x5600b922a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b922b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf0642f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5600b9230490 con 0x5600b92241a0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf0642f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600b9143740 con 0x5600b9224ac0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf0642f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5600b9230610 con 0x5600b922d6a0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf00e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b92241a0 0x5600b922a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf0642f740 10 client.0 _ll_register_callbacks cb 0x5600b913eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59208/0 (socket says 172.21.15.44:59208) 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 -- 172.21.15.44:0/2962616794 learned_addr learned my addr 172.21.15.44:0/2962616794 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf01800640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b922b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 -- 172.21.15.44:0/2962616794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 msgr2=0x5600b922b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b922b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 -- 172.21.15.44:0/2962616794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b92241a0 msgr2=0x5600b922a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b92241a0 0x5600b922a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcefbe00640 1 -- 172.21.15.44:0/2962616794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b8e9c3f0 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf01800640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b922b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.583+0000 7fcf00e00640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b92241a0 0x5600b922a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcefbe00640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922abc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcefc06b5e0 tx=0x7fcee802dd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab1133fc4095b781 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcee2a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcee2a00640 1 -- 172.21.15.44:0/2962616794 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8044070 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcee2a00640 1 -- 172.21.15.44:0/2962616794 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7fcee8004310 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5600b9197b70 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcee2a00640 1 -- 172.21.15.44:0/2962616794 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcee8050050 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcf0642f740 10 client.27575 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5600b8e5c0f0 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.584+0000 7fcee2a00640 1 -- 172.21.15.44:0/2962616794 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x5600b9197b70 con 0x5600b9224ac0 2024-04-20T11:30:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcee2a00640 1 -- 172.21.15.44:0/2962616794 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fcee8004bb0 con 0x5600b9224ac0 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf0642f740 10 client.27575 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf0642f740 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf0642f740 20 client.27575 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b923fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf0642f740 4 client.27575 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600b91919e0 con 0x5600b923d920 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.585+0000 7fcf00e00640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b923fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.586+0000 7fcf00e00640 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b923fd70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923a822b358802f7 server_cookie=b2e2e604514d8f7c in_seq=0 out_seq=0 2024-04-20T11:30:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.586+0000 7fcee2a00640 10 client.27575 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:38.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.586+0000 7fcee2a00640 1 -- 172.21.15.44:0/2962616794 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600b91919e0 con 0x5600b923d920 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.586+0000 7fcee2a00640 10 client.27575 handle_command_reply: tid=0 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.586+0000 7fcf0642f740 1 client.27575 shutdown 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 msgr2=0x5600b922abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922abc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcefc06b5e0 tx=0x7fcee802dd60 comp rx=0 tx=0).stop 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 shutdown_connections 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b923fd70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b922b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcee2a00640 0 client.27575 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b922abc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 --2- 172.21.15.44:0/2962616794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b92241a0 0x5600b922a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 >> 172.21.15.44:0/2962616794 conn(0x5600b921e8d0 msgr2=0x5600b9220620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 shutdown_connections 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 1 -- 172.21.15.44:0/2962616794 wait complete. 2024-04-20T11:30:38.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.587+0000 7fcf0642f740 20 client.27575 trim_cache size 0 max 16384 2024-04-20T11:30:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.606+0000 7fcf0642f740 1 Processor -- start 2024-04-20T11:30:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.606+0000 7fcf0642f740 1 -- start start 2024-04-20T11:30:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b92241a0 0x5600b922cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 0x5600b92295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b9239350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf0642f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1aff90 con 0x5600b9224ac0 2024-04-20T11:30:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf0642f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b1ad0 con 0x5600b92241a0 2024-04-20T11:30:38.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf0642f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b0660 con 0x5600b922d6a0 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 0x5600b92295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 0x5600b92295c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37752/0 (socket says 172.21.15.44:37752) 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcefbe00640 1 -- 172.21.15.44:0/3992446642 learned_addr learned my addr 172.21.15.44:0/3992446642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf01800640 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b9239350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.607+0000 7fcf00e00640 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b92241a0 0x5600b922cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcefbe00640 1 -- 172.21.15.44:0/3992446642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 msgr2=0x5600b9239350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcefbe00640 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b9239350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcefbe00640 1 -- 172.21.15.44:0/3992446642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b92241a0 msgr2=0x5600b922cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcefbe00640 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b92241a0 0x5600b922cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcefbe00640 1 -- 172.21.15.44:0/3992446642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b921fb00 con 0x5600b9224ac0 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcf00e00640 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b92241a0 0x5600b922cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fcefbe00640 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 0x5600b92295c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcee80096c0 tx=0x7fcee8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce07410c4465554a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fceed600640 1 -- 172.21.15.44:0/3992446642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee804d070 con 0x5600b9224ac0 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fceed600640 1 -- 172.21.15.44:0/3992446642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcee8003760 con 0x5600b9224ac0 2024-04-20T11:30:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.608+0000 7fceed600640 1 -- 172.21.15.44:0/3992446642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8044070 con 0x5600b9224ac0 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 -- 172.21.15.44:0/3992446642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 msgr2=0x5600b92295c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 0x5600b92295c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcee80096c0 tx=0x7fcee8001e70 comp rx=0 tx=0).stop 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 -- 172.21.15.44:0/3992446642 shutdown_connections 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b922d6a0 0x5600b9239350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b9224ac0 0x5600b92295c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 --2- 172.21.15.44:0/3992446642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b92241a0 0x5600b922cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 -- 172.21.15.44:0/3992446642 >> 172.21.15.44:0/3992446642 conn(0x5600b921e8d0 msgr2=0x5600b921d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 -- 172.21.15.44:0/3992446642 shutdown_connections 2024-04-20T11:30:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.609+0000 7fcf0642f740 1 -- 172.21.15.44:0/3992446642 wait complete. 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 -- start start 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 Processor -- start 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9227420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b9227960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b9227ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b1400 con 0x5600b922d6a0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b0d30 con 0x5600b9224ac0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1adf80 con 0x5600b92241a0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.610+0000 7fcf0642f740 10 client.0 _ll_register_callbacks cb 0x7fcee8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9227420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf01800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b9227ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9227420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59174/0 (socket says 172.21.15.44:59174) 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 -- 172.21.15.44:0/445870467 learned_addr learned my addr 172.21.15.44:0/445870467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b9227960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 -- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 msgr2=0x5600b9227960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b9227960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 -- 172.21.15.44:0/445870467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 msgr2=0x5600b9227ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b9227ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 -- 172.21.15.44:0/445870467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b92a2f80 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf00e00640 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9227420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcef000e9c0 tx=0x7fcef00045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7199a6e1f6d1a019 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcee2a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcee2a00640 1 -- 172.21.15.44:0/445870467 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef001c030 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcee2a00640 1 -- 172.21.15.44:0/445870467 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7fcef009aa40 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.611+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee8003760 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcee2a00640 1 -- 172.21.15.44:0/445870467 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcef00df070 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 10 client.27577 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5600b92a4290 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcee2a00640 1 -- 172.21.15.44:0/445870467 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcee8003760 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcee2a00640 1 -- 172.21.15.44:0/445870467 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fcef0098db0 con 0x5600b92241a0 2024-04-20T11:30:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 10 client.27577 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 20 client.27577 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b92a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 4 client.27577 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:30:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600b910d450 con 0x5600b923d920 2024-04-20T11:30:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.612+0000 7fcefbe00640 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b92a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcefbe00640 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b92a6530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628eb0a301a42575 server_cookie=b29769531aacaf5f in_seq=0 out_seq=0 2024-04-20T11:30:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcee2a00640 10 client.27577 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcee2a00640 1 -- 172.21.15.44:0/445870467 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600b910d450 con 0x5600b923d920 2024-04-20T11:30:38.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcee2a00640 10 client.27577 handle_command_reply: tid=0 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 client.27577 shutdown 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 msgr2=0x5600b9227420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9227420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcef000e9c0 tx=0x7fcef00045b0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 shutdown_connections 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5600b923d920 0x5600b92a6530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5600b922d6a0 0x5600b9227ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600b9224ac0 0x5600b9227960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 --2- 172.21.15.44:0/445870467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5600b92241a0 0x5600b9227420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcee2a00640 0 client.27577 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.613+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 >> 172.21.15.44:0/445870467 conn(0x5600b921e8d0 msgr2=0x5600b922a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 shutdown_connections 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf0642f740 1 -- 172.21.15.44:0/445870467 wait complete. 2024-04-20T11:30:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf0642f740 20 client.27577 trim_cache size 0 max 16384 2024-04-20T11:30:38.618 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:30:38.618 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:30:38.618 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:30:38.618 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:30:38.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcee40a3d20 msgr2=0x7fcee40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcee40a3d20 0x7fcee40a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcef40049d0 tx=0x7fcef4009500 comp rx=0 tx=0).stop 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 msgr2=0x7fcefc1a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 0x7fcefc1a0870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcef000e9f0 tx=0x7fcef000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 shutdown_connections 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcefc19aa70 0x7fcefc19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcee40a3d20 0x7fcee40a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcefc109480 0x7fcefc1a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 --2- 172.21.15.44:0/3319343201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc1043b0 0x7fcefc1a0870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.614+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 >> 172.21.15.44:0/3319343201 conn(0x7fcefc0ffbd0 msgr2=0x7fcefc100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.615+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 shutdown_connections 2024-04-20T11:30:38.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.615+0000 7fcf03800640 1 -- 172.21.15.44:0/3319343201 wait complete. 2024-04-20T11:30:38.634 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:30:38.634 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:30:38.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 -- 172.21.15.44:0/2716245728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa174109290 msgr2=0x7fa174110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 --2- 172.21.15.44:0/2716245728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa174109290 0x7fa174110e60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa170009ca0 tx=0x7fa17002c100 comp rx=0 tx=0).stop 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 -- 172.21.15.44:0/2716245728 shutdown_connections 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 --2- 172.21.15.44:0/2716245728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa174109290 0x7fa174110e60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 --2- 172.21.15.44:0/2716245728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174104b70 0x7fa174108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 --2- 172.21.15.44:0/2716245728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1741041c0 0x7fa1741045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.838+0000 7fa17b200640 1 -- 172.21.15.44:0/2716245728 >> 172.21.15.44:0/2716245728 conn(0x7fa1740ffa00 msgr2=0x7fa174101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.839+0000 7fa17b200640 1 -- 172.21.15.44:0/2716245728 shutdown_connections 2024-04-20T11:30:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.839+0000 7fa17b200640 1 -- 172.21.15.44:0/2716245728 wait complete. 2024-04-20T11:30:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 Processor -- start 2024-04-20T11:30:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 -- start start 2024-04-20T11:30:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1741041c0 0x7fa1741a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174109290 0x7fa1741a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa17419a6e0 0x7fa17419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114460 con 0x7fa174109290 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741142e0 con 0x7fa17419a6e0 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa17b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741145e0 con 0x7fa1741041c0 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa179400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174109290 0x7fa1741a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa179400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174109290 0x7fa1741a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37786/0 (socket says 172.21.15.44:37786) 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.840+0000 7fa179400640 1 -- 172.21.15.44:0/2527771509 learned_addr learned my addr 172.21.15.44:0/2527771509 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa17419a6e0 0x7fa17419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa179e00640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1741041c0 0x7fa1741a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 -- 172.21.15.44:0/2527771509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1741041c0 msgr2=0x7fa1741a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1741041c0 0x7fa1741a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 -- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174109290 msgr2=0x7fa1741a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174109290 0x7fa1741a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 -- 172.21.15.44:0/2527771509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17419b2e0 con 0x7fa17419a6e0 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa17a800640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa17419a6e0 0x7fa17419ab60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa17002c710 tx=0x7fa1700346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.841+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170039040 con 0x7fa17419a6e0 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.842+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17419b570 con 0x7fa17419a6e0 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.842+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa170044070 con 0x7fa17419a6e0 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.842+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170004510 con 0x7fa17419a6e0 2024-04-20T11:30:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.842+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1741a76b0 con 0x7fa17419a6e0 2024-04-20T11:30:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.843+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa174105690 con 0x7fa17419a6e0 2024-04-20T11:30:38.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.845+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa1700046b0 con 0x7fa17419a6e0 2024-04-20T11:30:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.846+0000 7fa16ea00640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa15c0a3d20 0x7fa15c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.847+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7fa1700f31b0 con 0x7fa17419a6e0 2024-04-20T11:30:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.847+0000 7fa179e00640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa15c0a3d20 0x7fa15c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.847+0000 7fa179e00640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa15c0a3d20 0x7fa15c0a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa168005d00 tx=0x7fa16800b410 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:38.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:38.853+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa170074020 con 0x7fa17419a6e0 2024-04-20T11:30:39.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.101+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa174104b70 con 0x7fa17419a6e0 2024-04-20T11:30:39.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.102+0000 7fa16ea00640 1 -- 172.21.15.44:0/2527771509 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fa1700bf940 con 0x7fa17419a6e0 2024-04-20T11:30:39.102 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:30:39.103 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:30:39.103 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:30:39.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.105+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa15c0a3d20 msgr2=0x7fa15c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.105+0000 7fa17b200640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa15c0a3d20 0x7fa15c0a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa168005d00 tx=0x7fa16800b410 comp rx=0 tx=0).stop 2024-04-20T11:30:39.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.105+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa17419a6e0 msgr2=0x7fa17419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.105+0000 7fa17b200640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa17419a6e0 0x7fa17419ab60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa17002c710 tx=0x7fa1700346f0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 shutdown_connections 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa17419a6e0 0x7fa17419ab60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa15c0a3d20 0x7fa15c0a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa174109290 0x7fa1741a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 --2- 172.21.15.44:0/2527771509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1741041c0 0x7fa1741a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 >> 172.21.15.44:0/2527771509 conn(0x7fa1740ffa00 msgr2=0x7fa174100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 shutdown_connections 2024-04-20T11:30:39.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.106+0000 7fa17b200640 1 -- 172.21.15.44:0/2527771509 wait complete. 2024-04-20T11:30:39.124 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:30:39.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.344+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4213441504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee010c3e0 msgr2=0x7f8ee010e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.344+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4213441504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee010c3e0 0x7f8ee010e880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009900 tx=0x7f8ed802bcc0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4213441504 shutdown_connections 2024-04-20T11:30:39.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4213441504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee010c3e0 0x7f8ee010e880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4213441504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee01038e0 0x7f8ee010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4213441504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ee0102fc0 0x7f8ee01033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4213441504 >> 172.21.15.44:0/4213441504 conn(0x7f8ee00fc7e0 msgr2=0x7f8ee00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4213441504 shutdown_connections 2024-04-20T11:30:39.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.345+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4213441504 wait complete. 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 Processor -- start 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 -- start start 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ee0102fc0 0x7f8ee01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee01038e0 0x7f8ee01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 0x7f8ee019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0112300 con 0x7f8ee0102fc0 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0112180 con 0x7f8ee010c3e0 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.346+0000 7f8ee4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0112480 con 0x7f8ee01038e0 2024-04-20T11:30:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ee0102fc0 0x7f8ee01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 0x7f8ee019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee01038e0 0x7f8ee01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 0x7f8ee019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59248/0 (socket says 172.21.15.44:59248) 2024-04-20T11:30:39.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 -- 172.21.15.44:0/4108199397 learned_addr learned my addr 172.21.15.44:0/4108199397 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 -- 172.21.15.44:0/4108199397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee01038e0 msgr2=0x7f8ee01a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee01038e0 0x7f8ee01a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 -- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ee0102fc0 msgr2=0x7f8ee01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ee0102fc0 0x7f8ee01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 -- 172.21.15.44:0/4108199397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee019af60 con 0x7f8ee010c3e0 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8edf200640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 0x7f8ee019a790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005ab0 tx=0x7f8ed802df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.347+0000 7f8ed7400640 1 -- 172.21.15.44:0/4108199397 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8043020 con 0x7f8ee010c3e0 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.348+0000 7f8ed7400640 1 -- 172.21.15.44:0/4108199397 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ed8004e50 con 0x7f8ee010c3e0 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.348+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee019b1f0 con 0x7f8ee010c3e0 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.348+0000 7f8ed7400640 1 -- 172.21.15.44:0/4108199397 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8003970 con 0x7f8ee010c3e0 2024-04-20T11:30:39.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.348+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee01a7470 con 0x7f8ee010c3e0 2024-04-20T11:30:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.351+0000 7f8ee81ae740 1 Processor -- start 2024-04-20T11:30:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.351+0000 7f8ed7400640 1 -- 172.21.15.44:0/4108199397 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8ed8046020 con 0x7f8ee010c3e0 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.351+0000 7f8ee81ae740 1 -- start start 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.351+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b46c114050 con 0x55b46c1afa90 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b46c0ba490 con 0x55b46c1b8670 2024-04-20T11:30:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b46c110010 con 0x55b46c1af170 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8edde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37802/0 (socket says 172.21.15.44:37802) 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 -- 172.21.15.44:0/1455094320 learned_addr learned my addr 172.21.15.44:0/1455094320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8edde00640 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 -- 172.21.15.44:0/1455094320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 msgr2=0x55b46c1af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 -- 172.21.15.44:0/1455094320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 msgr2=0x55b46c1baa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1baa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.352+0000 7f8ede800640 1 -- 172.21.15.44:0/1455094320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46c0c2070 con 0x55b46c1afa90 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8ede800640 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b8130 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00114f0 tx=0x7f8ed00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61ecca567a3a6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8ed7400640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ec40a3d20 0x7f8ec40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8ecb400640 1 -- 172.21.15.44:0/1455094320 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed001e070 con 0x55b46c1afa90 2024-04-20T11:30:39.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8ecb400640 1 -- 172.21.15.44:0/1455094320 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ed0011310 con 0x55b46c1afa90 2024-04-20T11:30:39.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8ecb400640 1 -- 172.21.15.44:0/1455094320 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0019aa0 con 0x55b46c1afa90 2024-04-20T11:30:39.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8ed7400640 1 -- 172.21.15.44:0/4108199397 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f8ed80f2660 con 0x7f8ee010c3e0 2024-04-20T11:30:39.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.353+0000 7f8edde00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ec40a3d20 0x7f8ec40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 -- 172.21.15.44:0/1455094320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 msgr2=0x55b46c1b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b8130 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00114f0 tx=0x7f8ed00119b0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 -- 172.21.15.44:0/1455094320 shutdown_connections 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b8130 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/1455094320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 -- 172.21.15.44:0/1455094320 >> 172.21.15.44:0/1455094320 conn(0x55b46c1a98a0 msgr2=0x55b46c1a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 -- 172.21.15.44:0/1455094320 shutdown_connections 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8edde00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ec40a3d20 0x7f8ec40a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b46c1aaa30 tx=0x7f8ecc0046c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.354+0000 7f8ee81ae740 1 -- 172.21.15.44:0/1455094320 wait complete. 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 -- start start 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 Processor -- start 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b46c1bb460 con 0x55b46c1afa90 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b46c0cd860 con 0x55b46c1b8670 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.355+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b46c1bb5e0 con 0x55b46c1af170 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ee81ae740 10 client.0 _ll_register_callbacks cb 0x55b46c09bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8edf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8edde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59250/0 (socket says 172.21.15.44:59250) 2024-04-20T11:30:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8edf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59264/0 (socket says 172.21.15.44:59264) 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 -- 172.21.15.44:0/2477515271 learned_addr learned my addr 172.21.15.44:0/2477515271 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 -- 172.21.15.44:0/2477515271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 msgr2=0x55b46c1b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 -- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 msgr2=0x55b46c1b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8ede800640 1 -- 172.21.15.44:0/2477515271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46c0d1950 con 0x55b46c1af170 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.356+0000 7f8edf200640 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ede800640 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1b56e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0011eb0 tx=0x7f8ed0004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83f6e76b7df9d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ebfe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ebfe00640 1 -- 172.21.15.44:0/2477515271 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0027020 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ebfe00640 1 -- 172.21.15.44:0/2477515271 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f8ed00199b0 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed0011310 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ebfe00640 1 -- 172.21.15.44:0/2477515271 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8ed001e040 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ee81ae740 10 client.27595 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.357+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b46c09f820 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ebfe00640 1 -- 172.21.15.44:0/2477515271 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ed0011310 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ebfe00640 1 -- 172.21.15.44:0/2477515271 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f8ed000f740 con 0x55b46c1af170 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ee81ae740 10 client.27595 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ee81ae740 10 client.27595 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ee81ae740 20 client.27595 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c1c85f0 0x55b46c1ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ee81ae740 4 client.27595 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b46c121a70 con 0x55b46c1c85f0 2024-04-20T11:30:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.358+0000 7f8edde00640 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c1c85f0 0x55b46c1ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.359+0000 7f8edde00640 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c1c85f0 0x55b46c1ca9e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cccef6e81c2f79a2 server_cookie=1c836524fabd8a98 in_seq=0 out_seq=0 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.359+0000 7f8ebfe00640 10 client.27595 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ebfe00640 1 -- 172.21.15.44:0/2477515271 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b46c121a70 con 0x55b46c1c85f0 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ebfe00640 10 client.27595 handle_command_reply: tid=0 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 client.27595 shutdown 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 msgr2=0x55b46c1b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1b56e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0011eb0 tx=0x7f8ed0004dd0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 shutdown_connections 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c1c85f0 0x55b46c1ca9e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1afa90 0x55b46c1b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/2477515271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1af170 0x55b46c1b56e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 >> 172.21.15.44:0/2477515271 conn(0x55b46c1a98a0 msgr2=0x55b46c1ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ebfe00640 0 client.27595 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:39.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 shutdown_connections 2024-04-20T11:30:39.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 1 -- 172.21.15.44:0/2477515271 wait complete. 2024-04-20T11:30:39.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.360+0000 7f8ee81ae740 20 client.27595 trim_cache size 0 max 16384 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.375+0000 7f8ee81ae740 1 Processor -- start 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.375+0000 7f8ee81ae740 1 -- start start 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01afcb0 con 0x55b46c1af170 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b17f0 con 0x55b46c1b8670 2024-04-20T11:30:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b0380 con 0x55b46c1afa90 2024-04-20T11:30:39.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37826/0 (socket says 172.21.15.44:37826) 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 -- 172.21.15.44:0/3881481973 learned_addr learned my addr 172.21.15.44:0/3881481973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8edde00640 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8edf200640 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 -- 172.21.15.44:0/3881481973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 msgr2=0x55b46c1c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 -- 172.21.15.44:0/3881481973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 msgr2=0x55b46c1afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.376+0000 7f8ede800640 1 -- 172.21.15.44:0/3881481973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46be263f0 con 0x55b46c1af170 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.377+0000 7f8ede800640 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1b7e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006010 tx=0x7f8ed00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1372c9bb6030f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.377+0000 7f8ecaa00640 1 -- 172.21.15.44:0/3881481973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0027040 con 0x55b46c1af170 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.377+0000 7f8ecaa00640 1 -- 172.21.15.44:0/3881481973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ed00194f0 con 0x55b46c1af170 2024-04-20T11:30:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.377+0000 7f8ecaa00640 1 -- 172.21.15.44:0/3881481973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004840 con 0x55b46c1af170 2024-04-20T11:30:39.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3881481973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 msgr2=0x55b46c1b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1b7e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006010 tx=0x7f8ed00196e0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3881481973 shutdown_connections 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c1afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3881481973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1b7e10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3881481973 >> 172.21.15.44:0/3881481973 conn(0x55b46c1a98a0 msgr2=0x55b46c1b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3881481973 shutdown_connections 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3881481973 wait complete. 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 -- start start 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.378+0000 7f8ee81ae740 1 Processor -- start 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c22ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b1120 con 0x55b46c1af170 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b0a50 con 0x55b46c1b8670 2024-04-20T11:30:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01adca0 con 0x55b46c1afa90 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ee81ae740 10 client.0 _ll_register_callbacks cb 0x7f8ed0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1bba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37834/0 (socket says 172.21.15.44:37834) 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 -- 172.21.15.44:0/3823502235 learned_addr learned my addr 172.21.15.44:0/3823502235 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8edf200640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c22ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8edde00640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 -- 172.21.15.44:0/3823502235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 msgr2=0x55b46c1bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1bbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 -- 172.21.15.44:0/3823502235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 msgr2=0x55b46c22ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c22ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.379+0000 7f8ede800640 1 -- 172.21.15.44:0/3823502235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46bef07d0 con 0x55b46c1af170 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8edf200640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c22ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ede800640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1bba40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0011eb0 tx=0x7f8ed000fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456baa18292ac0be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ebfe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ebfe00640 1 -- 172.21.15.44:0/3823502235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0019bc0 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ebfe00640 1 -- 172.21.15.44:0/3823502235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f8ed0019d60 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed00194f0 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ebfe00640 1 -- 172.21.15.44:0/3823502235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8ed0023460 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ee81ae740 10 client.17481 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.380+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b46c22e140 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ebfe00640 1 -- 172.21.15.44:0/3823502235 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ed00194f0 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ebfe00640 1 -- 172.21.15.44:0/3823502235 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f8ed00366a0 con 0x55b46c1af170 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ee81ae740 10 client.17481 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ee81ae740 10 client.17481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ee81ae740 20 client.17481 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c22e7a0 0x55b46c230bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ee81ae740 4 client.17481 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b46c11b8e0 con 0x55b46c22e7a0 2024-04-20T11:30:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.381+0000 7f8edde00640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c22e7a0 0x55b46c230bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8edde00640 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c22e7a0 0x55b46c230bc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25974d764a606ff7 server_cookie=26afe5003107dcfb in_seq=0 out_seq=0 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ebfe00640 10 client.17481 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ebfe00640 1 -- 172.21.15.44:0/3823502235 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b46c11b8e0 con 0x55b46c22e7a0 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ebfe00640 10 client.17481 handle_command_reply: tid=0 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 client.17481 shutdown 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 msgr2=0x55b46c1bba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1bba40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0011eb0 tx=0x7f8ed000fa60 comp rx=0 tx=0).stop 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 shutdown_connections 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55b46c22e7a0 0x55b46c230bc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b46c1b8670 0x55b46c22ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b46c1afa90 0x55b46c1bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ebfe00640 0 client.17481 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:30:39.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 --2- 172.21.15.44:0/3823502235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b46c1af170 0x55b46c1bba40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.382+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 >> 172.21.15.44:0/3823502235 conn(0x55b46c1a98a0 msgr2=0x55b46c1a80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 shutdown_connections 2024-04-20T11:30:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee81ae740 1 -- 172.21.15.44:0/3823502235 wait complete. 2024-04-20T11:30:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee81ae740 20 client.17481 trim_cache size 0 max 16384 2024-04-20T11:30:39.386 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ec40a3d20 msgr2=0x7f8ec40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ec40a3d20 0x7f8ec40a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b46c1aaa30 tx=0x7f8ecc0046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 msgr2=0x7f8ee019a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 0x7f8ee019a790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8005ab0 tx=0x7f8ed802df90 comp rx=0 tx=0).stop 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 shutdown_connections 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ee010c3e0 0x7f8ee019a790 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ee01038e0 0x7f8ee01a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f8ec40a3d20 0x7f8ec40a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 --2- 172.21.15.44:0/4108199397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ee0102fc0 0x7f8ee01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.383+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 >> 172.21.15.44:0/4108199397 conn(0x7f8ee00fc7e0 msgr2=0x7f8ee00fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.384+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 shutdown_connections 2024-04-20T11:30:39.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.384+0000 7f8ee4a00640 1 -- 172.21.15.44:0/4108199397 wait complete. 2024-04-20T11:30:39.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:30:39.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.612+0000 7efe0c400640 1 -- 172.21.15.44:0/358853367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104dc0 msgr2=0x7efe04108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.612+0000 7efe0c400640 1 --2- 172.21.15.44:0/358853367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104dc0 0x7efe04108e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009be0 tx=0x7efdf002bef0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 -- 172.21.15.44:0/358853367 shutdown_connections 2024-04-20T11:30:39.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 --2- 172.21.15.44:0/358853367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe041094e0 0x7efe041110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 --2- 172.21.15.44:0/358853367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104dc0 0x7efe04108e00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 --2- 172.21.15.44:0/358853367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe04104410 0x7efe041047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 -- 172.21.15.44:0/358853367 >> 172.21.15.44:0/358853367 conn(0x7efe040ffcf0 msgr2=0x7efe04102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 -- 172.21.15.44:0/358853367 shutdown_connections 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.613+0000 7efe0c400640 1 -- 172.21.15.44:0/358853367 wait complete. 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.614+0000 7efe0c400640 1 Processor -- start 2024-04-20T11:30:39.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.614+0000 7efe0c400640 1 -- start start 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0c400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 0x7efe041a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0c400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe041094e0 0x7efe041a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0c400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0419a9e0 0x7efe0419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0c400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe04114700 con 0x7efe0419a9e0 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0c400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe04114580 con 0x7efe04104410 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0c400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe04114880 con 0x7efe041094e0 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe041094e0 0x7efe041a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 0x7efe041a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe0a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0419a9e0 0x7efe0419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 0x7efe041a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59304/0 (socket says 172.21.15.44:59304) 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 -- 172.21.15.44:0/509387816 learned_addr learned my addr 172.21.15.44:0/509387816 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 -- 172.21.15.44:0/509387816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe041094e0 msgr2=0x7efe041a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe041094e0 0x7efe041a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 -- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0419a9e0 msgr2=0x7efe0419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0419a9e0 0x7efe0419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.615+0000 7efe09a00640 1 -- 172.21.15.44:0/509387816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0419b500 con 0x7efe04104410 2024-04-20T11:30:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.616+0000 7efe09a00640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 0x7efe041a0830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efdf400e9c0 tx=0x7efdf400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.616+0000 7efe02000640 1 -- 172.21.15.44:0/509387816 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf400cde0 con 0x7efe04104410 2024-04-20T11:30:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.616+0000 7efe02000640 1 -- 172.21.15.44:0/509387816 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdf40035f0 con 0x7efe04104410 2024-04-20T11:30:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.616+0000 7efe02000640 1 -- 172.21.15.44:0/509387816 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4010670 con 0x7efe04104410 2024-04-20T11:30:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.616+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0419b7f0 con 0x7efe04104410 2024-04-20T11:30:39.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.616+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041a7a90 con 0x7efe04104410 2024-04-20T11:30:39.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.619+0000 7efe0f056740 1 Processor -- start 2024-04-20T11:30:39.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.619+0000 7efe0f056740 1 -- start start 2024-04-20T11:30:39.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe02000640 1 -- 172.21.15.44:0/509387816 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efdf40037a0 con 0x7efe04104410 2024-04-20T11:30:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0f056740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 0x55cf6b87a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0f056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b882f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0f056740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b8858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0f056740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cf6b7df050 con 0x55cf6b8834b0 2024-04-20T11:30:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0f056740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cf6b785490 con 0x55cf6b879fb0 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0f056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cf6b7db010 con 0x55cf6b87a8d0 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 0x55cf6b87a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 0x55cf6b87a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59320/0 (socket says 172.21.15.44:59320) 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe0a400640 1 -- 172.21.15.44:0/409534752 learned_addr learned my addr 172.21.15.44:0/409534752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe09000640 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b8858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.620+0000 7efe09a00640 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b882f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe0a400640 1 -- 172.21.15.44:0/409534752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 msgr2=0x55cf6b882f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe0a400640 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b882f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe0a400640 1 -- 172.21.15.44:0/409534752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 msgr2=0x55cf6b8858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe0a400640 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b8858a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe0a400640 1 -- 172.21.15.44:0/409534752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6b78d070 con 0x55cf6b879fb0 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe09000640 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b8858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efe0a400640 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 0x55cf6b87a390 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009c20 tx=0x7efdfc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e55c2ce07915ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efdfaa00640 1 -- 172.21.15.44:0/409534752 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc01e070 con 0x55cf6b879fb0 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efdfaa00640 1 -- 172.21.15.44:0/409534752 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdfc019b30 con 0x55cf6b879fb0 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.621+0000 7efdfaa00640 1 -- 172.21.15.44:0/409534752 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00eae0 con 0x55cf6b879fb0 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.622+0000 7efe0f056740 1 -- 172.21.15.44:0/409534752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 msgr2=0x55cf6b87a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.622+0000 7efe0f056740 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 0x55cf6b87a390 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009c20 tx=0x7efdfc011810 comp rx=0 tx=0).stop 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.622+0000 7efe02000640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efde80a3da0 0x7efde80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 -- 172.21.15.44:0/409534752 shutdown_connections 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b8858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b882f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 --2- 172.21.15.44:0/409534752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b879fb0 0x55cf6b87a390 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe02000640 1 -- 172.21.15.44:0/509387816 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7efdf40d0ac0 con 0x7efe04104410 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 -- 172.21.15.44:0/409534752 >> 172.21.15.44:0/409534752 conn(0x55cf6b8746e0 msgr2=0x55cf6b874ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0a400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efde80a3da0 0x7efde80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 -- 172.21.15.44:0/409534752 shutdown_connections 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 -- 172.21.15.44:0/409534752 wait complete. 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 -- start start 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.623+0000 7efe0f056740 1 Processor -- start 2024-04-20T11:30:39.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0a400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efde80a3da0 0x7efde80a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00a010 tx=0x7efdfc00e5a0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b880530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b880a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b8834b0 0x55cf6b880fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cf6b8862a0 con 0x55cf6b879fb0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cf6b798860 con 0x55cf6b87a8d0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cf6b886420 con 0x55cf6b8834b0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0f056740 10 client.0 _ll_register_callbacks cb 0x55cf6b766fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe09a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b880530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe0a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b8834b0 0x55cf6b880fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe09a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b880530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37872/0 (socket says 172.21.15.44:37872) 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.624+0000 7efe09a00640 1 -- 172.21.15.44:0/3829997704 learned_addr learned my addr 172.21.15.44:0/3829997704 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09000640 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b880a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09a00640 1 -- 172.21.15.44:0/3829997704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b8834b0 msgr2=0x55cf6b880fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09a00640 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b8834b0 0x55cf6b880fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09a00640 1 -- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 msgr2=0x55cf6b880a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09a00640 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b880a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09a00640 1 -- 172.21.15.44:0/3829997704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6b79c950 con 0x55cf6b879fb0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe09a00640 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b880530 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c6a0 tx=0x7efdf409c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557cb9a4890e3556 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efdef400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efdef400640 1 -- 172.21.15.44:0/3829997704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf40d7070 con 0x55cf6b879fb0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efdef400640 1 -- 172.21.15.44:0/3829997704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7efdf409be90 con 0x55cf6b879fb0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.625+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc019b30 con 0x55cf6b879fb0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efdef400640 1 -- 172.21.15.44:0/3829997704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efdf40ce670 con 0x55cf6b879fb0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 10 client.17493 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cf6b76a820 con 0x55cf6b879fb0 2024-04-20T11:30:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efdef400640 1 -- 172.21.15.44:0/3829997704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdfc019b30 con 0x55cf6b879fb0 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efdef400640 1 -- 172.21.15.44:0/3829997704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7efdf4004000 con 0x55cf6b879fb0 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 10 client.17493 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 10 client.17493 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 20 client.17493 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b893440 0x55cf6b895830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 4 client.17493 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.626+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6b7eca70 con 0x55cf6b893440 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.627+0000 7efe09000640 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b893440 0x55cf6b895830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.627+0000 7efe09000640 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b893440 0x55cf6b895830 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc9be5a08b94f4ac server_cookie=8116219ba39a9967 in_seq=0 out_seq=0 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.627+0000 7efdef400640 10 client.17493 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efdef400640 1 -- 172.21.15.44:0/3829997704 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf6b7eca70 con 0x55cf6b893440 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efdef400640 10 client.17493 handle_command_reply: tid=0 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 client.17493 shutdown 2024-04-20T11:30:39.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 msgr2=0x55cf6b880530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b880530 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c6a0 tx=0x7efdf409c6b0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 shutdown_connections 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b8834b0 0x55cf6b880fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b880a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b893440 0x55cf6b895830 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 --2- 172.21.15.44:0/3829997704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b880530 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 >> 172.21.15.44:0/3829997704 conn(0x55cf6b8746e0 msgr2=0x55cf6b876430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efdef400640 0 client.17493 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 shutdown_connections 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.628+0000 7efe0f056740 1 -- 172.21.15.44:0/3829997704 wait complete. 2024-04-20T11:30:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.629+0000 7efe0f056740 20 client.17493 trim_cache size 0 max 16384 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.643+0000 7efe0f056740 1 Processor -- start 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.643+0000 7efe0f056740 1 -- start start 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0f056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b879fb0 0x55cf6b882c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0f056740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b88f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0f056740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b880c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0f056740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe041aff90 con 0x55cf6b8834b0 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0f056740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b1ad0 con 0x55cf6b87a8d0 2024-04-20T11:30:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0f056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b0660 con 0x55cf6b879fb0 2024-04-20T11:30:39.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b88f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe0a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b880c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b88f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59330/0 (socket says 172.21.15.44:59330) 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 -- 172.21.15.44:0/150095447 learned_addr learned my addr 172.21.15.44:0/150095447 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09a00640 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b879fb0 0x55cf6b882c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 -- 172.21.15.44:0/150095447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b879fb0 msgr2=0x55cf6b882c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b879fb0 0x55cf6b882c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 -- 172.21.15.44:0/150095447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 msgr2=0x55cf6b880c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b880c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.644+0000 7efe09000640 1 -- 172.21.15.44:0/150095447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6b87a700 con 0x55cf6b87a8d0 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.645+0000 7efe09000640 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b88f160 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efdf002c4d0 tx=0x7efdf0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478942e6eaeadbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.645+0000 7efdfa000640 1 -- 172.21.15.44:0/150095447 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf004e070 con 0x55cf6b87a8d0 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.645+0000 7efdfa000640 1 -- 172.21.15.44:0/150095447 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdf0048b50 con 0x55cf6b87a8d0 2024-04-20T11:30:39.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.645+0000 7efdfa000640 1 -- 172.21.15.44:0/150095447 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf002caf0 con 0x55cf6b87a8d0 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 -- 172.21.15.44:0/150095447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 msgr2=0x55cf6b88f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b88f160 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efdf002c4d0 tx=0x7efdf0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 -- 172.21.15.44:0/150095447 shutdown_connections 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b8834b0 0x55cf6b880c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b87a8d0 0x55cf6b88f160 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 --2- 172.21.15.44:0/150095447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b879fb0 0x55cf6b882c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 -- 172.21.15.44:0/150095447 >> 172.21.15.44:0/150095447 conn(0x55cf6b8746e0 msgr2=0x55cf6b8804f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 -- 172.21.15.44:0/150095447 shutdown_connections 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 -- 172.21.15.44:0/150095447 wait complete. 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.646+0000 7efe0f056740 1 -- start start 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 Processor -- start 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b88ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b87cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 0x55cf6b87d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b1400 con 0x55cf6b879fb0 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b0d30 con 0x55cf6b8834b0 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe041adf80 con 0x55cf6b87a8d0 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0f056740 10 client.0 _ll_register_callbacks cb 0x7efdf4098db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe09a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b88ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 0x55cf6b87d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe09a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b88ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37900/0 (socket says 172.21.15.44:37900) 2024-04-20T11:30:39.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe0a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 0x55cf6b87d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59344/0 (socket says 172.21.15.44:59344) 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe09a00640 1 -- 172.21.15.44:0/598684071 learned_addr learned my addr 172.21.15.44:0/598684071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.647+0000 7efe09000640 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b87cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efe0a400640 1 -- 172.21.15.44:0/598684071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 msgr2=0x55cf6b87cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efe0a400640 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b87cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efe0a400640 1 -- 172.21.15.44:0/598684071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 msgr2=0x55cf6b88ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efe0a400640 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b88ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efe0a400640 1 -- 172.21.15.44:0/598684071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6b4f13f0 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efe0a400640 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 0x55cf6b87d4f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009bf0 tx=0x7efdfc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa602e6c77c8cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efdef400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.648+0000 7efdef400640 1 -- 172.21.15.44:0/598684071 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc037070 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf0048b50 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efdef400640 1 -- 172.21.15.44:0/598684071 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7efdfc004a90 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efdef400640 1 -- 172.21.15.44:0/598684071 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efdfc043050 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 10 client.27623 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cf6b5bb7d0 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efdef400640 1 -- 172.21.15.44:0/598684071 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efdf0048b50 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efdef400640 1 -- 172.21.15.44:0/598684071 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7efdfc02e040 con 0x55cf6b8834b0 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 10 client.27623 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 10 client.27623 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 20 client.27623 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.649+0000 7efe0f056740 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b8f9ba0 0x55cf6b8fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.650+0000 7efe0f056740 4 client.27623 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.650+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf6b7e68e0 con 0x55cf6b8f9ba0 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.650+0000 7efe09a00640 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b8f9ba0 0x55cf6b8fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.650+0000 7efe09a00640 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b8f9ba0 0x55cf6b8fbf90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7d7df217b986422 server_cookie=948c28ef29788600 in_seq=0 out_seq=0 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.650+0000 7efdef400640 10 client.27623 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efdef400640 1 -- 172.21.15.44:0/598684071 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf6b7e68e0 con 0x55cf6b8f9ba0 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efdef400640 10 client.27623 handle_command_reply: tid=0 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 client.27623 shutdown 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 msgr2=0x55cf6b87d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 0x55cf6b87d4f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009bf0 tx=0x7efdfc011810 comp rx=0 tx=0).stop 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 shutdown_connections 2024-04-20T11:30:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf6b8834b0 0x55cf6b87d4f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf6b87a8d0 0x55cf6b87cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55cf6b8f9ba0 0x55cf6b8fbf90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 --2- 172.21.15.44:0/598684071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cf6b879fb0 0x55cf6b88ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efdef400640 0 client.27623 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 >> 172.21.15.44:0/598684071 conn(0x55cf6b8746e0 msgr2=0x55cf6b876250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 shutdown_connections 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 1 -- 172.21.15.44:0/598684071 wait complete. 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.651+0000 7efe0f056740 20 client.27623 trim_cache size 0 max 16384 2024-04-20T11:30:39.654 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efde80a3da0 msgr2=0x7efde80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efde80a3da0 0x7efde80a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00a010 tx=0x7efdfc00e5a0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 msgr2=0x7efe041a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 0x7efe041a0830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efdf400e9c0 tx=0x7efdf400ee80 comp rx=0 tx=0).stop 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 shutdown_connections 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efde80a3da0 0x7efde80a6260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe0419a9e0 0x7efe0419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe041094e0 0x7efe041a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 --2- 172.21.15.44:0/509387816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe04104410 0x7efe041a0830 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 >> 172.21.15.44:0/509387816 conn(0x7efe040ffcf0 msgr2=0x7efe04101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 shutdown_connections 2024-04-20T11:30:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.652+0000 7efe0c400640 1 -- 172.21.15.44:0/509387816 wait complete. 2024-04-20T11:30:39.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:30:39.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 -- 172.21.15.44:0/2122912823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec1094e0 msgr2=0x7febec1110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 --2- 172.21.15.44:0/2122912823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec1094e0 0x7febec1110b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7febe4009be0 tx=0x7febe402bef0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 -- 172.21.15.44:0/2122912823 shutdown_connections 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 --2- 172.21.15.44:0/2122912823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec1094e0 0x7febec1110b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 --2- 172.21.15.44:0/2122912823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febec104dc0 0x7febec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 --2- 172.21.15.44:0/2122912823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec104410 0x7febec1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 -- 172.21.15.44:0/2122912823 >> 172.21.15.44:0/2122912823 conn(0x7febec0ffcf0 msgr2=0x7febec102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 -- 172.21.15.44:0/2122912823 shutdown_connections 2024-04-20T11:30:39.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.868+0000 7febf1600640 1 -- 172.21.15.44:0/2122912823 wait complete. 2024-04-20T11:30:39.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 Processor -- start 2024-04-20T11:30:39.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 -- start start 2024-04-20T11:30:39.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 0x7febec1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec1094e0 0x7febec1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febec19a980 0x7febec19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febec114700 con 0x7febec1094e0 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febec114580 con 0x7febec104410 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.869+0000 7febf1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febec114880 con 0x7febec19a980 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 0x7febec1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febe3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec1094e0 0x7febec1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 0x7febec1a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59364/0 (socket says 172.21.15.44:59364) 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 -- 172.21.15.44:0/78185667 learned_addr learned my addr 172.21.15.44:0/78185667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeb400640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febec19a980 0x7febec19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 -- 172.21.15.44:0/78185667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febec19a980 msgr2=0x7febec19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febec19a980 0x7febec19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 -- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec1094e0 msgr2=0x7febec1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec1094e0 0x7febec1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 -- 172.21.15.44:0/78185667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec19b4a0 con 0x7febec104410 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febe3e00640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec1094e0 0x7febec1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.870+0000 7febeaa00640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 0x7febec1a07d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7febdc00e9c0 tx=0x7febdc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.871+0000 7febe3400640 1 -- 172.21.15.44:0/78185667 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc00ccc0 con 0x7febec104410 2024-04-20T11:30:39.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.871+0000 7febe3400640 1 -- 172.21.15.44:0/78185667 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febdc005520 con 0x7febec104410 2024-04-20T11:30:39.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.871+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec19b790 con 0x7febec104410 2024-04-20T11:30:39.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.871+0000 7febe3400640 1 -- 172.21.15.44:0/78185667 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc010690 con 0x7febec104410 2024-04-20T11:30:39.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.871+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec1a79e0 con 0x7febec104410 2024-04-20T11:30:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.874+0000 7febf43d6740 1 Processor -- start 2024-04-20T11:30:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.874+0000 7febe3400640 1 -- 172.21.15.44:0/78185667 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7febdc0046c0 con 0x7febec104410 2024-04-20T11:30:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.874+0000 7febf43d6740 1 -- start start 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febf43d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febf43d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febf43d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febf43d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b4c81b050 con 0x557b4c8b5fb0 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febf43d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b4c7c1490 con 0x557b4c8bf4b0 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febf43d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b4c817010 con 0x557b4c8b68d0 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febeb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febeb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37928/0 (socket says 172.21.15.44:37928) 2024-04-20T11:30:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febeb400640 1 -- 172.21.15.44:0/2273181160 learned_addr learned my addr 172.21.15.44:0/2273181160 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febe3e00640 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.875+0000 7febeaa00640 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3e00640 1 -- 172.21.15.44:0/2273181160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 msgr2=0x557b4c8bef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3e00640 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8bef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3e00640 1 -- 172.21.15.44:0/2273181160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 msgr2=0x557b4c8b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3e00640 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3e00640 1 -- 172.21.15.44:0/2273181160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4c7c9070 con 0x557b4c8bf4b0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febeb400640 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3e00640 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c18a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7febec19c170 tx=0x7febd4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8935d436381b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3400640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febd00a3d20 0x7febd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febdb400640 1 -- 172.21.15.44:0/2273181160 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4004230 con 0x557b4c8bf4b0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febdb400640 1 -- 172.21.15.44:0/2273181160 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febd40043d0 con 0x557b4c8bf4b0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febdb400640 1 -- 172.21.15.44:0/2273181160 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40035f0 con 0x557b4c8bf4b0 2024-04-20T11:30:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.876+0000 7febe3400640 1 -- 172.21.15.44:0/78185667 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7febdc0d0aa0 con 0x7febec104410 2024-04-20T11:30:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febeb400640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febd00a3d20 0x7febd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 -- 172.21.15.44:0/2273181160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 msgr2=0x557b4c8c18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c18a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7febec19c170 tx=0x7febd4011810 comp rx=0 tx=0).stop 2024-04-20T11:30:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 -- 172.21.15.44:0/2273181160 shutdown_connections 2024-04-20T11:30:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c18a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8bef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 --2- 172.21.15.44:0/2273181160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 -- 172.21.15.44:0/2273181160 >> 172.21.15.44:0/2273181160 conn(0x557b4c8b06e0 msgr2=0x557b4c8b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febeb400640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febd00a3d20 0x7febd00a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febe40099a0 tx=0x7febe4004a40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 -- 172.21.15.44:0/2273181160 shutdown_connections 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.877+0000 7febf43d6740 1 -- 172.21.15.44:0/2273181160 wait complete. 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.878+0000 7febf43d6740 1 -- start start 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.878+0000 7febf43d6740 1 Processor -- start 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.878+0000 7febf43d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febf43d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b68d0 0x557b4c8bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febf43d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febf43d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b4c8c22a0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febf43d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b4c7d4860 con 0x557b4c8b68d0 2024-04-20T11:30:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febf43d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b4c8c2420 con 0x557b4c8bf4b0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febf43d6740 10 client.0 _ll_register_callbacks cb 0x557b4c7a2fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37932/0 (socket says 172.21.15.44:37932) 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 -- 172.21.15.44:0/3312844456 learned_addr learned my addr 172.21.15.44:0/3312844456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febe3e00640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b68d0 0x557b4c8bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeb400640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 -- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 msgr2=0x557b4c8bcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8bcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 -- 172.21.15.44:0/3312844456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b68d0 msgr2=0x557b4c8bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b68d0 0x557b4c8bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.879+0000 7febeaa00640 1 -- 172.21.15.44:0/3312844456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4c7d8950 con 0x557b4c8b5fb0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febeaa00640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bc520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7febdc09c9b0 tx=0x7febdc098cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecea3daad9481995 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febcfe00640 1 -- 172.21.15.44:0/3312844456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc09c380 con 0x557b4c8b5fb0 2024-04-20T11:30:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febcfe00640 1 -- 172.21.15.44:0/3312844456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7febdc09ce80 con 0x557b4c8b5fb0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd40043d0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febcfe00640 1 -- 172.21.15.44:0/3312844456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7febdc0ce660 con 0x557b4c8b5fb0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febf43d6740 10 client.17511 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557b4c7a6820 con 0x557b4c8b5fb0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.880+0000 7febcfe00640 1 -- 172.21.15.44:0/3312844456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febd40043d0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febcfe00640 1 -- 172.21.15.44:0/3312844456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7febdc0a5200 con 0x557b4c8b5fb0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febf43d6740 10 client.17511 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febf43d6740 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febf43d6740 20 client.17511 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febf43d6740 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c8cf470 0x557b4c8d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febf43d6740 4 client.17511 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4c828a70 con 0x557b4c8cf470 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.881+0000 7febe3e00640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c8cf470 0x557b4c8d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.882+0000 7febe3e00640 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c8cf470 0x557b4c8d1880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48b015ad1dff1c2a server_cookie=de55c19ea6e1c983 in_seq=0 out_seq=0 2024-04-20T11:30:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.882+0000 7febcfe00640 10 client.17511 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febcfe00640 1 -- 172.21.15.44:0/3312844456 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b4c828a70 con 0x557b4c8cf470 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febcfe00640 10 client.17511 handle_command_reply: tid=0 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 client.17511 shutdown 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 msgr2=0x557b4c8bc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bc520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7febdc09c9b0 tx=0x7febdc098cc0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 shutdown_connections 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c8cf470 0x557b4c8d1880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8bcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b68d0 0x557b4c8bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febcfe00640 0 client.17511 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 --2- 172.21.15.44:0/3312844456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bc520 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 >> 172.21.15.44:0/3312844456 conn(0x557b4c8b06e0 msgr2=0x557b4c8b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 shutdown_connections 2024-04-20T11:30:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 1 -- 172.21.15.44:0/3312844456 wait complete. 2024-04-20T11:30:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.883+0000 7febf43d6740 20 client.17511 trim_cache size 0 max 16384 2024-04-20T11:30:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 Processor -- start 2024-04-20T11:30:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 -- start start 2024-04-20T11:30:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 0x557b4c8cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febec1afe70 con 0x557b4c8b68d0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.898+0000 7febf43d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b19b0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febf43d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b0540 con 0x557b4c8bf4b0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 0x557b4c8cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 0x557b4c8cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37940/0 (socket says 172.21.15.44:37940) 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 -- 172.21.15.44:0/1101803002 learned_addr learned my addr 172.21.15.44:0/1101803002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febeaa00640 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febeb400640 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 -- 172.21.15.44:0/1101803002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 msgr2=0x557b4c8d7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8d7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 -- 172.21.15.44:0/1101803002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b5fb0 msgr2=0x557b4c8bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febe3e00640 1 -- 172.21.15.44:0/1101803002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4c8b6700 con 0x557b4c8b68d0 2024-04-20T11:30:39.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.899+0000 7febeaa00640 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.900+0000 7febe3e00640 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 0x557b4c8cb190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7febd4004fd0 tx=0x7febd4004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720c5ff14cae0d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.900+0000 7febdaa00640 1 -- 172.21.15.44:0/1101803002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4032070 con 0x557b4c8b68d0 2024-04-20T11:30:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.900+0000 7febdaa00640 1 -- 172.21.15.44:0/1101803002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febd402a070 con 0x557b4c8b68d0 2024-04-20T11:30:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.900+0000 7febdaa00640 1 -- 172.21.15.44:0/1101803002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd401e040 con 0x557b4c8b68d0 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 -- 172.21.15.44:0/1101803002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 msgr2=0x557b4c8cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 0x557b4c8cb190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7febd4004fd0 tx=0x7febd4004bd0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 -- 172.21.15.44:0/1101803002 shutdown_connections 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8bf4b0 0x557b4c8d7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b68d0 0x557b4c8cb190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 --2- 172.21.15.44:0/1101803002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8b5fb0 0x557b4c8bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 -- 172.21.15.44:0/1101803002 >> 172.21.15.44:0/1101803002 conn(0x557b4c8b06e0 msgr2=0x557b4c8bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 -- 172.21.15.44:0/1101803002 shutdown_connections 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 -- 172.21.15.44:0/1101803002 wait complete. 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 -- start start 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.901+0000 7febf43d6740 1 Processor -- start 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b12e0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b0c10 con 0x557b4c8bf4b0 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febec1ade60 con 0x557b4c8b68d0 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febf43d6740 10 client.0 _ll_register_callbacks cb 0x7febdc004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febeaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febeaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37942/0 (socket says 172.21.15.44:37942) 2024-04-20T11:30:39.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febeaa00640 1 -- 172.21.15.44:0/3791622082 learned_addr learned my addr 172.21.15.44:0/3791622082 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febe3e00640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febeb400640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febeaa00640 1 -- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 msgr2=0x557b4c8c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.902+0000 7febeaa00640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febeaa00640 1 -- 172.21.15.44:0/3791622082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 msgr2=0x557b4c8c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febeaa00640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febeaa00640 1 -- 172.21.15.44:0/3791622082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4c52d3f0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febeaa00640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8c3360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7febdc0993f0 tx=0x7febdc09ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1a8a48be7329dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febcfe00640 1 -- 172.21.15.44:0/3791622082 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc09c380 con 0x557b4c8b5fb0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febcfe00640 1 -- 172.21.15.44:0/3791622082 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7febdc098770 con 0x557b4c8b5fb0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd402a070 con 0x557b4c8b5fb0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febcfe00640 1 -- 172.21.15.44:0/3791622082 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7febdc09c7f0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febf43d6740 10 client.17517 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:39.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.903+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557b4c5f77d0 con 0x557b4c8b5fb0 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febcfe00640 1 -- 172.21.15.44:0/3791622082 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febd402a070 con 0x557b4c8b5fb0 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febcfe00640 1 -- 172.21.15.44:0/3791622082 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7febdc0a1f60 con 0x557b4c8b5fb0 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febf43d6740 10 client.17517 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febf43d6740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febf43d6740 20 client.17517 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febf43d6740 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c936150 0x557b4c938540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febf43d6740 4 client.17517 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b4c8228e0 con 0x557b4c936150 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.904+0000 7febe3e00640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c936150 0x557b4c938540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febe3e00640 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c936150 0x557b4c938540 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec118f1a39516fcc server_cookie=946e06e6b5589792 in_seq=0 out_seq=0 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febcfe00640 10 client.17517 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febcfe00640 1 -- 172.21.15.44:0/3791622082 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b4c8228e0 con 0x557b4c936150 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febcfe00640 10 client.17517 handle_command_reply: tid=0 2024-04-20T11:30:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 client.17517 shutdown 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 msgr2=0x557b4c8c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8c3360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7febdc0993f0 tx=0x7febdc09ceb0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 shutdown_connections 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x557b4c936150 0x557b4c938540 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b4c8bf4b0 0x557b4c8c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4c8b68d0 0x557b4c8c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febcfe00640 0 client.17517 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 --2- 172.21.15.44:0/3791622082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b4c8b5fb0 0x557b4c8c3360 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.905+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 >> 172.21.15.44:0/3791622082 conn(0x557b4c8b06e0 msgr2=0x557b4c8aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 shutdown_connections 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf43d6740 1 -- 172.21.15.44:0/3791622082 wait complete. 2024-04-20T11:30:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf43d6740 20 client.17517 trim_cache size 0 max 16384 2024-04-20T11:30:39.910 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febd00a3d20 msgr2=0x7febd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febd00a3d20 0x7febd00a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febe40099a0 tx=0x7febe4004a40 comp rx=0 tx=0).stop 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 msgr2=0x7febec1a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 0x7febec1a07d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7febdc00e9c0 tx=0x7febdc00ee80 comp rx=0 tx=0).stop 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 shutdown_connections 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febec19a980 0x7febec19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.906+0000 7febf1600640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7febd00a3d20 0x7febd00a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.907+0000 7febf1600640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febec1094e0 0x7febec1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.907+0000 7febf1600640 1 --2- 172.21.15.44:0/78185667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febec104410 0x7febec1a07d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.907+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 >> 172.21.15.44:0/78185667 conn(0x7febec0ffcf0 msgr2=0x7febec100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.907+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 shutdown_connections 2024-04-20T11:30:39.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:39.907+0000 7febf1600640 1 -- 172.21.15.44:0/78185667 wait complete. 2024-04-20T11:30:39.922 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:39 smithi044 ceph-mon[16855]: pgmap v248: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:39.922 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:39 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2527771509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:30:39.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:30:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:39 smithi165 ceph-mon[21975]: pgmap v248: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:39 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2527771509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:30:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 -- 172.21.15.44:0/260491423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074c30 msgr2=0x7ff1d410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 --2- 172.21.15.44:0/260491423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074c30 0x7ff1d410e180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009900 tx=0x7ff1bc02bcc0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 -- 172.21.15.44:0/260491423 shutdown_connections 2024-04-20T11:30:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 --2- 172.21.15.44:0/260491423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d410e6c0 0x7ff1d4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 --2- 172.21.15.44:0/260491423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074c30 0x7ff1d410e180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 --2- 172.21.15.44:0/260491423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d4074310 0x7ff1d40746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 -- 172.21.15.44:0/260491423 >> 172.21.15.44:0/260491423 conn(0x7ff1d40fd930 msgr2=0x7ff1d40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 -- 172.21.15.44:0/260491423 shutdown_connections 2024-04-20T11:30:40.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.131+0000 7ff1d9200640 1 -- 172.21.15.44:0/260491423 wait complete. 2024-04-20T11:30:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.132+0000 7ff1d9200640 1 Processor -- start 2024-04-20T11:30:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 -- start start 2024-04-20T11:30:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 0x7ff1d41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4074c30 0x7ff1d41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d410e6c0 0x7ff1d419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4114500 con 0x7ff1d410e6c0 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4114380 con 0x7ff1d4074310 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d9200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4114680 con 0x7ff1d4074c30 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 0x7ff1d41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d410e6c0 0x7ff1d419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 0x7ff1d41a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59422/0 (socket says 172.21.15.44:59422) 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d3400640 1 -- 172.21.15.44:0/2161162400 learned_addr learned my addr 172.21.15.44:0/2161162400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.133+0000 7ff1d2a00640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4074c30 0x7ff1d41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d3400640 1 -- 172.21.15.44:0/2161162400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4074c30 msgr2=0x7ff1d41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4074c30 0x7ff1d41a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d3400640 1 -- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d410e6c0 msgr2=0x7ff1d419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d410e6c0 0x7ff1d419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d3400640 1 -- 172.21.15.44:0/2161162400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d419b010 con 0x7ff1d4074310 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 0x7ff1d41a0680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400ebc0 tx=0x7ff1c400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1cbe00640 1 -- 172.21.15.44:0/2161162400 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4010430 con 0x7ff1d4074310 2024-04-20T11:30:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d419b300 con 0x7ff1d4074310 2024-04-20T11:30:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1cbe00640 1 -- 172.21.15.44:0/2161162400 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff1c4010ad0 con 0x7ff1d4074310 2024-04-20T11:30:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.134+0000 7ff1cbe00640 1 -- 172.21.15.44:0/2161162400 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4018be0 con 0x7ff1d4074310 2024-04-20T11:30:40.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.135+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d41a74d0 con 0x7ff1d4074310 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 Processor -- start 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1cbe00640 1 -- 172.21.15.44:0/2161162400 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff1c4010c80 con 0x7ff1d4074310 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 -- start start 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8170 0x56181dae8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 0x56181daf1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daf3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56181da4d050 con 0x56181daf1670 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56181d9f3490 con 0x56181dae8170 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.138+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56181da49010 con 0x56181dae8a90 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 0x56181daf1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 0x56181daf1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59416/0 (socket says 172.21.15.44:59416) 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 -- 172.21.15.44:0/3823386113 learned_addr learned my addr 172.21.15.44:0/3823386113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8170 0x56181dae8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d2a00640 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daf3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 -- 172.21.15.44:0/3823386113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8170 msgr2=0x56181dae8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8170 0x56181dae8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 -- 172.21.15.44:0/3823386113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 msgr2=0x56181daf3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daf3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.139+0000 7ff1d3400640 1 -- 172.21.15.44:0/3823386113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181d9fb070 con 0x56181dae8a90 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1cbe00640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff1b80a3d20 0x7ff1b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1d3400640 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 0x56181daf1130 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40ce6a0 tx=0x7ff1c40d96f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55b81cebfe440a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1cbe00640 1 -- 172.21.15.44:0/2161162400 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7ff1c40d0fa0 con 0x7ff1d4074310 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1c8c00640 1 -- 172.21.15.44:0/3823386113 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40b6070 con 0x56181dae8a90 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff1b80a3d20 0x7ff1b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1c8c00640 1 -- 172.21.15.44:0/3823386113 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff1c4018420 con 0x56181dae8a90 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.140+0000 7ff1c8c00640 1 -- 172.21.15.44:0/3823386113 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c409cac0 con 0x56181dae8a90 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/3823386113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 msgr2=0x56181daf1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 0x56181daf1130 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40ce6a0 tx=0x7ff1c40d96f0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff1b80a3d20 0x7ff1b80a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009d10 tx=0x7ff1cc005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/3823386113 shutdown_connections 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daf3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8a90 0x56181daf1130 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/3823386113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8170 0x56181dae8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/3823386113 >> 172.21.15.44:0/3823386113 conn(0x56181dae28a0 msgr2=0x56181dae2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/3823386113 shutdown_connections 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.141+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/3823386113 wait complete. 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 -- start start 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 Processor -- start 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181daeec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56181daf4460 con 0x56181daf1670 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56181da06860 con 0x56181dae8a90 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56181daf45e0 con 0x56181dae8170 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.142+0000 7ff1dd5ff740 10 client.0 _ll_register_callbacks cb 0x56181d9d4fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:40.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37968/0 (socket says 172.21.15.44:37968) 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3e00640 1 -- 172.21.15.44:0/826266118 learned_addr learned my addr 172.21.15.44:0/826266118 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d2a00640 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181daeec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 -- 172.21.15.44:0/826266118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 msgr2=0x56181daeec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181daeec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 -- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 msgr2=0x56181daef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 -- 172.21.15.44:0/826266118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181da0a950 con 0x56181dae8170 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1d3400640 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daee700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009270 tx=0x7ff1c400cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac08b1142d38b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.143+0000 7ff1c1600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1c1600640 1 -- 172.21.15.44:0/826266118 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40b6040 con 0x56181dae8170 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56181da5aa70 con 0x56181dae8170 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1c1600640 1 -- 172.21.15.44:0/826266118 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7ff1c409d1e0 con 0x56181dae8170 2024-04-20T11:30:40.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1c1600640 1 -- 172.21.15.44:0/826266118 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff1c409c8e0 con 0x56181dae8170 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1dd5ff740 10 client.27649 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56181d9d8820 con 0x56181dae8170 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1c1600640 1 -- 172.21.15.44:0/826266118 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x56181da5aa70 con 0x56181dae8170 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1c1600640 1 -- 172.21.15.44:0/826266118 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7ff1c40a5760 con 0x56181dae8170 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1dd5ff740 10 client.27649 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.144+0000 7ff1dd5ff740 10 client.27649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1dd5ff740 20 client.27649 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db01550 0x56181db03940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1dd5ff740 4 client.27649 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:40.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56181da548e0 con 0x56181db01550 2024-04-20T11:30:40.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1d2a00640 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db01550 0x56181db03940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1d2a00640 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db01550 0x56181db03940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31b7e0bef7296a82 server_cookie=d3b805df5e367ac4 in_seq=0 out_seq=0 2024-04-20T11:30:40.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.145+0000 7ff1c1600640 10 client.27649 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:40.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.146+0000 7ff1c1600640 1 -- 172.21.15.44:0/826266118 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56181da548e0 con 0x56181db01550 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.146+0000 7ff1c1600640 10 client.27649 handle_command_reply: tid=0 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.146+0000 7ff1dd5ff740 1 client.27649 shutdown 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 msgr2=0x56181daee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daee700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009270 tx=0x7ff1c400cf10 comp rx=0 tx=0).stop 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 shutdown_connections 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db01550 0x56181db03940 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181daef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181daeec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/826266118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daee700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 >> 172.21.15.44:0/826266118 conn(0x56181dae28a0 msgr2=0x56181dae45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1c1600640 0 client.27649 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 shutdown_connections 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/826266118 wait complete. 2024-04-20T11:30:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.147+0000 7ff1dd5ff740 20 client.27649 trim_cache size 0 max 16384 2024-04-20T11:30:40.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 Processor -- start 2024-04-20T11:30:40.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 -- start start 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daf0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181dae8a90 0x56181daed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 0x56181dae17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41afd60 con 0x56181dae8a90 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41b18a0 con 0x56181daf1670 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.162+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41b0430 con 0x56181dae8170 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 0x56181dae17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 0x56181dae17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59448/0 (socket says 172.21.15.44:59448) 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 -- 172.21.15.44:0/93229523 learned_addr learned my addr 172.21.15.44:0/93229523 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d2a00640 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181dae8a90 0x56181daed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3400640 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daf0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 -- 172.21.15.44:0/93229523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 msgr2=0x56181daf0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daf0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 -- 172.21.15.44:0/93229523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181dae8a90 msgr2=0x56181daed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181dae8a90 0x56181daed860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.163+0000 7ff1d3e00640 1 -- 172.21.15.44:0/93229523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181d75f3f0 con 0x56181daf1670 2024-04-20T11:30:40.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.164+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 0x56181dae17e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc01d730 tx=0x7ff1cc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5cf31d61468b822 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.164+0000 7ff1c3e00640 1 -- 172.21.15.44:0/93229523 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc01dd20 con 0x56181daf1670 2024-04-20T11:30:40.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.164+0000 7ff1c3e00640 1 -- 172.21.15.44:0/93229523 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff1cc02e070 con 0x56181daf1670 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.164+0000 7ff1c3e00640 1 -- 172.21.15.44:0/93229523 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00f8d0 con 0x56181daf1670 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.164+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/93229523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 msgr2=0x56181dae17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.164+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 0x56181dae17e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc01d730 tx=0x7ff1cc017000 comp rx=0 tx=0).stop 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/93229523 shutdown_connections 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181daf1670 0x56181dae17e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181dae8a90 0x56181daed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/93229523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181daf0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/93229523 >> 172.21.15.44:0/93229523 conn(0x56181dae28a0 msgr2=0x56181dae10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/93229523 shutdown_connections 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/93229523 wait complete. 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 -- start start 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.165+0000 7ff1dd5ff740 1 Processor -- start 2024-04-20T11:30:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181db68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181db68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181db03c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41b11d0 con 0x56181daf1670 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41b0b00 con 0x56181dae8a90 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d41add50 con 0x56181dae8170 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1dd5ff740 10 client.0 _ll_register_callbacks cb 0x7ff1cc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181db03c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181db03c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37994/0 (socket says 172.21.15.44:37994) 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 -- 172.21.15.44:0/2643278201 learned_addr learned my addr 172.21.15.44:0/2643278201 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181db68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181db68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 -- 172.21.15.44:0/2643278201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 msgr2=0x56181db68340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181db68340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 -- 172.21.15.44:0/2643278201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 msgr2=0x56181db68880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181db68880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3e00640 1 -- 172.21.15.44:0/2643278201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181d8297d0 con 0x56181daf1670 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.166+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181db68340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1d3e00640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181db03c60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0055b0 tx=0x7ff1cc0055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4b5b3f0779fce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1c1600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1c1600640 1 -- 172.21.15.44:0/2643278201 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004ad0 con 0x56181daf1670 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1c1600640 1 -- 172.21.15.44:0/2643278201 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7ff1cc004c70 con 0x56181daf1670 2024-04-20T11:30:40.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc02e070 con 0x56181daf1670 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1c1600640 1 -- 172.21.15.44:0/2643278201 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc042050 con 0x56181daf1670 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1dd5ff740 10 client.17535 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.167+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56181db6f920 con 0x56181daf1670 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1c1600640 1 -- 172.21.15.44:0/2643278201 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff1cc02e070 con 0x56181daf1670 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1c1600640 1 -- 172.21.15.44:0/2643278201 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7ff1cc00f5d0 con 0x56181daf1670 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1dd5ff740 10 client.17535 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1dd5ff740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1dd5ff740 20 client.17535 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db6ff30 0x56181db72320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1dd5ff740 4 client.17535 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56181d9d05a0 con 0x56181db6ff30 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.168+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db6ff30 0x56181db72320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1d3400640 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db6ff30 0x56181db72320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a2a1fa77e612f97 server_cookie=6143d818d532ac98 in_seq=0 out_seq=0 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1c1600640 10 client.17535 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1c1600640 1 -- 172.21.15.44:0/2643278201 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56181d9d05a0 con 0x56181db6ff30 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1c1600640 10 client.17535 handle_command_reply: tid=0 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1dd5ff740 1 client.17535 shutdown 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 msgr2=0x56181db03c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.169+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181db03c60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0055b0 tx=0x7ff1cc0055e0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 shutdown_connections 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56181db6ff30 0x56181db72320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56181daf1670 0x56181db03c60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56181dae8a90 0x56181db68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 --2- 172.21.15.44:0/2643278201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56181dae8170 0x56181db68340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 >> 172.21.15.44:0/2643278201 conn(0x56181dae28a0 msgr2=0x56181dae3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1c1600640 0 client.17535 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 shutdown_connections 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 1 -- 172.21.15.44:0/2643278201 wait complete. 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.170+0000 7ff1dd5ff740 20 client.17535 trim_cache size 0 max 16384 2024-04-20T11:30:40.173 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff1b80a3d20 msgr2=0x7ff1b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff1b80a3d20 0x7ff1b80a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009d10 tx=0x7ff1cc005f90 comp rx=0 tx=0).stop 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 msgr2=0x7ff1d41a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 0x7ff1d41a0680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400ebc0 tx=0x7ff1c400c2d0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 shutdown_connections 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff1b80a3d20 0x7ff1b80a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1d410e6c0 0x7ff1d419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4074c30 0x7ff1d41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 --2- 172.21.15.44:0/2161162400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d4074310 0x7ff1d41a0680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 >> 172.21.15.44:0/2161162400 conn(0x7ff1d40fd930 msgr2=0x7ff1d40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 shutdown_connections 2024-04-20T11:30:40.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.171+0000 7ff1d9200640 1 -- 172.21.15.44:0/2161162400 wait complete. 2024-04-20T11:30:40.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:39 smithi078 ceph-mon[21464]: pgmap v248: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:40.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:39 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2527771509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:30:40.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 -- 172.21.15.44:0/2940500872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 msgr2=0x7f86b010e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 --2- 172.21.15.44:0/2940500872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b010e9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009930 tx=0x7f86a402bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 -- 172.21.15.44:0/2940500872 shutdown_connections 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 --2- 172.21.15.44:0/2940500872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b0108f50 0x7f86b01153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 --2- 172.21.15.44:0/2940500872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 --2- 172.21.15.44:0/2940500872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b010e9e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 -- 172.21.15.44:0/2940500872 >> 172.21.15.44:0/2940500872 conn(0x7f86b00764a0 msgr2=0x7f86b00788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 -- 172.21.15.44:0/2940500872 shutdown_connections 2024-04-20T11:30:40.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.406+0000 7f86b7400640 1 -- 172.21.15.44:0/2940500872 wait complete. 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.407+0000 7f86b7400640 1 Processor -- start 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.407+0000 7f86b7400640 1 -- start start 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b0108f50 0x7f86b019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b01a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b7400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01189c0 con 0x7f86b0108f50 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b7400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0118840 con 0x7f86b0108590 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b7400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0118b40 con 0x7f86b010e600 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b0108f50 0x7f86b019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59466/0 (socket says 172.21.15.44:59466) 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b01a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.408+0000 7f86b4a00640 1 -- 172.21.15.44:0/1053722155 learned_addr learned my addr 172.21.15.44:0/1053722155 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b4a00640 1 -- 172.21.15.44:0/1053722155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 msgr2=0x7f86b01a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b4a00640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b01a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b4a00640 1 -- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b0108f50 msgr2=0x7f86b019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b4a00640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b0108f50 0x7f86b019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b4a00640 1 -- 172.21.15.44:0/1053722155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b01a3ea0 con 0x7f86b0108590 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b5400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b01a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.409+0000 7f86b4a00640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b019ef00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009900 tx=0x7f86a402dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.410+0000 7f86ad600640 1 -- 172.21.15.44:0/1053722155 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4043020 con 0x7f86b0108590 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.410+0000 7f86ad600640 1 -- 172.21.15.44:0/1053722155 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86a4004250 con 0x7f86b0108590 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.410+0000 7f86ad600640 1 -- 172.21.15.44:0/1053722155 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4003b90 con 0x7f86b0108590 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.410+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b01a4130 con 0x7f86b0108590 2024-04-20T11:30:40.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.410+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01aba20 con 0x7f86b0108590 2024-04-20T11:30:40.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.413+0000 7f86ba048740 1 Processor -- start 2024-04-20T11:30:40.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.413+0000 7f86ba048740 1 -- start start 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.413+0000 7f86ba048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973af7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.413+0000 7f86ba048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af78d0 0x561973afff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86ba048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973b028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86ba048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561973a5c050 con 0x561973af78d0 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86ba048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561973a02490 con 0x561973b004b0 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86ba048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561973a58010 con 0x561973af6fb0 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973af7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973af7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59492/0 (socket says 172.21.15.44:59492) 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 -- 172.21.15.44:0/4262464379 learned_addr learned my addr 172.21.15.44:0/4262464379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86ad600640 1 -- 172.21.15.44:0/1053722155 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f86a4046020 con 0x7f86b0108590 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86afe00640 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973b028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 -- 172.21.15.44:0/4262464379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 msgr2=0x561973b028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973b028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 -- 172.21.15.44:0/4262464379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af78d0 msgr2=0x561973afff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af78d0 0x561973afff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86b5400640 1 -- 172.21.15.44:0/4262464379 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561973a0a070 con 0x561973af6fb0 2024-04-20T11:30:40.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.414+0000 7f86afe00640 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973b028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:40.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.415+0000 7f86b5400640 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973af7390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86a80107e0 tx=0x7f86a8010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81550753e587e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.415+0000 7f869dc00640 1 -- 172.21.15.44:0/4262464379 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003b70 con 0x561973af6fb0 2024-04-20T11:30:40.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.415+0000 7f869dc00640 1 -- 172.21.15.44:0/4262464379 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86a800ee30 con 0x561973af6fb0 2024-04-20T11:30:40.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.415+0000 7f869dc00640 1 -- 172.21.15.44:0/4262464379 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8011440 con 0x561973af6fb0 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ad600640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f86980a3da0 0x7f86980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 -- 172.21.15.44:0/4262464379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 msgr2=0x561973af7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973af7390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86a80107e0 tx=0x7f86a8010ca0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 -- 172.21.15.44:0/4262464379 shutdown_connections 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ad600640 1 -- 172.21.15.44:0/1053722155 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f86a40f26c0 con 0x7f86b0108590 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973b028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af78d0 0x561973afff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 --2- 172.21.15.44:0/4262464379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973af7390 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 -- 172.21.15.44:0/4262464379 >> 172.21.15.44:0/4262464379 conn(0x561973af16e0 msgr2=0x561973af1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 -- 172.21.15.44:0/4262464379 shutdown_connections 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86b4a00640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f86980a3da0 0x7f86980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.416+0000 7f86ba048740 1 -- 172.21.15.44:0/4262464379 wait complete. 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 -- start start 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86b4a00640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f86980a3da0 0x7f86980a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x561973af6a80 tx=0x7f86a402bc30 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 Processor -- start 2024-04-20T11:30:40.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973afd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af78d0 0x561973afda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973afdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561973b032a0 con 0x561973b004b0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561973a15860 con 0x561973af78d0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.417+0000 7f86ba048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561973b03420 con 0x561973af6fb0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86ba048740 10 client.0 _ll_register_callbacks cb 0x5619739e3fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973afd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973afd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59506/0 (socket says 172.21.15.44:59506) 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 -- 172.21.15.44:0/3542906657 learned_addr learned my addr 172.21.15.44:0/3542906657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973afdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86afe00640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af78d0 0x561973afda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 -- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af78d0 msgr2=0x561973afda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af78d0 0x561973afda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 -- 172.21.15.44:0/3542906657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 msgr2=0x561973afdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973afdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.418+0000 7f86b5400640 1 -- 172.21.15.44:0/3542906657 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561973a19950 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f86b5400640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973afd4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f86a8004600 tx=0x7f86a8004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f12ea56c962f5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f8692000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f8692000640 1 -- 172.21.15.44:0/3542906657 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8025040 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f8692000640 1 -- 172.21.15.44:0/3542906657 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f86a8003970 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a800ee30 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f8692000640 1 -- 172.21.15.44:0/3542906657 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f86a8032050 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f86ba048740 10 client.27667 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.419+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5619739e7820 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f8692000640 1 -- 172.21.15.44:0/3542906657 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86a800ee30 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f8692000640 1 -- 172.21.15.44:0/3542906657 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f86a801aa50 con 0x561973af6fb0 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f86ba048740 10 client.27667 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f86ba048740 10 client.27667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f86ba048740 20 client.27667 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f86ba048740 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b10450 0x561973b12840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f86ba048740 4 client.27667 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.420+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561973a69a70 con 0x561973b10450 2024-04-20T11:30:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.421+0000 7f86afe00640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b10450 0x561973b12840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.421+0000 7f86afe00640 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b10450 0x561973b12840 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59874c1e1d48213 server_cookie=114193638b60d89d in_seq=0 out_seq=0 2024-04-20T11:30:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.421+0000 7f8692000640 10 client.27667 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f8692000640 1 -- 172.21.15.44:0/3542906657 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561973a69a70 con 0x561973b10450 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f8692000640 10 client.27667 handle_command_reply: tid=0 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 client.27667 shutdown 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 msgr2=0x561973afd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973afd4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f86a8004600 tx=0x7f86a8004460 comp rx=0 tx=0).stop 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 shutdown_connections 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b10450 0x561973b12840 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973afdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af78d0 0x561973afda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 --2- 172.21.15.44:0/3542906657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af6fb0 0x561973afd4c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f8692000640 0 client.27667 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 >> 172.21.15.44:0/3542906657 conn(0x561973af16e0 msgr2=0x561973af3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 shutdown_connections 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 1 -- 172.21.15.44:0/3542906657 wait complete. 2024-04-20T11:30:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.422+0000 7f86ba048740 20 client.27667 trim_cache size 0 max 16384 2024-04-20T11:30:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.438+0000 7f86ba048740 1 Processor -- start 2024-04-20T11:30:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 -- start start 2024-04-20T11:30:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af6fb0 0x561973affbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b0c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973b17fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b4280 con 0x561973b004b0 2024-04-20T11:30:40.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b5dc0 con 0x561973af6fb0 2024-04-20T11:30:40.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86ba048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b4950 con 0x561973af78d0 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86afe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b0c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86b5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af6fb0 0x561973affbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86afe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b0c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59520/0 (socket says 172.21.15.44:59520) 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86afe00640 1 -- 172.21.15.44:0/1570672582 learned_addr learned my addr 172.21.15.44:0/1570672582 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.439+0000 7f86b5400640 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973b17fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f86afe00640 1 -- 172.21.15.44:0/1570672582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af6fb0 msgr2=0x561973affbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f86afe00640 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af6fb0 0x561973affbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f86afe00640 1 -- 172.21.15.44:0/1570672582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 msgr2=0x561973b17fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f86afe00640 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973b17fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f86afe00640 1 -- 172.21.15.44:0/1570672582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197376e3f0 con 0x561973af78d0 2024-04-20T11:30:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f86afe00640 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b0c190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f86a00045f0 tx=0x7f86a000be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257c56967640b4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.440+0000 7f869d200640 1 -- 172.21.15.44:0/1570672582 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0003de0 con 0x561973af78d0 2024-04-20T11:30:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f869d200640 1 -- 172.21.15.44:0/1570672582 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86a001d930 con 0x561973af78d0 2024-04-20T11:30:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f869d200640 1 -- 172.21.15.44:0/1570672582 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0010430 con 0x561973af78d0 2024-04-20T11:30:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 -- 172.21.15.44:0/1570672582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 msgr2=0x561973b0c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b0c190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f86a00045f0 tx=0x7f86a000be00 comp rx=0 tx=0).stop 2024-04-20T11:30:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 -- 172.21.15.44:0/1570672582 shutdown_connections 2024-04-20T11:30:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973b004b0 0x561973b17fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b0c190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 --2- 172.21.15.44:0/1570672582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973af6fb0 0x561973affbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 -- 172.21.15.44:0/1570672582 >> 172.21.15.44:0/1570672582 conn(0x561973af16e0 msgr2=0x561973afc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 -- 172.21.15.44:0/1570672582 shutdown_connections 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.441+0000 7f86ba048740 1 -- 172.21.15.44:0/1570672582 wait complete. 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.442+0000 7f86ba048740 1 -- start start 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.442+0000 7f86ba048740 1 Processor -- start 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 0x561973b086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b08be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973af9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b56f0 con 0x561973af6fb0 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b5020 con 0x561973b004b0 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b2270 con 0x561973af78d0 2024-04-20T11:30:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86ba048740 10 client.0 _ll_register_callbacks cb 0x7f86a800ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 0x561973b086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 0x561973b086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38030/0 (socket says 172.21.15.44:38030) 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 -- 172.21.15.44:0/1732031973 learned_addr learned my addr 172.21.15.44:0/1732031973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86afe00640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b08be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973af9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 -- 172.21.15.44:0/1732031973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 msgr2=0x561973b08be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b08be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 -- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 msgr2=0x561973af9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973af9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.443+0000 7f86b5400640 1 -- 172.21.15.44:0/1732031973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619738387d0 con 0x561973af6fb0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f86b5400640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 0x561973b086a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f86a80044d0 tx=0x7f86a8004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7ba1be9f850146 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f8692000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f8692000640 1 -- 172.21.15.44:0/1732031973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8004c00 con 0x561973af6fb0 2024-04-20T11:30:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f8692000640 1 -- 172.21.15.44:0/1732031973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f86a8004da0 con 0x561973af6fb0 2024-04-20T11:30:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a001d930 con 0x561973af6fb0 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f8692000640 1 -- 172.21.15.44:0/1732031973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f86a8004290 con 0x561973af6fb0 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f86ba048740 10 client.17550 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.444+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561973b75e70 con 0x561973af6fb0 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f8692000640 1 -- 172.21.15.44:0/1732031973 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86a001d930 con 0x561973af6fb0 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f8692000640 1 -- 172.21.15.44:0/1732031973 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f86a801a860 con 0x561973af6fb0 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86ba048740 10 client.17550 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86ba048740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86ba048740 20 client.17550 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86ba048740 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b76520 0x561973b78940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86ba048740 4 client.17550 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561973a638e0 con 0x561973b76520 2024-04-20T11:30:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.445+0000 7f86afe00640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b76520 0x561973b78940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.446+0000 7f86afe00640 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b76520 0x561973b78940 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca2de30fec8a25f server_cookie=5923115a3605b5b1 in_seq=0 out_seq=0 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.446+0000 7f8692000640 10 client.17550 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.446+0000 7f8692000640 1 -- 172.21.15.44:0/1732031973 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561973a638e0 con 0x561973b76520 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.446+0000 7f8692000640 10 client.17550 handle_command_reply: tid=0 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.446+0000 7f86ba048740 1 client.17550 shutdown 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 msgr2=0x561973b086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 0x561973b086a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f86a80044d0 tx=0x7f86a8004220 comp rx=0 tx=0).stop 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 shutdown_connections 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561973b76520 0x561973b78940 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561973b004b0 0x561973af9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561973af78d0 0x561973b08be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 --2- 172.21.15.44:0/1732031973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561973af6fb0 0x561973b086a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 >> 172.21.15.44:0/1732031973 conn(0x561973af16e0 msgr2=0x561973afd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f8692000640 0 client.17550 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 shutdown_connections 2024-04-20T11:30:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 1 -- 172.21.15.44:0/1732031973 wait complete. 2024-04-20T11:30:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.447+0000 7f86ba048740 20 client.17550 trim_cache size 0 max 16384 2024-04-20T11:30:40.450 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f86980a3da0 msgr2=0x7f86980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f86980a3da0 0x7f86980a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x561973af6a80 tx=0x7f86a402bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 msgr2=0x7f86b019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b019ef00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009900 tx=0x7f86a402dd50 comp rx=0 tx=0).stop 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 shutdown_connections 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86b010e600 0x7f86b01a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f86980a3da0 0x7f86980a6260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86b0108f50 0x7f86b019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 --2- 172.21.15.44:0/1053722155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86b0108590 0x7f86b019ef00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 >> 172.21.15.44:0/1053722155 conn(0x7f86b00764a0 msgr2=0x7f86b0077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 shutdown_connections 2024-04-20T11:30:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:40.448+0000 7f86b7400640 1 -- 172.21.15.44:0/1053722155 wait complete. 2024-04-20T11:30:41.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:40 smithi078 ceph-mon[21464]: pgmap v249: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:41.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:40 smithi044 ceph-mon[16855]: pgmap v249: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:41.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:40 smithi165 ceph-mon[21975]: pgmap v249: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:41.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:30:41.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:30:41.466 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:30:41.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 -- 172.21.15.44:0/2233310965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4258107140 msgr2=0x7f425810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 --2- 172.21.15.44:0/2233310965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4258107140 0x7f425810ed10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4250009ca0 tx=0x7f425002c100 comp rx=0 tx=0).stop 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 -- 172.21.15.44:0/2233310965 shutdown_connections 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 --2- 172.21.15.44:0/2233310965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4258107140 0x7f425810ed10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 --2- 172.21.15.44:0/2233310965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102a20 0x7f4258106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 --2- 172.21.15.44:0/2233310965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258102070 0x7f4258102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 -- 172.21.15.44:0/2233310965 >> 172.21.15.44:0/2233310965 conn(0x7f42580fd930 msgr2=0x7f42580ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 -- 172.21.15.44:0/2233310965 shutdown_connections 2024-04-20T11:30:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.693+0000 7f4257e00640 1 -- 172.21.15.44:0/2233310965 wait complete. 2024-04-20T11:30:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.694+0000 7f4257e00640 1 Processor -- start 2024-04-20T11:30:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.694+0000 7f4257e00640 1 -- start start 2024-04-20T11:30:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102070 0x7f42581a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 0x7f42581a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f425819a890 0x7f425819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4258112330 con 0x7f4258102070 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42581121b0 con 0x7f425819a890 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42581124b0 con 0x7f4258107140 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4256000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 0x7f42581a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102070 0x7f42581a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4256000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 0x7f42581a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59534/0 (socket says 172.21.15.44:59534) 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4256000640 1 -- 172.21.15.44:0/3729950699 learned_addr learned my addr 172.21.15.44:0/3729950699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.695+0000 7f4257400640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f425819a890 0x7f425819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256000640 1 -- 172.21.15.44:0/3729950699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f425819a890 msgr2=0x7f425819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256000640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f425819a890 0x7f425819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256000640 1 -- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102070 msgr2=0x7f42581a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256000640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102070 0x7f42581a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256000640 1 -- 172.21.15.44:0/3729950699 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425819b3b0 con 0x7f4258107140 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256a00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102070 0x7f42581a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.696+0000 7f4256000640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 0x7f42581a0c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f424400e9c0 tx=0x7f424400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.697+0000 7f425ca00640 1 -- 172.21.15.44:0/3729950699 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424400cd80 con 0x7f4258107140 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.697+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425819b6a0 con 0x7f4258107140 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.697+0000 7f425ca00640 1 -- 172.21.15.44:0/3729950699 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4244005520 con 0x7f4258107140 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.697+0000 7f425ca00640 1 -- 172.21.15.44:0/3729950699 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244010690 con 0x7f4258107140 2024-04-20T11:30:41.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.697+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42581a7940 con 0x7f4258107140 2024-04-20T11:30:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.700+0000 7f4260ba2740 1 Processor -- start 2024-04-20T11:30:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.700+0000 7f4260ba2740 1 -- start start 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 0x556e2c8604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c860a10 0x556e2c8690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8695f0 0x556e2c86b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4260ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556e2c7c4d20 con 0x556e2c860a10 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4260ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e2c76b3e0 con 0x556e2c8695f0 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4260ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556e2c7c0ce0 con 0x556e2c8600f0 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f425ca00640 1 -- 172.21.15.44:0/3729950699 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f42440046c0 con 0x7f4258107140 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4257400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 0x556e2c8604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c860a10 0x556e2c8690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4257400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 0x556e2c8604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59548/0 (socket says 172.21.15.44:59548) 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4257400640 1 -- 172.21.15.44:0/1975408626 learned_addr learned my addr 172.21.15.44:0/1975408626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.701+0000 7f4256000640 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8695f0 0x556e2c86b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4257400640 1 -- 172.21.15.44:0/1975408626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8695f0 msgr2=0x556e2c86b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4257400640 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8695f0 0x556e2c86b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4257400640 1 -- 172.21.15.44:0/1975408626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c860a10 msgr2=0x556e2c8690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4257400640 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c860a10 0x556e2c8690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4257400640 1 -- 172.21.15.44:0/1975408626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2c78a980 con 0x556e2c8600f0 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4256a00640 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c860a10 0x556e2c8690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4256000640 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8695f0 0x556e2c86b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4257400640 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 0x556e2c8604d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f425002c710 tx=0x7f4250005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ccd0331dbe73a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4243e00640 1 -- 172.21.15.44:0/1975408626 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250043070 con 0x556e2c8600f0 2024-04-20T11:30:41.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4243e00640 1 -- 172.21.15.44:0/1975408626 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4250032d40 con 0x556e2c8600f0 2024-04-20T11:30:41.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.702+0000 7f4243e00640 1 -- 172.21.15.44:0/1975408626 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250039040 con 0x556e2c8600f0 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f425ca00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f423c0a3da0 0x7f423c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f425ca00640 1 -- 172.21.15.44:0/3729950699 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f42440d0a40 con 0x7f4258107140 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4256a00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f423c0a3da0 0x7f423c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 -- 172.21.15.44:0/1975408626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 msgr2=0x556e2c8604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 0x556e2c8604d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f425002c710 tx=0x7f4250005f80 comp rx=0 tx=0).stop 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 -- 172.21.15.44:0/1975408626 shutdown_connections 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8695f0 0x556e2c86b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c860a10 0x556e2c8690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1975408626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8600f0 0x556e2c8604d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.703+0000 7f4260ba2740 1 -- 172.21.15.44:0/1975408626 >> 172.21.15.44:0/1975408626 conn(0x556e2c85a820 msgr2=0x556e2c85ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.704+0000 7f4260ba2740 1 -- 172.21.15.44:0/1975408626 shutdown_connections 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.704+0000 7f4260ba2740 1 -- 172.21.15.44:0/1975408626 wait complete. 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.704+0000 7f4256a00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f423c0a3da0 0x7f423c0a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4248009fd0 tx=0x7f4248005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.704+0000 7f4260ba2740 1 -- start start 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.704+0000 7f4260ba2740 1 Processor -- start 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8600f0 0x556e2c866600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 0x556e2c866b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8695f0 0x556e2c867080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556e2c86c380 con 0x556e2c8695f0 2024-04-20T11:30:41.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e2c77e7a0 con 0x556e2c8600f0 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556e2c86c500 con 0x556e2c860a10 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4260ba2740 10 client.0 _ll_register_callbacks cb 0x556e2c780b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4257400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8695f0 0x556e2c867080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4256000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 0x556e2c866b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4256a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8600f0 0x556e2c866600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4256000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 0x556e2c866b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59558/0 (socket says 172.21.15.44:59558) 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.705+0000 7f4256000640 1 -- 172.21.15.44:0/1631066809 learned_addr learned my addr 172.21.15.44:0/1631066809 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4256000640 1 -- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8600f0 msgr2=0x556e2c866600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4256000640 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8600f0 0x556e2c866600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4256000640 1 -- 172.21.15.44:0/1631066809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8695f0 msgr2=0x556e2c867080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4256000640 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8695f0 0x556e2c867080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4256000640 1 -- 172.21.15.44:0/1631066809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2c59f7e0 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4257400640 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8695f0 0x556e2c867080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4256000640 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 0x556e2c866b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42440993f0 tx=0x7f42440bd1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e96311ad2c984 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4240c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4240c00640 1 -- 172.21.15.44:0/1631066809 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424409c380 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4240c00640 1 -- 172.21.15.44:0/1631066809 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f424409ce80 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.706+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4250032d40 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4240c00640 1 -- 172.21.15.44:0/1631066809 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f42440ce660 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 10 client.27685 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556e2c5fff70 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4240c00640 1 -- 172.21.15.44:0/1631066809 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4250032d40 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4240c00640 1 -- 172.21.15.44:0/1631066809 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f424409be70 con 0x556e2c860a10 2024-04-20T11:30:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 10 client.27685 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 10 client.27685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 20 client.27685 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c879850 0x556e2c87bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 4 client.27685 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.707+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e2c7d2740 con 0x556e2c879850 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.708+0000 7f4256a00640 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c879850 0x556e2c87bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.708+0000 7f4256a00640 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c879850 0x556e2c87bc40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69380b9118203f16 server_cookie=fadd92e7e63d0286 in_seq=0 out_seq=0 2024-04-20T11:30:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.708+0000 7f4240c00640 10 client.27685 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4240c00640 1 -- 172.21.15.44:0/1631066809 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e2c7d2740 con 0x556e2c879850 2024-04-20T11:30:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4240c00640 10 client.27685 handle_command_reply: tid=0 2024-04-20T11:30:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 client.27685 shutdown 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 msgr2=0x556e2c866b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 0x556e2c866b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42440993f0 tx=0x7f42440bd1a0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 shutdown_connections 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c879850 0x556e2c87bc40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8695f0 0x556e2c867080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c860a10 0x556e2c866b40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 --2- 172.21.15.44:0/1631066809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c8600f0 0x556e2c866600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 >> 172.21.15.44:0/1631066809 conn(0x556e2c85a820 msgr2=0x556e2c85c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4240c00640 0 client.27685 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 shutdown_connections 2024-04-20T11:30:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.709+0000 7f4260ba2740 1 -- 172.21.15.44:0/1631066809 wait complete. 2024-04-20T11:30:41.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.710+0000 7f4260ba2740 20 client.27685 trim_cache size 0 max 16384 2024-04-20T11:30:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 Processor -- start 2024-04-20T11:30:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 -- start start 2024-04-20T11:30:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c881400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c881940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42581afc20 con 0x556e2c8600f0 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42581b1760 con 0x556e2c860a10 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4260ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42581b02f0 con 0x556e2c8695f0 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c881400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c881400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38070/0 (socket says 172.21.15.44:38070) 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4256a00640 1 -- 172.21.15.44:0/2630364845 learned_addr learned my addr 172.21.15.44:0/2630364845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.730+0000 7f4257400640 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256000640 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c881940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256a00640 1 -- 172.21.15.44:0/2630364845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 msgr2=0x556e2c8638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256a00640 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256a00640 1 -- 172.21.15.44:0/2630364845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 msgr2=0x556e2c881940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256a00640 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c881940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256a00640 1 -- 172.21.15.44:0/2630364845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2c497170 con 0x556e2c8600f0 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4256a00640 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c881400 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4248009e80 tx=0x7f424801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d631a199501edd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4243400640 1 -- 172.21.15.44:0/2630364845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424802b530 con 0x556e2c8600f0 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.731+0000 7f4243400640 1 -- 172.21.15.44:0/2630364845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4248005520 con 0x556e2c8600f0 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4243400640 1 -- 172.21.15.44:0/2630364845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248004c40 con 0x556e2c8600f0 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 -- 172.21.15.44:0/2630364845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 msgr2=0x556e2c881400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c881400 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4248009e80 tx=0x7f424801b000 comp rx=0 tx=0).stop 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 -- 172.21.15.44:0/2630364845 shutdown_connections 2024-04-20T11:30:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c881940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 --2- 172.21.15.44:0/2630364845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c881400 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 -- 172.21.15.44:0/2630364845 >> 172.21.15.44:0/2630364845 conn(0x556e2c85a820 msgr2=0x556e2c85c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.732+0000 7f4260ba2740 1 -- 172.21.15.44:0/2630364845 shutdown_connections 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.733+0000 7f4260ba2740 1 -- 172.21.15.44:0/2630364845 wait complete. 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.733+0000 7f4260ba2740 1 -- start start 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.733+0000 7f4260ba2740 1 Processor -- start 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c86f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c86fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42581b1090 con 0x556e2c8600f0 2024-04-20T11:30:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42581b09c0 con 0x556e2c860a10 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42581adc10 con 0x556e2c8695f0 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4260ba2740 10 client.0 _ll_register_callbacks cb 0x7f42440057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c86f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c86f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38076/0 (socket says 172.21.15.44:38076) 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 -- 172.21.15.44:0/4096274335 learned_addr learned my addr 172.21.15.44:0/4096274335 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256000640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c86fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4257400640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 -- 172.21.15.44:0/4096274335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 msgr2=0x556e2c8dec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8dec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 -- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 msgr2=0x556e2c86fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c86fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.734+0000 7f4256a00640 1 -- 172.21.15.44:0/4096274335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2c8df1c0 con 0x556e2c8600f0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4256a00640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c86f580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4248004a60 tx=0x7f42480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893ea318b99306e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4240c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4240c00640 1 -- 172.21.15.44:0/4096274335 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424803a020 con 0x556e2c8600f0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4240c00640 1 -- 172.21.15.44:0/4096274335 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f4248004010 con 0x556e2c8600f0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4248005520 con 0x556e2c8600f0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4240c00640 1 -- 172.21.15.44:0/4096274335 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4248045050 con 0x556e2c8600f0 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4260ba2740 10 client.17568 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.735+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556e2c8e03a0 con 0x556e2c8600f0 2024-04-20T11:30:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4240c00640 1 -- 172.21.15.44:0/4096274335 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4248005520 con 0x556e2c8600f0 2024-04-20T11:30:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4240c00640 1 -- 172.21.15.44:0/4096274335 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f424800f540 con 0x556e2c8600f0 2024-04-20T11:30:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4260ba2740 10 client.17568 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4260ba2740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4260ba2740 20 client.17568 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4260ba2740 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c8e0ab0 0x556e2c8e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4260ba2740 4 client.17568 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e2c85a080 con 0x556e2c8e0ab0 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.736+0000 7f4256000640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c8e0ab0 0x556e2c8e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.737+0000 7f4256000640 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c8e0ab0 0x556e2c8e2ed0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8097be5052424292 server_cookie=d9ddc70976f8d536 in_seq=0 out_seq=0 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.737+0000 7f4240c00640 10 client.17568 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.738+0000 7f4240c00640 1 -- 172.21.15.44:0/4096274335 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e2c85a080 con 0x556e2c8e0ab0 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.738+0000 7f4240c00640 10 client.17568 handle_command_reply: tid=0 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.738+0000 7f4260ba2740 1 client.17568 shutdown 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.738+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 msgr2=0x556e2c86f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.738+0000 7f4260ba2740 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c86f580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4248004a60 tx=0x7f42480046c0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.738+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 shutdown_connections 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556e2c8e0ab0 0x556e2c8e2ed0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e2c8695f0 0x556e2c8dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e2c860a10 0x556e2c86fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 --2- 172.21.15.44:0/4096274335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e2c8600f0 0x556e2c86f580 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4240c00640 0 client.17568 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 >> 172.21.15.44:0/4096274335 conn(0x556e2c85a820 msgr2=0x556e2c867cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 shutdown_connections 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 1 -- 172.21.15.44:0/4096274335 wait complete. 2024-04-20T11:30:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.739+0000 7f4260ba2740 20 client.17568 trim_cache size 0 max 16384 2024-04-20T11:30:41.743 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:30:41.743 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:30:41.743 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "b0be046c-a704-427c-a0b5-1355bfd4f11d", 2024-04-20T11:30:41.743 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:30:41.743 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:30:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f423c0a3da0 msgr2=0x7f423c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f423c0a3da0 0x7f423c0a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4248009fd0 tx=0x7f4248005f90 comp rx=0 tx=0).stop 2024-04-20T11:30:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 msgr2=0x7f42581a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 0x7f42581a0c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f424400e9c0 tx=0x7f424400ee80 comp rx=0 tx=0).stop 2024-04-20T11:30:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 shutdown_connections 2024-04-20T11:30:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f425819a890 0x7f425819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4258107140 0x7f42581a0c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f423c0a3da0 0x7f423c0a6260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 --2- 172.21.15.44:0/3729950699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4258102070 0x7f42581a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 >> 172.21.15.44:0/3729950699 conn(0x7f42580fd930 msgr2=0x7f425810e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 shutdown_connections 2024-04-20T11:30:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.740+0000 7f4257e00640 1 -- 172.21.15.44:0/3729950699 wait complete. 2024-04-20T11:30:41.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:30:41.760 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 -- 172.21.15.44:0/1384316123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501009b0 msgr2=0x7f3150100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 --2- 172.21.15.44:0/1384316123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501009b0 0x7f3150100d90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f314002bc30 comp rx=0 tx=0).stop 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 -- 172.21.15.44:0/1384316123 shutdown_connections 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 --2- 172.21.15.44:0/1384316123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31501071c0 0x7f315010ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 --2- 172.21.15.44:0/1384316123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3150101360 0x7f3150106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 --2- 172.21.15.44:0/1384316123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501009b0 0x7f3150100d90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 -- 172.21.15.44:0/1384316123 >> 172.21.15.44:0/1384316123 conn(0x7f3150075ff0 msgr2=0x7f3150076400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 -- 172.21.15.44:0/1384316123 shutdown_connections 2024-04-20T11:30:41.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.970+0000 7f314f200640 1 -- 172.21.15.44:0/1384316123 wait complete. 2024-04-20T11:30:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.971+0000 7f314f200640 1 Processor -- start 2024-04-20T11:30:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.971+0000 7f314f200640 1 -- start start 2024-04-20T11:30:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31501009b0 0x7f315019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 0x7f315019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3150198610 0x7f3150198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3150112310 con 0x7f31501071c0 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3150112190 con 0x7f31501009b0 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3150112490 con 0x7f3150198610 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 0x7f315019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 0x7f315019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38088/0 (socket says 172.21.15.44:38088) 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 -- 172.21.15.44:0/489974763 learned_addr learned my addr 172.21.15.44:0/489974763 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314e800640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3150198610 0x7f3150198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f314de00640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31501009b0 0x7f315019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 -- 172.21.15.44:0/489974763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3150198610 msgr2=0x7f3150198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3150198610 0x7f3150198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 -- 172.21.15.44:0/489974763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31501009b0 msgr2=0x7f315019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.972+0000 7f3147e00640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31501009b0 0x7f315019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f3147e00640 1 -- 172.21.15.44:0/489974763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3150199130 con 0x7f31501071c0 2024-04-20T11:30:41.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f314de00640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31501009b0 0x7f315019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:41.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f3147e00640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 0x7f315019e9a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f313800e970 tx=0x7f313800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f3146a00640 1 -- 172.21.15.44:0/489974763 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313800cd80 con 0x7f31501071c0 2024-04-20T11:30:41.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3150199420 con 0x7f31501071c0 2024-04-20T11:30:41.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f3146a00640 1 -- 172.21.15.44:0/489974763 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31380035f0 con 0x7f31501071c0 2024-04-20T11:30:41.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f3146a00640 1 -- 172.21.15.44:0/489974763 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138010690 con 0x7f31501071c0 2024-04-20T11:30:41.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.973+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31501a56c0 con 0x7f31501071c0 2024-04-20T11:30:41.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 Processor -- start 2024-04-20T11:30:41.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3146a00640 1 -- 172.21.15.44:0/489974763 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f31380037a0 con 0x7f31501071c0 2024-04-20T11:30:41.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 -- start start 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965f9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 0x55e965fa5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965fa5880 0x55e965fa7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e965f02150 con 0x55e965f9c380 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e965ea8370 con 0x55e965fa5880 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.977+0000 7f3157e59740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e965efe110 con 0x55e965f9cca0 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965f9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 0x55e965fa5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 0x55e965fa5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59606/0 (socket says 172.21.15.44:59606) 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 -- 172.21.15.44:0/1387400132 learned_addr learned my addr 172.21.15.44:0/1387400132 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f3147e00640 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965fa5880 0x55e965fa7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 -- 172.21.15.44:0/1387400132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965fa5880 msgr2=0x55e965fa7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965fa5880 0x55e965fa7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 -- 172.21.15.44:0/1387400132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 msgr2=0x55e965f9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965f9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.978+0000 7f314de00640 1 -- 172.21.15.44:0/1387400132 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e965cde7d0 con 0x55e965f9cca0 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f314de00640 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 0x55e965fa5340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f31400095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb192211f1e2fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f3146a00640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31340a3d20 0x7f31340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f3146a00640 1 -- 172.21.15.44:0/489974763 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f31380d0a70 con 0x7f31501071c0 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f313f400640 1 -- 172.21.15.44:0/1387400132 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140043070 con 0x55e965f9cca0 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f314e800640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31340a3d20 0x7f31340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f313f400640 1 -- 172.21.15.44:0/1387400132 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f314002c6f0 con 0x55e965f9cca0 2024-04-20T11:30:41.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.979+0000 7f313f400640 1 -- 172.21.15.44:0/1387400132 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140003b30 con 0x55e965f9cca0 2024-04-20T11:30:41.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 -- 172.21.15.44:0/1387400132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 msgr2=0x55e965fa5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f314e800640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31340a3d20 0x7f31340a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3148005fd0 tx=0x7f3148009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 0x55e965fa5340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f31400095f0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 -- 172.21.15.44:0/1387400132 shutdown_connections 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965fa5880 0x55e965fa7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9cca0 0x55e965fa5340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 --2- 172.21.15.44:0/1387400132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965f9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 -- 172.21.15.44:0/1387400132 >> 172.21.15.44:0/1387400132 conn(0x55e965f96ab0 msgr2=0x55e965f96e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 -- 172.21.15.44:0/1387400132 shutdown_connections 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.980+0000 7f3157e59740 1 -- 172.21.15.44:0/1387400132 wait complete. 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 -- start start 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 Processor -- start 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9c380 0x55e965fa28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 0x55e965fa2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fa3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e965fa8610 con 0x55e965f9cca0 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e965ebb740 con 0x55e965f9c380 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e965fa8790 con 0x55e965fa5880 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.981+0000 7f3157e59740 10 client.0 _ll_register_callbacks cb 0x55e965eb6b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 0x55e965fa2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 0x55e965fa2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38106/0 (socket says 172.21.15.44:38106) 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 -- 172.21.15.44:0/3544377325 learned_addr learned my addr 172.21.15.44:0/3544377325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f314de00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9c380 0x55e965fa28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f314e800640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fa3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 -- 172.21.15.44:0/3544377325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 msgr2=0x55e965fa3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fa3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 -- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9c380 msgr2=0x55e965fa28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9c380 0x55e965fa28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 -- 172.21.15.44:0/3544377325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e965c143f0 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f314de00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9c380 0x55e965fa28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3147e00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 0x55e965fa2df0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f313809c9e0 tx=0x7f3138009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671123140fe911bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.982+0000 7f3133e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3133e00640 1 -- 172.21.15.44:0/3544377325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313809c3b0 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3133e00640 1 -- 172.21.15.44:0/3544377325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f313809ad50 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f314002c6f0 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3133e00640 1 -- 172.21.15.44:0/3544377325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f313809d150 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3157e59740 10 client.17580 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e965bd40f0 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3133e00640 1 -- 172.21.15.44:0/3544377325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f314002c6f0 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3133e00640 1 -- 172.21.15.44:0/3544377325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f31380990c0 con 0x55e965f9cca0 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3157e59740 10 client.17580 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.983+0000 7f3157e59740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f3157e59740 20 client.17580 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f3157e59740 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e965fb9f50 0x55e965fbc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f3157e59740 4 client.17580 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e965f0fb70 con 0x55e965fb9f50 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f314de00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e965fb9f50 0x55e965fbc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f314de00640 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e965fb9f50 0x55e965fbc340 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492801649e482630 server_cookie=eb7efdbe0e93a58 in_seq=0 out_seq=0 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.984+0000 7f3133e00640 10 client.17580 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3133e00640 1 -- 172.21.15.44:0/3544377325 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e965f0fb70 con 0x55e965fb9f50 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3133e00640 10 client.17580 handle_command_reply: tid=0 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 client.17580 shutdown 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 msgr2=0x55e965fa2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 0x55e965fa2df0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f313809c9e0 tx=0x7f3138009550 comp rx=0 tx=0).stop 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 shutdown_connections 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e965fb9f50 0x55e965fbc340 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fa3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9cca0 0x55e965fa2df0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 --2- 172.21.15.44:0/3544377325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9c380 0x55e965fa28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3133e00640 0 client.17580 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.985+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 >> 172.21.15.44:0/3544377325 conn(0x55e965f96ab0 msgr2=0x55e965f98800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.986+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 shutdown_connections 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.986+0000 7f3157e59740 1 -- 172.21.15.44:0/3544377325 wait complete. 2024-04-20T11:30:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:41.986+0000 7f3157e59740 20 client.17580 trim_cache size 0 max 16384 2024-04-20T11:30:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.005+0000 7f3157e59740 1 Processor -- start 2024-04-20T11:30:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.005+0000 7f3157e59740 1 -- start start 2024-04-20T11:30:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.005+0000 7f3157e59740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9c380 0x55e965fa4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.005+0000 7f3157e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965f96e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.005+0000 7f3157e59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965fa5880 0x55e965fa2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3157e59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31501ab930 con 0x55e965fa5880 2024-04-20T11:30:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3157e59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31501ad470 con 0x55e965f9cca0 2024-04-20T11:30:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3157e59740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31501ac000 con 0x55e965f9c380 2024-04-20T11:30:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965f96e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965f96e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:59588/0 (socket says 172.21.15.44:59588) 2024-04-20T11:30:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 -- 172.21.15.44:0/2846997363 learned_addr learned my addr 172.21.15.44:0/2846997363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f314de00640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9c380 0x55e965fa4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f314e800640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965fa5880 0x55e965fa2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 -- 172.21.15.44:0/2846997363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9c380 msgr2=0x55e965fa4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9c380 0x55e965fa4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 -- 172.21.15.44:0/2846997363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965fa5880 msgr2=0x55e965fa2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965fa5880 0x55e965fa2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f3147e00640 1 -- 172.21.15.44:0/2846997363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e965f9cad0 con 0x55e965f9cca0 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f314de00640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9c380 0x55e965fa4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.006+0000 7f314e800640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965fa5880 0x55e965fa2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.007+0000 7f3147e00640 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965f96e90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f31380ce800 tx=0x7f3138087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f3e7cede2045fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.007+0000 7f313ea00640 1 -- 172.21.15.44:0/2846997363 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31380a0e00 con 0x55e965f9cca0 2024-04-20T11:30:42.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.007+0000 7f313ea00640 1 -- 172.21.15.44:0/2846997363 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31380a56c0 con 0x55e965f9cca0 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.007+0000 7f313ea00640 1 -- 172.21.15.44:0/2846997363 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313809bfe0 con 0x55e965f9cca0 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 -- 172.21.15.44:0/2846997363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 msgr2=0x55e965f96e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965f96e90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f31380ce800 tx=0x7f3138087040 comp rx=0 tx=0).stop 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 -- 172.21.15.44:0/2846997363 shutdown_connections 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965fa5880 0x55e965fa2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965f96e90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 --2- 172.21.15.44:0/2846997363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965f9c380 0x55e965fa4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 -- 172.21.15.44:0/2846997363 >> 172.21.15.44:0/2846997363 conn(0x55e965f96ab0 msgr2=0x55e965fa2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 -- 172.21.15.44:0/2846997363 shutdown_connections 2024-04-20T11:30:42.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 -- 172.21.15.44:0/2846997363 wait complete. 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 -- start start 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.008+0000 7f3157e59740 1 Processor -- start 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965fb21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965fad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31501acda0 con 0x55e965f9c380 2024-04-20T11:30:42.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31501ac6d0 con 0x55e965f9cca0 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31501a9920 con 0x55e965fa5880 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3157e59740 10 client.0 _ll_register_callbacks cb 0x7f3138003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f314de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965fb21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f314de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965fb21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38124/0 (socket says 172.21.15.44:38124) 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f314de00640 1 -- 172.21.15.44:0/3679031186 learned_addr learned my addr 172.21.15.44:0/3679031186 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f3147e00640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965fad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.009+0000 7f314e800640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314de00640 1 -- 172.21.15.44:0/3679031186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 msgr2=0x55e965fada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314de00640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314de00640 1 -- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 msgr2=0x55e965fad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314de00640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965fad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314de00640 1 -- 172.21.15.44:0/3679031186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e965fae280 con 0x55e965f9c380 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314e800640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:30:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f314de00640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965fb21f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f31400037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86de0c489c7afc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f3133e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f3133e00640 1 -- 172.21.15.44:0/3679031186 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140043040 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f3133e00640 1 -- 172.21.15.44:0/3679031186 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8455+0+0 (secure 0 0 0) 0x7f3140034760 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31380a56c0 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.010+0000 7f3133e00640 1 -- 172.21.15.44:0/3679031186 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3140034960 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 10 client.17586 fetch_fsmap learned FSMap version 55 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e966020410 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3133e00640 1 -- 172.21.15.44:0/3679031186 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31380a56c0 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3133e00640 1 -- 172.21.15.44:0/3679031186 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3140048b80 con 0x55e965f9c380 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 10 client.17586 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 20 client.17586 populate_metadata read hostname 'smithi044' 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e9660207a0 0x55e966022b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 4 client.17586 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:30:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.011+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e965f099e0 con 0x55e9660207a0 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.012+0000 7f3147e00640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e9660207a0 0x55e966022b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.012+0000 7f3147e00640 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e9660207a0 0x55e966022b90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfd291a2d394035 server_cookie=40d10aa4b4e08a22 in_seq=0 out_seq=0 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.012+0000 7f3133e00640 10 client.17586 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.012+0000 7f3133e00640 1 -- 172.21.15.44:0/3679031186 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e965f099e0 con 0x55e9660207a0 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.012+0000 7f3133e00640 10 client.17586 handle_command_reply: tid=0 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.012+0000 7f3157e59740 1 client.17586 shutdown 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 msgr2=0x55e965fb21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965fb21f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f31400037d0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 shutdown_connections 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e9660207a0 0x55e966022b90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e965fa5880 0x55e965fada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e965f9cca0 0x55e965fad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3133e00640 0 client.17586 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:30:42.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 --2- 172.21.15.44:0/3679031186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e965f9c380 0x55e965fb21f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 >> 172.21.15.44:0/3679031186 conn(0x55e965f96ab0 msgr2=0x55e965fa3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 shutdown_connections 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 1 -- 172.21.15.44:0/3679031186 wait complete. 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.013+0000 7f3157e59740 20 client.17586 trim_cache size 0 max 16384 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stdout: "b0be046c-a704-427c-a0b5-1355bfd4f11d": { 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:30:42.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "b0be046c-a704-427c-a0b5-1355bfd4f11d", 2024-04-20T11:30:42.019 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:30:42.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:30:42.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:30:42.019 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:30:42.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31340a3d20 msgr2=0x7f31340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31340a3d20 0x7f31340a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3148005fd0 tx=0x7f3148009500 comp rx=0 tx=0).stop 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 msgr2=0x7f315019e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 0x7f315019e9a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f313800e970 tx=0x7f313800ee30 comp rx=0 tx=0).stop 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 shutdown_connections 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3150198610 0x7f3150198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31340a3d20 0x7f31340a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31501071c0 0x7f315019e9a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 --2- 172.21.15.44:0/489974763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31501009b0 0x7f315019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 >> 172.21.15.44:0/489974763 conn(0x7f3150075ff0 msgr2=0x7f31500fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 shutdown_connections 2024-04-20T11:30:42.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:30:42.014+0000 7f314f200640 1 -- 172.21.15.44:0/489974763 wait complete. 2024-04-20T11:30:42.031 INFO:tasks.cephfs.filesystem:scrub status for tag:b0be046c-a704-427c-a0b5-1355bfd4f11d - {'path': '/', 'tag': 'b0be046c-a704-427c-a0b5-1355bfd4f11d', 'options': 'recursive,force'} 2024-04-20T11:30:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:43 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:30:43.689 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:43 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:43.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:43 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:30:43.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:43 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:43.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:43 smithi078 ceph-mon[21464]: pgmap v250: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:43.690 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:43 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:30:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:43 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:30:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:43 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:43 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:30:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:43 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:43 smithi044 ceph-mon[16855]: pgmap v250: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:43 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:30:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:43 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:30:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:43 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:43 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:30:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:43 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:30:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:43 smithi165 ceph-mon[21975]: pgmap v250: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:43 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:30:45.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:45 smithi078 ceph-mon[21464]: pgmap v251: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:45.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:45 smithi044 ceph-mon[16855]: pgmap v251: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:45 smithi165 ceph-mon[21975]: pgmap v251: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:47.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:47 smithi078 ceph-mon[21464]: pgmap v252: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:47 smithi044 ceph-mon[16855]: pgmap v252: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:47 smithi165 ceph-mon[21975]: pgmap v252: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:49.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:49 smithi078 ceph-mon[21464]: pgmap v253: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:49.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:49 smithi044 ceph-mon[16855]: pgmap v253: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:49 smithi165 ceph-mon[21975]: pgmap v253: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:51.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:51 smithi078 ceph-mon[21464]: pgmap v254: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:51.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:51 smithi044 ceph-mon[16855]: pgmap v254: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:51.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:51 smithi165 ceph-mon[21975]: pgmap v254: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:30:53.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:53 smithi078 ceph-mon[21464]: pgmap v255: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:53.698 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:53 smithi044 ceph-mon[16855]: pgmap v255: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:53.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:53 smithi165 ceph-mon[21975]: pgmap v255: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:30:55.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:55 smithi078 ceph-mon[21464]: pgmap v256: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:55 smithi044 ceph-mon[16855]: pgmap v256: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:55.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:55 smithi165 ceph-mon[21975]: pgmap v256: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:57.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:57 smithi078 ceph-mon[21464]: pgmap v257: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:57.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:57 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:30:57.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:57 smithi044 ceph-mon[16855]: pgmap v257: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:57.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:57 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:30:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:57 smithi165 ceph-mon[21975]: pgmap v257: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:30:57.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:57 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:30:58.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:58 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:30:58.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:58 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:30:58.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:58 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:30:58.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:58 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:30:58.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:58 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:30:58.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:58 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:30:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:58 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:30:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:58 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:30:58.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:58 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:30:59.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:30:59 smithi078 ceph-mon[21464]: pgmap v258: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:59.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:30:59 smithi044 ceph-mon[16855]: pgmap v258: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:30:59.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:30:59 smithi165 ceph-mon[21975]: pgmap v258: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:01.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:01 smithi078 ceph-mon[21464]: pgmap v259: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:01.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:01 smithi044 ceph-mon[16855]: pgmap v259: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:01.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:01 smithi165 ceph-mon[21975]: pgmap v259: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:03.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:03 smithi044 ceph-mon[16855]: pgmap v260: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:31:03.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:03 smithi165 ceph-mon[21975]: pgmap v260: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:31:03.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:03 smithi078 ceph-mon[21464]: pgmap v260: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:31:05.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:05 smithi165 ceph-mon[21975]: pgmap v261: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:31:05.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:05 smithi078 ceph-mon[21464]: pgmap v261: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:31:05.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:05 smithi044 ceph-mon[16855]: pgmap v261: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:31:07.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:07 smithi165 ceph-mon[21975]: pgmap v262: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:31:07.739 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:07 smithi078 ceph-mon[21464]: pgmap v262: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:31:07.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:07 smithi044 ceph-mon[16855]: pgmap v262: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:31:08.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:08 smithi165 ceph-mon[21975]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T11:31:08.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:08 smithi078 ceph-mon[21464]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T11:31:08.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:08 smithi044 ceph-mon[16855]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T11:31:09.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:09 smithi165 ceph-mon[21975]: pgmap v264: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:31:09.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:09 smithi078 ceph-mon[21464]: pgmap v264: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:31:09.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:09 smithi044 ceph-mon[16855]: pgmap v264: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:31:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:11 smithi165 ceph-mon[21975]: pgmap v265: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:31:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:11 smithi078 ceph-mon[21464]: pgmap v265: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:31:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:11 smithi044 ceph-mon[16855]: pgmap v265: 97 pgs: 97 active+clean; 267 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:31:12.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:31:12.032 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:31:12.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 -- 172.21.15.44:0/2846158279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc102a20 msgr2=0x7ff7dc106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/2846158279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc102a20 0x7ff7dc106a60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009900 tx=0x7ff7c802bc50 comp rx=0 tx=0).stop 2024-04-20T11:31:12.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 -- 172.21.15.44:0/2846158279 shutdown_connections 2024-04-20T11:31:12.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/2846158279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7dc107140 0x7ff7dc10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/2846158279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc102a20 0x7ff7dc106a60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/2846158279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 -- 172.21.15.44:0/2846158279 >> 172.21.15.44:0/2846158279 conn(0x7ff7dc0fd930 msgr2=0x7ff7dc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 -- 172.21.15.44:0/2846158279 shutdown_connections 2024-04-20T11:31:12.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.262+0000 7ff7e2a00640 1 -- 172.21.15.44:0/2846158279 wait complete. 2024-04-20T11:31:12.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.263+0000 7ff7e2a00640 1 Processor -- start 2024-04-20T11:31:12.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.263+0000 7ff7e2a00640 1 -- start start 2024-04-20T11:31:12.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7dc107140 0x7ff7dc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc198610 0x7ff7dc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1122f0 con 0x7ff7dc198610 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc112170 con 0x7ff7dc102070 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc112470 con 0x7ff7dc107140 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc198610 0x7ff7dc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41778/0 (socket says 172.21.15.44:41778) 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e1600640 1 -- 172.21.15.44:0/4054930894 learned_addr learned my addr 172.21.15.44:0/4054930894 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.264+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7dc107140 0x7ff7dc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e1600640 1 -- 172.21.15.44:0/4054930894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7dc107140 msgr2=0x7ff7dc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e1600640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7dc107140 0x7ff7dc19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e1600640 1 -- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc198610 msgr2=0x7ff7dc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e1600640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc198610 0x7ff7dc198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e1600640 1 -- 172.21.15.44:0/4054930894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc199130 con 0x7ff7dc102070 2024-04-20T11:31:12.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e1600640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc19e460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000ea60 tx=0x7ff7d000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7da000640 1 -- 172.21.15.44:0/4054930894 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000cd20 con 0x7ff7dc102070 2024-04-20T11:31:12.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc199420 con 0x7ff7dc102070 2024-04-20T11:31:12.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7da000640 1 -- 172.21.15.44:0/4054930894 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff7d00035f0 con 0x7ff7dc102070 2024-04-20T11:31:12.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7da000640 1 -- 172.21.15.44:0/4054930894 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0010690 con 0x7ff7dc102070 2024-04-20T11:31:12.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.265+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc1a56c0 con 0x7ff7dc102070 2024-04-20T11:31:12.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 Processor -- start 2024-04-20T11:31:12.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 -- start start 2024-04-20T11:31:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd43340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd45c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563e8bca0150 con 0x563e8bd3a380 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563e8bc46370 con 0x563e8bd43880 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7da000640 1 -- 172.21.15.44:0/4054930894 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff7d00037a0 con 0x7ff7dc102070 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e8bc9c110 con 0x563e8bd3aca0 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.269+0000 7ff7e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd43340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd3a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42880/0 (socket says 172.21.15.44:42880) 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 -- 172.21.15.44:0/878497713 learned_addr learned my addr 172.21.15.44:0/878497713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd45c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 -- 172.21.15.44:0/878497713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 msgr2=0x563e8bd43340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd43340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 -- 172.21.15.44:0/878497713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 msgr2=0x563e8bd45c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd45c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 -- 172.21.15.44:0/878497713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8ba7c7d0 con 0x563e8bd3a380 2024-04-20T11:31:12.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e1600640 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd43340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:12.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.270+0000 7ff7e2000640 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd3a760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009f60 tx=0x7ff7d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6315c720c9356f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7ce600640 1 -- 172.21.15.44:0/878497713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d401e070 con 0x563e8bd3a380 2024-04-20T11:31:12.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7ce600640 1 -- 172.21.15.44:0/878497713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff7d4003c30 con 0x563e8bd3a380 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7ce600640 1 -- 172.21.15.44:0/878497713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400f9e0 con 0x563e8bd3a380 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7da000640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7c40a3da0 0x7ff7c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7da000640 1 -- 172.21.15.44:0/4054930894 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8499+0+0 (secure 0 0 0) 0x7ff7d00d0b60 con 0x7ff7dc102070 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7e1600640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7c40a3da0 0x7ff7c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7e6c29740 1 -- 172.21.15.44:0/878497713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 msgr2=0x563e8bd3a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.271+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd3a760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009f60 tx=0x7ff7d4009f90 comp rx=0 tx=0).stop 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 -- 172.21.15.44:0/878497713 shutdown_connections 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd45c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd43340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/878497713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd3a760 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 -- 172.21.15.44:0/878497713 >> 172.21.15.44:0/878497713 conn(0x563e8bd34ab0 msgr2=0x563e8bd34e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 -- 172.21.15.44:0/878497713 shutdown_connections 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 -- 172.21.15.44:0/878497713 wait complete. 2024-04-20T11:31:12.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e1600640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7c40a3da0 0x7ff7c40a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563e8bd39e50 tx=0x7ff7d00d4610 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.272+0000 7ff7e6c29740 1 -- start start 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 Processor -- start 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd40910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563e8bd46670 con 0x563e8bd3a380 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563e8bc59740 con 0x563e8bd43880 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e8bd467f0 con 0x563e8bd3aca0 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e6c29740 10 client.0 _ll_register_callbacks cb 0x563e8bc54b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd40910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd40910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42894/0 (socket says 172.21.15.44:42894) 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e2000640 1 -- 172.21.15.44:0/2506771174 learned_addr learned my addr 172.21.15.44:0/2506771174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.273+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 msgr2=0x563e8bd41390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd41390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2506771174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 msgr2=0x563e8bd40910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd40910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2506771174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8b9b23f0 con 0x563e8bd3aca0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd41390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd40e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8036040 tx=0x7ff7c802dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3836473886233cca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.274+0000 7ff7c2a00640 1 -- 172.21.15.44:0/2506771174 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8044070 con 0x563e8bd3aca0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d4003c30 con 0x563e8bd3aca0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7c2a00640 1 -- 172.21.15.44:0/2506771174 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8499+0+0 (secure 0 0 0) 0x7ff7c8004310 con 0x563e8bd3aca0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7c2a00640 1 -- 172.21.15.44:0/2506771174 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff7c8050050 con 0x563e8bd3aca0 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 10 client.27721 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:12.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563e8b9720f0 con 0x563e8bd3aca0 2024-04-20T11:31:12.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7c2a00640 1 -- 172.21.15.44:0/2506771174 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff7d4003c30 con 0x563e8bd3aca0 2024-04-20T11:31:12.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7c2a00640 1 -- 172.21.15.44:0/2506771174 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7ff7c8004cc0 con 0x563e8bd3aca0 2024-04-20T11:31:12.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 10 client.27721 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:12.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 10 client.27721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:12.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 20 client.27721 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bd58010 0x563e8bd5a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 4 client.27721 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.275+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e8bcadb70 con 0x563e8bd58010 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.276+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bd58010 0x563e8bd5a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.276+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bd58010 0x563e8bd5a400 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eda7f009a274f20 server_cookie=7f2b28a4d847f962 in_seq=0 out_seq=0 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.276+0000 7ff7c2a00640 10 client.27721 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.277+0000 7ff7c2a00640 1 -- 172.21.15.44:0/2506771174 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e8bcadb70 con 0x563e8bd58010 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.277+0000 7ff7c2a00640 10 client.27721 handle_command_reply: tid=0 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.277+0000 7ff7e6c29740 1 client.27721 shutdown 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 msgr2=0x563e8bd40e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd40e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8036040 tx=0x7ff7c802dd70 comp rx=0 tx=0).stop 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 shutdown_connections 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bd58010 0x563e8bd5a400 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd43880 0x563e8bd41390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd40e50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2506771174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd3a380 0x563e8bd40910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7c2a00640 0 client.27721 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 >> 172.21.15.44:0/2506771174 conn(0x563e8bd34ab0 msgr2=0x563e8bd36800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 shutdown_connections 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2506771174 wait complete. 2024-04-20T11:31:12.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.278+0000 7ff7e6c29740 20 client.27721 trim_cache size 0 max 16384 2024-04-20T11:31:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.297+0000 7ff7e6c29740 1 Processor -- start 2024-04-20T11:31:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 -- start start 2024-04-20T11:31:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3a380 0x563e8bd539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ab920 con 0x563e8bd43880 2024-04-20T11:31:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ad460 con 0x563e8bd3a380 2024-04-20T11:31:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1abff0 con 0x563e8bd3aca0 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd41160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58508/0 (socket says 172.21.15.44:58508) 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2192617723 learned_addr learned my addr 172.21.15.44:0/2192617723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3a380 0x563e8bd539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.298+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2192617723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3a380 msgr2=0x563e8bd539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3a380 0x563e8bd539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2192617723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 msgr2=0x563e8bd5fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd5fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e0c00640 1 -- 172.21.15.44:0/2192617723 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8bd5a770 con 0x563e8bd3aca0 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e2000640 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3a380 0x563e8bd539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd41160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009850 tx=0x7ff7c8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b67e1aff4afaf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7cdc00640 1 -- 172.21.15.44:0/2192617723 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8044040 con 0x563e8bd3aca0 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7cdc00640 1 -- 172.21.15.44:0/2192617723 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff7c8003830 con 0x563e8bd3aca0 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.299+0000 7ff7cdc00640 1 -- 172.21.15.44:0/2192617723 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8005280 con 0x563e8bd3aca0 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2192617723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 msgr2=0x563e8bd41160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd41160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009850 tx=0x7ff7c8009650 comp rx=0 tx=0).stop 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2192617723 shutdown_connections 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd5fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3aca0 0x563e8bd41160 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/2192617723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3a380 0x563e8bd539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2192617723 >> 172.21.15.44:0/2192617723 conn(0x563e8bd34ab0 msgr2=0x563e8bd35db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2192617723 shutdown_connections 2024-04-20T11:31:12.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.300+0000 7ff7e6c29740 1 -- 172.21.15.44:0/2192617723 wait complete. 2024-04-20T11:31:12.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 -- start start 2024-04-20T11:31:12.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 Processor -- start 2024-04-20T11:31:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3a380 0x563e8bd4fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 0x563e8bd50410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd4cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1acd90 con 0x563e8bd43880 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ac6c0 con 0x563e8bd3aca0 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1a9930 con 0x563e8bd3a380 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.301+0000 7ff7e6c29740 10 client.0 _ll_register_callbacks cb 0x7ff7c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 0x563e8bd50410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd4cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 0x563e8bd50410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41832/0 (socket says 172.21.15.44:41832) 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 -- 172.21.15.44:0/1690214516 learned_addr learned my addr 172.21.15.44:0/1690214516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e2000640 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3a380 0x563e8bd4fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 -- 172.21.15.44:0/1690214516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3a380 msgr2=0x563e8bd4fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3a380 0x563e8bd4fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 -- 172.21.15.44:0/1690214516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 msgr2=0x563e8bd4cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd4cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.302+0000 7ff7e0c00640 1 -- 172.21.15.44:0/1690214516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8bd4d670 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7e0c00640 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 0x563e8bd50410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80045c0 tx=0x7ff7c80045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c3abe28d85f277 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7c2a00640 1 -- 172.21.15.44:0/1690214516 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8044070 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7c2a00640 1 -- 172.21.15.44:0/1690214516 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8499+0+0 (secure 0 0 0) 0x7ff7c802cbf0 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c8003830 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7c2a00640 1 -- 172.21.15.44:0/1690214516 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff7c8065050 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7e6c29740 10 client.27725 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.303+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563e8bdbefa0 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7c2a00640 1 -- 172.21.15.44:0/1690214516 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff7c8003830 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7c2a00640 1 -- 172.21.15.44:0/1690214516 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7ff7c8004c70 con 0x563e8bd3aca0 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7e6c29740 10 client.27725 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:12.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7e6c29740 10 client.27725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7e6c29740 20 client.27725 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bdbf610 0x563e8bdc1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7e6c29740 4 client.27725 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.304+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e8bca79e0 con 0x563e8bdbf610 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.305+0000 7ff7e2000640 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bdbf610 0x563e8bdc1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.305+0000 7ff7e2000640 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bdbf610 0x563e8bdc1a30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2e5fbc2067f035 server_cookie=ab959e6f2957e6d in_seq=0 out_seq=0 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.305+0000 7ff7c2a00640 10 client.27725 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.305+0000 7ff7c2a00640 1 -- 172.21.15.44:0/1690214516 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563e8bca79e0 con 0x563e8bdbf610 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.305+0000 7ff7c2a00640 10 client.27725 handle_command_reply: tid=0 2024-04-20T11:31:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.305+0000 7ff7e6c29740 1 client.27725 shutdown 2024-04-20T11:31:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 msgr2=0x563e8bd50410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 0x563e8bd50410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80045c0 tx=0x7ff7c80045f0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 shutdown_connections 2024-04-20T11:31:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x563e8bdbf610 0x563e8bdc1a30 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563e8bd43880 0x563e8bd4cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e8bd3aca0 0x563e8bd50410 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7c2a00640 0 client.27725 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 --2- 172.21.15.44:0/1690214516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e8bd3a380 0x563e8bd4fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 >> 172.21.15.44:0/1690214516 conn(0x563e8bd34ab0 msgr2=0x563e8bd40fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 shutdown_connections 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 1 -- 172.21.15.44:0/1690214516 wait complete. 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.306+0000 7ff7e6c29740 20 client.27725 trim_cache size 0 max 16384 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:31:12.310 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:31:12.311 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:31:12.311 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:31:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7c40a3da0 msgr2=0x7ff7c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7c40a3da0 0x7ff7c40a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563e8bd39e50 tx=0x7ff7d00d4610 comp rx=0 tx=0).stop 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 msgr2=0x7ff7dc19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc19e460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000ea60 tx=0x7ff7d000ee60 comp rx=0 tx=0).stop 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 shutdown_connections 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7c40a3da0 0x7ff7c40a6260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7dc198610 0x7ff7dc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7dc107140 0x7ff7dc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 --2- 172.21.15.44:0/4054930894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7dc102070 0x7ff7dc19e460 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 >> 172.21.15.44:0/4054930894 conn(0x7ff7dc0fd930 msgr2=0x7ff7dc0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 shutdown_connections 2024-04-20T11:31:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.307+0000 7ff7e2a00640 1 -- 172.21.15.44:0/4054930894 wait complete. 2024-04-20T11:31:12.330 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:31:12.330 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 -- 172.21.15.44:0/1625320576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb804108f50 msgr2=0x7fb804115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 --2- 172.21.15.44:0/1625320576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb804108f50 0x7fb804115410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009ca0 tx=0x7fb7fc02c100 comp rx=0 tx=0).stop 2024-04-20T11:31:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 -- 172.21.15.44:0/1625320576 shutdown_connections 2024-04-20T11:31:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 --2- 172.21.15.44:0/1625320576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb804108f50 0x7fb804115410 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 --2- 172.21.15.44:0/1625320576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb804108590 0x7fb804108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 --2- 172.21.15.44:0/1625320576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb80410e650 0x7fb80410ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 -- 172.21.15.44:0/1625320576 >> 172.21.15.44:0/1625320576 conn(0x7fb804075760 msgr2=0x7fb804106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 -- 172.21.15.44:0/1625320576 shutdown_connections 2024-04-20T11:31:12.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.533+0000 7fb80b400640 1 -- 172.21.15.44:0/1625320576 wait complete. 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.534+0000 7fb80b400640 1 Processor -- start 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.534+0000 7fb80b400640 1 -- start start 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.534+0000 7fb80b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 0x7fb80419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb80b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb804108f50 0x7fb80419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb80b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb80410e650 0x7fb8041a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb80b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb804118ab0 con 0x7fb80410e650 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb80b400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb804118930 con 0x7fb804108f50 2024-04-20T11:31:12.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb80b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb804118c30 con 0x7fb804108590 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb809400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb80410e650 0x7fb8041a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 0x7fb80419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 0x7fb80419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58530/0 (socket says 172.21.15.44:58530) 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 -- 172.21.15.44:0/4219945077 learned_addr learned my addr 172.21.15.44:0/4219945077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb803e00640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb804108f50 0x7fb80419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 -- 172.21.15.44:0/4219945077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb804108f50 msgr2=0x7fb80419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb804108f50 0x7fb80419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 -- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb80410e650 msgr2=0x7fb8041a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb80410e650 0x7fb8041a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.535+0000 7fb808a00640 1 -- 172.21.15.44:0/4219945077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8041a3dc0 con 0x7fb804108590 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.536+0000 7fb808a00640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 0x7fb80419edd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800ea60 tx=0x7fb7f800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.536+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800cde0 con 0x7fb804108590 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.536+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb7f80035f0 con 0x7fb804108590 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.536+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8010720 con 0x7fb804108590 2024-04-20T11:31:12.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.536+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8041a40b0 con 0x7fb804108590 2024-04-20T11:31:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.536+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8041ab940 con 0x7fb804108590 2024-04-20T11:31:12.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.537+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8040691d0 con 0x7fb804108590 2024-04-20T11:31:12.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.539+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb7f80037a0 con 0x7fb804108590 2024-04-20T11:31:12.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.541+0000 7fb801600640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb7ec0a3da0 0x7fb7ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.541+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8499+0+0 (secure 0 0 0) 0x7fb7f80d0b90 con 0x7fb804108590 2024-04-20T11:31:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.541+0000 7fb803e00640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb7ec0a3da0 0x7fb7ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:12.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.542+0000 7fb803e00640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb7ec0a3da0 0x7fb7ec0a6260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb804075cf0 tx=0x7fb7f0005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:12.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.546+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb7f809d1e0 con 0x7fb804108590 2024-04-20T11:31:12.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.792+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8041abc30 con 0x7fb804108590 2024-04-20T11:31:12.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.794+0000 7fb801600640 1 -- 172.21.15.44:0/4219945077 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fb7f809cae0 con 0x7fb804108590 2024-04-20T11:31:12.795 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:31:12.795 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:31:12.795 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:31:12.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.796+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb7ec0a3da0 msgr2=0x7fb7ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.796+0000 7fb80b400640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb7ec0a3da0 0x7fb7ec0a6260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb804075cf0 tx=0x7fb7f0005f90 comp rx=0 tx=0).stop 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.796+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 msgr2=0x7fb80419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.796+0000 7fb80b400640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 0x7fb80419edd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800ea60 tx=0x7fb7f800ef20 comp rx=0 tx=0).stop 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 shutdown_connections 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fb7ec0a3da0 0x7fb7ec0a6260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb80410e650 0x7fb8041a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb804108f50 0x7fb80419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 --2- 172.21.15.44:0/4219945077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb804108590 0x7fb80419edd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 >> 172.21.15.44:0/4219945077 conn(0x7fb804075760 msgr2=0x7fb80410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 shutdown_connections 2024-04-20T11:31:12.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:12.797+0000 7fb80b400640 1 -- 172.21.15.44:0/4219945077 wait complete. 2024-04-20T11:31:12.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 -- 172.21.15.44:0/1929963181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 msgr2=0x7f6d20108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/1929963181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 0x7f6d20108960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009900 tx=0x7f6d0802bcc0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 -- 172.21.15.44:0/1929963181 shutdown_connections 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/1929963181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d20108ea0 0x7f6d201152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/1929963181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 0x7f6d20108960 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/1929963181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d2010d3d0 0x7f6d2010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 -- 172.21.15.44:0/1929963181 >> 172.21.15.44:0/1929963181 conn(0x7f6d20077d80 msgr2=0x7f6d20106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 -- 172.21.15.44:0/1929963181 shutdown_connections 2024-04-20T11:31:13.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.030+0000 7f6d1fe00640 1 -- 172.21.15.44:0/1929963181 wait complete. 2024-04-20T11:31:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 Processor -- start 2024-04-20T11:31:13.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 -- start start 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 0x7f6d2019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 0x7f6d2019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 0x7f6d201a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20118950 con 0x7f6d20108ea0 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201187d0 con 0x7f6d20108500 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.031+0000 7f6d1fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20118ad0 con 0x7f6d2010d3d0 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 0x7f6d2019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 0x7f6d201a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 0x7f6d201a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58544/0 (socket says 172.21.15.44:58544) 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 -- 172.21.15.44:0/3587208868 learned_addr learned my addr 172.21.15.44:0/3587208868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 0x7f6d2019f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42944/0 (socket says 172.21.15.44:42944) 2024-04-20T11:31:13.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 0x7f6d2019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 -- 172.21.15.44:0/3587208868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 msgr2=0x7f6d2019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 0x7f6d2019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 -- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 msgr2=0x7f6d2019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 0x7f6d2019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d1f400640 1 -- 172.21.15.44:0/3587208868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d201a3da0 con 0x7f6d2010d3d0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.032+0000 7f6d17e00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 0x7f6d2019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.033+0000 7f6d1f400640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 0x7f6d201a3670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800ea60 tx=0x7f6d1800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.033+0000 7f6d17400640 1 -- 172.21.15.44:0/3587208868 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800ce40 con 0x7f6d2010d3d0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.033+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d201a4090 con 0x7f6d2010d3d0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.033+0000 7f6d17400640 1 -- 172.21.15.44:0/3587208868 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6d18004b20 con 0x7f6d2010d3d0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.033+0000 7f6d17400640 1 -- 172.21.15.44:0/3587208868 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18010760 con 0x7f6d2010d3d0 2024-04-20T11:31:13.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.033+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d201ab920 con 0x7f6d2010d3d0 2024-04-20T11:31:13.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.036+0000 7f6d283fd740 1 Processor -- start 2024-04-20T11:31:13.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.036+0000 7f6d17400640 1 -- 172.21.15.44:0/3587208868 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6d18004cd0 con 0x7f6d2010d3d0 2024-04-20T11:31:13.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 -- start start 2024-04-20T11:31:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fafb543050 con 0x55fafb5e7670 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fafb4e9490 con 0x55fafb5dea90 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d283fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fafb53f010 con 0x55fafb5de170 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.037+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42948/0 (socket says 172.21.15.44:42948) 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d17e00640 1 -- 172.21.15.44:0/3966113138 learned_addr learned my addr 172.21.15.44:0/3966113138 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d17e00640 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 -- 172.21.15.44:0/3966113138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 msgr2=0x55fafb5de550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5de550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 -- 172.21.15.44:0/3966113138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 msgr2=0x55fafb5e9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 -- 172.21.15.44:0/3966113138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafb4f1070 con 0x55fafb5dea90 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e7130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005fd0 tx=0x7f6d100107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d375e612f4d3a3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.038+0000 7f6d17400640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6d040a3da0 0x7f6d040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d0fe00640 1 -- 172.21.15.44:0/3966113138 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1001c480 con 0x55fafb5dea90 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d0fe00640 1 -- 172.21.15.44:0/3966113138 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6d1001cb20 con 0x55fafb5dea90 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d0fe00640 1 -- 172.21.15.44:0/3966113138 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011ba0 con 0x55fafb5dea90 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d17400640 1 -- 172.21.15.44:0/3587208868 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8499+0+0 (secure 0 0 0) 0x7f6d180d0dd0 con 0x7f6d2010d3d0 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d17e00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6d040a3da0 0x7f6d040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d283fd740 1 -- 172.21.15.44:0/3966113138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 msgr2=0x55fafb5e7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d17e00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6d040a3da0 0x7f6d040a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d080096c0 tx=0x7f6d08005b70 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.039+0000 7f6d283fd740 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e7130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005fd0 tx=0x7f6d100107a0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 -- 172.21.15.44:0/3966113138 shutdown_connections 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e7130 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 --2- 172.21.15.44:0/3966113138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5de550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 -- 172.21.15.44:0/3966113138 >> 172.21.15.44:0/3966113138 conn(0x55fafb5d88a0 msgr2=0x55fafb5d8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 -- 172.21.15.44:0/3966113138 shutdown_connections 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 -- 172.21.15.44:0/3966113138 wait complete. 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 -- start start 2024-04-20T11:31:13.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.040+0000 7f6d283fd740 1 Processor -- start 2024-04-20T11:31:13.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fafb5ea460 con 0x55fafb5e7670 2024-04-20T11:31:13.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fafb4fc860 con 0x55fafb5dea90 2024-04-20T11:31:13.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fafb5ea5e0 con 0x55fafb5de170 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d283fd740 10 client.0 _ll_register_callbacks cb 0x55fafb4cafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41896/0 (socket says 172.21.15.44:41896) 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d17e00640 1 -- 172.21.15.44:0/2028064357 learned_addr learned my addr 172.21.15.44:0/2028064357 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.041+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1f400640 1 -- 172.21.15.44:0/2028064357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 msgr2=0x55fafb5e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1f400640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5e4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1f400640 1 -- 172.21.15.44:0/2028064357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 msgr2=0x55fafb5e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1f400640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1f400640 1 -- 172.21.15.44:0/2028064357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafb500950 con 0x55fafb5e7670 2024-04-20T11:31:13.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d17e00640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d1f400640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e51d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d180ce4d0 tx=0x7f6d1809d4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791f2d04f70e56ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d0cc00640 1 -- 172.21.15.44:0/2028064357 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180bd270 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d0cc00640 1 -- 172.21.15.44:0/2028064357 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8499+0+0 (secure 0 0 0) 0x7f6d1809c170 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d1001cb20 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d0cc00640 1 -- 172.21.15.44:0/2028064357 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6d180a5990 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d283fd740 10 client.17622 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.042+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55fafb4ce820 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d0cc00640 1 -- 172.21.15.44:0/2028064357 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6d1001cb20 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d0cc00640 1 -- 172.21.15.44:0/2028064357 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f6d180a2260 con 0x55fafb5e7670 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d283fd740 10 client.17622 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d283fd740 10 client.17622 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:31:13.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d283fd740 20 client.17622 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d283fd740 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb5f76c0 0x55fafb5f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d283fd740 4 client.17622 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fafb550a70 con 0x55fafb5f76c0 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.043+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb5f76c0 0x55fafb5f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.044+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb5f76c0 0x55fafb5f9af0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=715274d5f3a0c2c server_cookie=29e8394ff2752f69 in_seq=0 out_seq=0 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.044+0000 7f6d0cc00640 10 client.17622 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d0cc00640 1 -- 172.21.15.44:0/2028064357 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fafb550a70 con 0x55fafb5f76c0 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d0cc00640 10 client.17622 handle_command_reply: tid=0 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 client.17622 shutdown 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 msgr2=0x55fafb5e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e51d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d180ce4d0 tx=0x7f6d1809d4e0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 shutdown_connections 2024-04-20T11:31:13.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5e7670 0x55fafb5e51d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb5f76c0 0x55fafb5f9af0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 --2- 172.21.15.44:0/2028064357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5de170 0x55fafb5e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d0cc00640 0 client.17622 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:13.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 >> 172.21.15.44:0/2028064357 conn(0x55fafb5d88a0 msgr2=0x55fafb5da5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 shutdown_connections 2024-04-20T11:31:13.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 1 -- 172.21.15.44:0/2028064357 wait complete. 2024-04-20T11:31:13.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.045+0000 7f6d283fd740 20 client.17622 trim_cache size 0 max 16384 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.061+0000 7f6d283fd740 1 Processor -- start 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 -- start start 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5de170 0x55fafb5e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 0x55fafb5f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b4110 con 0x55fafb5dea90 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b5c50 con 0x55fafb5de170 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d283fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b47e0 con 0x55fafb5e7670 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 0x55fafb5f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d1ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5de170 0x55fafb5e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 0x55fafb5f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42960/0 (socket says 172.21.15.44:42960) 2024-04-20T11:31:13.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d1ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5de170 0x55fafb5e6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41902/0 (socket says 172.21.15.44:41902) 2024-04-20T11:31:13.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d17e00640 1 -- 172.21.15.44:0/1519221935 learned_addr learned my addr 172.21.15.44:0/1519221935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.062+0000 7f6d1f400640 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d17e00640 1 -- 172.21.15.44:0/1519221935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 msgr2=0x55fafb5d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d17e00640 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d17e00640 1 -- 172.21.15.44:0/1519221935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5de170 msgr2=0x55fafb5e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d17e00640 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5de170 0x55fafb5e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d17e00640 1 -- 172.21.15.44:0/1519221935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafb2553f0 con 0x55fafb5dea90 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d1f400640 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.063+0000 7f6d17e00640 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 0x55fafb5f3320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d0802c200 tx=0x7f6d08046070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be54b7045e26c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.064+0000 7f6d0f400640 1 -- 172.21.15.44:0/1519221935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08052070 con 0x55fafb5dea90 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.064+0000 7f6d0f400640 1 -- 172.21.15.44:0/1519221935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6d08048040 con 0x55fafb5dea90 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.064+0000 7f6d0f400640 1 -- 172.21.15.44:0/1519221935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08003b80 con 0x55fafb5dea90 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.064+0000 7f6d283fd740 1 -- 172.21.15.44:0/1519221935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 msgr2=0x55fafb5f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.064+0000 7f6d283fd740 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 0x55fafb5f3320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d0802c200 tx=0x7f6d08046070 comp rx=0 tx=0).stop 2024-04-20T11:31:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 -- 172.21.15.44:0/1519221935 shutdown_connections 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5dea90 0x55fafb5f3320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 --2- 172.21.15.44:0/1519221935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5de170 0x55fafb5e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 -- 172.21.15.44:0/1519221935 >> 172.21.15.44:0/1519221935 conn(0x55fafb5d88a0 msgr2=0x55fafb5d70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 -- 172.21.15.44:0/1519221935 shutdown_connections 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 -- 172.21.15.44:0/1519221935 wait complete. 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 -- start start 2024-04-20T11:31:13.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.065+0000 7f6d283fd740 1 Processor -- start 2024-04-20T11:31:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 0x55fafb5efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b5580 con 0x55fafb5de170 2024-04-20T11:31:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b4eb0 con 0x55fafb5dea90 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b2100 con 0x55fafb5e7670 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d283fd740 10 client.0 _ll_register_callbacks cb 0x7f6d1809c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 0x55fafb5efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 0x55fafb5efb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42966/0 (socket says 172.21.15.44:42966) 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 -- 172.21.15.44:0/277705482 learned_addr learned my addr 172.21.15.44:0/277705482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5eb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58586/0 (socket says 172.21.15.44:58586) 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d17e00640 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 -- 172.21.15.44:0/277705482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 msgr2=0x55fafb5eb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5eb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 -- 172.21.15.44:0/277705482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 msgr2=0x55fafb5eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.066+0000 7f6d1ea00640 1 -- 172.21.15.44:0/277705482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafb31f7d0 con 0x55fafb5de170 2024-04-20T11:31:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d1ea00640 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 0x55fafb5efb40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000b470 tx=0x7f6d1000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5b31c3bd13128 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d0cc00640 1 -- 172.21.15.44:0/277705482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000be60 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d08048040 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d0cc00640 1 -- 172.21.15.44:0/277705482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f6d10005250 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d0cc00640 1 -- 172.21.15.44:0/277705482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6d10039050 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d283fd740 10 client.17628 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.067+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55fafb65d990 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d0cc00640 1 -- 172.21.15.44:0/277705482 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6d08048040 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d0cc00640 1 -- 172.21.15.44:0/277705482 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f6d1002a120 con 0x55fafb5de170 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d283fd740 10 client.17628 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d283fd740 10 client.17628 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d283fd740 20 client.17628 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d283fd740 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb65de30 0x55fafb660250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d283fd740 4 client.17628 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fafb54a8e0 con 0x55fafb65de30 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.068+0000 7f6d17e00640 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb65de30 0x55fafb660250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d17e00640 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb65de30 0x55fafb660250 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bdda5f219fe84d7 server_cookie=44ec4f3cd8dc2a48 in_seq=0 out_seq=0 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d0cc00640 10 client.17628 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d0cc00640 1 -- 172.21.15.44:0/277705482 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fafb54a8e0 con 0x55fafb65de30 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d0cc00640 10 client.17628 handle_command_reply: tid=0 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 client.17628 shutdown 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 msgr2=0x55fafb5efb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 0x55fafb5efb40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000b470 tx=0x7f6d1000bcf0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 shutdown_connections 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fafb5e7670 0x55fafb5eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55fafb65de30 0x55fafb660250 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fafb5dea90 0x55fafb5eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 --2- 172.21.15.44:0/277705482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fafb5de170 0x55fafb5efb40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 >> 172.21.15.44:0/277705482 conn(0x55fafb5d88a0 msgr2=0x55fafb5e4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.069+0000 7f6d0cc00640 0 client.17628 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:13.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 shutdown_connections 2024-04-20T11:31:13.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d283fd740 1 -- 172.21.15.44:0/277705482 wait complete. 2024-04-20T11:31:13.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d283fd740 20 client.17628 trim_cache size 0 max 16384 2024-04-20T11:31:13.074 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6d040a3da0 msgr2=0x7f6d040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6d040a3da0 0x7f6d040a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d080096c0 tx=0x7f6d08005b70 comp rx=0 tx=0).stop 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 msgr2=0x7f6d201a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 0x7f6d201a3670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800ea60 tx=0x7f6d1800ef20 comp rx=0 tx=0).stop 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 shutdown_connections 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d2010d3d0 0x7f6d201a3670 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f6d040a3da0 0x7f6d040a6260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20108ea0 0x7f6d2019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 --2- 172.21.15.44:0/3587208868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d20108500 0x7f6d2019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.070+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 >> 172.21.15.44:0/3587208868 conn(0x7f6d20077d80 msgr2=0x7f6d200783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.071+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 shutdown_connections 2024-04-20T11:31:13.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.071+0000 7f6d1fe00640 1 -- 172.21.15.44:0/3587208868 wait complete. 2024-04-20T11:31:13.088 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:31:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 -- 172.21.15.44:0/2159453499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 msgr2=0x7f3b58104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 --2- 172.21.15.44:0/2159453499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 0x7f3b58104660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009900 tx=0x7f3b4802bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 -- 172.21.15.44:0/2159453499 shutdown_connections 2024-04-20T11:31:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 --2- 172.21.15.44:0/2159453499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b58109350 0x7f3b58110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 --2- 172.21.15.44:0/2159453499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58104c30 0x7f3b58108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 --2- 172.21.15.44:0/2159453499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 0x7f3b58104660 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 -- 172.21.15.44:0/2159453499 >> 172.21.15.44:0/2159453499 conn(0x7f3b580ffaa0 msgr2=0x7f3b58101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 -- 172.21.15.44:0/2159453499 shutdown_connections 2024-04-20T11:31:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.278+0000 7f3b57e00640 1 -- 172.21.15.44:0/2159453499 wait complete. 2024-04-20T11:31:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 Processor -- start 2024-04-20T11:31:13.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 -- start start 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 0x7f3b581a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 0x7f3b581a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b5819a8c0 0x7f3b5819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58114520 con 0x7f3b5819a8c0 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581143a0 con 0x7f3b58104280 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.280+0000 7f3b57e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581146a0 con 0x7f3b58109350 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 0x7f3b581a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b5819a8c0 0x7f3b5819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 0x7f3b581a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58592/0 (socket says 172.21.15.44:58592) 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 -- 172.21.15.44:0/1214248071 learned_addr learned my addr 172.21.15.44:0/1214248071 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56a00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 0x7f3b581a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 -- 172.21.15.44:0/1214248071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 msgr2=0x7f3b581a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 0x7f3b581a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 -- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b5819a8c0 msgr2=0x7f3b5819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b5819a8c0 0x7f3b5819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 -- 172.21.15.44:0/1214248071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5819b3e0 con 0x7f3b58109350 2024-04-20T11:31:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.281+0000 7f3b56000640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 0x7f3b581a0c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400e9c0 tx=0x7f3b4400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.282+0000 7f3b4f400640 1 -- 172.21.15.44:0/1214248071 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400cd80 con 0x7f3b58109350 2024-04-20T11:31:13.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.282+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5819b6d0 con 0x7f3b58109350 2024-04-20T11:31:13.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.282+0000 7f3b4f400640 1 -- 172.21.15.44:0/1214248071 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b44005520 con 0x7f3b58109350 2024-04-20T11:31:13.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.282+0000 7f3b4f400640 1 -- 172.21.15.44:0/1214248071 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44010690 con 0x7f3b58109350 2024-04-20T11:31:13.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.282+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b581a7970 con 0x7f3b58109350 2024-04-20T11:31:13.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.285+0000 7f3b605df740 1 Processor -- start 2024-04-20T11:31:13.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 -- start start 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d14550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ab57c79050 con 0x55ab57d14170 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab57c1f490 con 0x55ab57d1d670 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b605df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab57c75010 con 0x55ab57d14a90 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d14550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d14550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42990/0 (socket says 172.21.15.44:42990) 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b57400640 1 -- 172.21.15.44:0/4293093885 learned_addr learned my addr 172.21.15.44:0/4293093885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.286+0000 7f3b56a00640 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.287+0000 7f3b57400640 1 -- 172.21.15.44:0/4293093885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 msgr2=0x55ab57d1d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.287+0000 7f3b57400640 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.287+0000 7f3b57400640 1 -- 172.21.15.44:0/4293093885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 msgr2=0x55ab57d1fa60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:31:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.287+0000 7f3b57400640 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1fa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.287+0000 7f3b57400640 1 -- 172.21.15.44:0/4293093885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab57c27070 con 0x55ab57d14170 2024-04-20T11:31:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.287+0000 7f3b4f400640 1 -- 172.21.15.44:0/1214248071 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3b440046c0 con 0x7f3b58109350 2024-04-20T11:31:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.288+0000 7f3b57400640 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d14550 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009f90 tx=0x7f3b50010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c788f5e6c383244 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.288+0000 7f3b43e00640 1 -- 172.21.15.44:0/4293093885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50011700 con 0x55ab57d14170 2024-04-20T11:31:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.288+0000 7f3b43e00640 1 -- 172.21.15.44:0/4293093885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b50011da0 con 0x55ab57d14170 2024-04-20T11:31:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b43e00640 1 -- 172.21.15.44:0/4293093885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50018380 con 0x55ab57d14170 2024-04-20T11:31:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 -- 172.21.15.44:0/4293093885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 msgr2=0x55ab57d14550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d14550 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009f90 tx=0x7f3b50010860 comp rx=0 tx=0).stop 2024-04-20T11:31:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 -- 172.21.15.44:0/4293093885 shutdown_connections 2024-04-20T11:31:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 --2- 172.21.15.44:0/4293093885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d14550 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 -- 172.21.15.44:0/4293093885 >> 172.21.15.44:0/4293093885 conn(0x55ab57d0e8a0 msgr2=0x55ab57d0ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 -- 172.21.15.44:0/4293093885 shutdown_connections 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.289+0000 7f3b605df740 1 -- 172.21.15.44:0/4293093885 wait complete. 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.290+0000 7f3b4f400640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3b3c0a3d30 0x7f3b3c0a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.290+0000 7f3b4f400640 1 -- 172.21.15.44:0/1214248071 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f3b440d0b30 con 0x7f3b58109350 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.290+0000 7f3b605df740 1 -- start start 2024-04-20T11:31:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.290+0000 7f3b56a00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3b3c0a3d30 0x7f3b3c0a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.290+0000 7f3b605df740 1 Processor -- start 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b56a00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3b3c0a3d30 0x7f3b3c0a61f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005ab0 tx=0x7f3b48005a20 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d1a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ab57d20500 con 0x55ab57d14170 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ab57c32860 con 0x55ab57d1d670 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab57d20680 con 0x55ab57d14a90 2024-04-20T11:31:13.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b605df740 10 client.0 _ll_register_callbacks cb 0x55ab57c00fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d1a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b57400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d1a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43000/0 (socket says 172.21.15.44:43000) 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b56a00640 1 -- 172.21.15.44:0/4127755550 learned_addr learned my addr 172.21.15.44:0/4127755550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.291+0000 7f3b56000640 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b57400640 1 -- 172.21.15.44:0/4127755550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 msgr2=0x55ab57d1ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b57400640 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b57400640 1 -- 172.21.15.44:0/4127755550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 msgr2=0x55ab57d1a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b57400640 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d1a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b57400640 1 -- 172.21.15.44:0/4127755550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab57c36950 con 0x55ab57d1d670 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b57400640 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1b0c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3b50000f80 tx=0x7f3b50009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ca836ed7cf484f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b40c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b40c00640 1 -- 172.21.15.44:0/4127755550 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50027030 con 0x55ab57d1d670 2024-04-20T11:31:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b40c00640 1 -- 172.21.15.44:0/4127755550 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f3b5001f070 con 0x55ab57d1d670 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b50011da0 con 0x55ab57d1d670 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b40c00640 1 -- 172.21.15.44:0/4127755550 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3b50031050 con 0x55ab57d1d670 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b605df740 10 client.27758 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.292+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ab57c04820 con 0x55ab57d1d670 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b40c00640 1 -- 172.21.15.44:0/4127755550 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b50011da0 con 0x55ab57d1d670 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b40c00640 1 -- 172.21.15.44:0/4127755550 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3b50003b00 con 0x55ab57d1d670 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b605df740 10 client.27758 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b605df740 10 client.27758 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b605df740 20 client.27758 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b605df740 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d2d640 0x55ab57d2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b605df740 4 client.27758 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.293+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab57c86a70 con 0x55ab57d2d640 2024-04-20T11:31:13.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.294+0000 7f3b56a00640 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d2d640 0x55ab57d2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.294+0000 7f3b56a00640 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d2d640 0x55ab57d2fa30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51d3c2cd83e6d26f server_cookie=7be96b3c435abc56 in_seq=0 out_seq=0 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.294+0000 7f3b40c00640 10 client.27758 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b40c00640 1 -- 172.21.15.44:0/4127755550 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab57c86a70 con 0x55ab57d2d640 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b40c00640 10 client.27758 handle_command_reply: tid=0 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 client.27758 shutdown 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 msgr2=0x55ab57d1b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1b0c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3b50000f80 tx=0x7f3b50009bf0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 shutdown_connections 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d1d670 0x55ab57d1b0c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d1ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d2d640 0x55ab57d2fa30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 --2- 172.21.15.44:0/4127755550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14170 0x55ab57d1a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b40c00640 0 client.27758 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.295+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 >> 172.21.15.44:0/4127755550 conn(0x55ab57d0e8a0 msgr2=0x55ab57d105f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.296+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 shutdown_connections 2024-04-20T11:31:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.296+0000 7f3b605df740 1 -- 172.21.15.44:0/4127755550 wait complete. 2024-04-20T11:31:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.296+0000 7f3b605df740 20 client.27758 trim_cache size 0 max 16384 2024-04-20T11:31:13.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.310+0000 7f3b605df740 1 Processor -- start 2024-04-20T11:31:13.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.310+0000 7f3b605df740 1 -- start start 2024-04-20T11:31:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b605df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b605df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 0x55ab57d293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b605df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d1d670 0x55ab57d15db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b605df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581afdf0 con 0x55ab57d14a90 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b605df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581b1930 con 0x55ab57d14170 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b605df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581b04c0 con 0x55ab57d1d670 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b56000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 0x55ab57d293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b56000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 0x55ab57d293c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43014/0 (socket says 172.21.15.44:43014) 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b56000640 1 -- 172.21.15.44:0/2647466633 learned_addr learned my addr 172.21.15.44:0/2647466633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b56a00640 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.311+0000 7f3b57400640 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d1d670 0x55ab57d15db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56000640 1 -- 172.21.15.44:0/2647466633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d1d670 msgr2=0x55ab57d15db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56000640 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d1d670 0x55ab57d15db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56000640 1 -- 172.21.15.44:0/2647466633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 msgr2=0x55ab57d1cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56000640 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d1cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56000640 1 -- 172.21.15.44:0/2647466633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5798b3f0 con 0x55ab57d14a90 2024-04-20T11:31:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56a00640 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d1cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b56000640 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 0x55ab57d293c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b440ce2c0 tx=0x7f3b4400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ac0ecbcf500972 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b43400640 1 -- 172.21.15.44:0/2647466633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4409d1b0 con 0x55ab57d14a90 2024-04-20T11:31:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b43400640 1 -- 172.21.15.44:0/2647466633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b440b6020 con 0x55ab57d14a90 2024-04-20T11:31:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.312+0000 7f3b43400640 1 -- 172.21.15.44:0/2647466633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440992b0 con 0x55ab57d14a90 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 -- 172.21.15.44:0/2647466633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 msgr2=0x55ab57d293c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 0x55ab57d293c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b440ce2c0 tx=0x7f3b4400efd0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 -- 172.21.15.44:0/2647466633 shutdown_connections 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d1d670 0x55ab57d15db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d14a90 0x55ab57d293c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 --2- 172.21.15.44:0/2647466633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d1cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 -- 172.21.15.44:0/2647466633 >> 172.21.15.44:0/2647466633 conn(0x55ab57d0e8a0 msgr2=0x55ab57d18de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 -- 172.21.15.44:0/2647466633 shutdown_connections 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 -- 172.21.15.44:0/2647466633 wait complete. 2024-04-20T11:31:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 -- start start 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.313+0000 7f3b605df740 1 Processor -- start 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d17130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d1d670 0x55ab57d17850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581b1260 con 0x55ab57d1d670 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581b0b90 con 0x55ab57d14170 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581adde0 con 0x55ab57d14a90 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b605df740 10 client.0 _ll_register_callbacks cb 0x7f3b500098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d25e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:41974/0 (socket says 172.21.15.44:41974) 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b56a00640 1 -- 172.21.15.44:0/2065339876 learned_addr learned my addr 172.21.15.44:0/2065339876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b57400640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d1d670 0x55ab57d17850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.314+0000 7f3b56000640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d17130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b56a00640 1 -- 172.21.15.44:0/2065339876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 msgr2=0x55ab57d17130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b56a00640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d17130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b56a00640 1 -- 172.21.15.44:0/2065339876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d1d670 msgr2=0x55ab57d17850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b56a00640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d1d670 0x55ab57d17850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b56a00640 1 -- 172.21.15.44:0/2065339876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab57a557d0 con 0x55ab57d14170 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b57400640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d1d670 0x55ab57d17850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b56a00640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d25e20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3b4802c180 tx=0x7f3b4802bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62bc51e9e85bd784 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b40c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b40c00640 1 -- 172.21.15.44:0/2065339876 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48058060 con 0x55ab57d14170 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b440b6020 con 0x55ab57d14170 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b40c00640 1 -- 172.21.15.44:0/2065339876 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f3b48052250 con 0x55ab57d14170 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b40c00640 1 -- 172.21.15.44:0/2065339876 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3b48052450 con 0x55ab57d14170 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b605df740 10 client.27764 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.315+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ab57d93850 con 0x55ab57d14170 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b40c00640 1 -- 172.21.15.44:0/2065339876 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b440b6020 con 0x55ab57d14170 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b40c00640 1 -- 172.21.15.44:0/2065339876 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3b480035f0 con 0x55ab57d14170 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b605df740 10 client.27764 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b605df740 10 client.27764 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b605df740 20 client.27764 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b605df740 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d93f50 0x55ab57d96370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b605df740 4 client.27764 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab57c808e0 con 0x55ab57d93f50 2024-04-20T11:31:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.316+0000 7f3b56000640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d93f50 0x55ab57d96370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b56000640 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d93f50 0x55ab57d96370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9aad4d4164ccceb server_cookie=c4941bdb9d14ba14 in_seq=0 out_seq=0 2024-04-20T11:31:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b40c00640 10 client.27764 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b40c00640 1 -- 172.21.15.44:0/2065339876 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab57c808e0 con 0x55ab57d93f50 2024-04-20T11:31:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b40c00640 10 client.27764 handle_command_reply: tid=0 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 client.27764 shutdown 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 msgr2=0x55ab57d25e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d25e20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3b4802c180 tx=0x7f3b4802bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 shutdown_connections 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55ab57d93f50 0x55ab57d96370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ab57d1d670 0x55ab57d17850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab57d14a90 0x55ab57d17130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 --2- 172.21.15.44:0/2065339876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab57d14170 0x55ab57d25e20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b40c00640 0 client.27764 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 >> 172.21.15.44:0/2065339876 conn(0x55ab57d0e8a0 msgr2=0x55ab57d1ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 shutdown_connections 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.317+0000 7f3b605df740 1 -- 172.21.15.44:0/2065339876 wait complete. 2024-04-20T11:31:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b605df740 20 client.27764 trim_cache size 0 max 16384 2024-04-20T11:31:13.323 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:13.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3b3c0a3d30 msgr2=0x7f3b3c0a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3b3c0a3d30 0x7f3b3c0a61f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005ab0 tx=0x7f3b48005a20 comp rx=0 tx=0).stop 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 msgr2=0x7f3b581a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 0x7f3b581a0c50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400e9c0 tx=0x7f3b4400ee80 comp rx=0 tx=0).stop 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 shutdown_connections 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3b3c0a3d30 0x7f3b3c0a61f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b5819a8c0 0x7f3b5819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b58109350 0x7f3b581a0c50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 --2- 172.21.15.44:0/1214248071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b58104280 0x7f3b581a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 >> 172.21.15.44:0/1214248071 conn(0x7f3b580ffaa0 msgr2=0x7f3b58101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.318+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 shutdown_connections 2024-04-20T11:31:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.319+0000 7f3b57e00640 1 -- 172.21.15.44:0/1214248071 wait complete. 2024-04-20T11:31:13.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 -- 172.21.15.44:0/3653789687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104c20 msgr2=0x7fbfe4108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 --2- 172.21.15.44:0/3653789687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104c20 0x7fbfe4108c60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009860 tx=0x7fbfd002bb70 comp rx=0 tx=0).stop 2024-04-20T11:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 -- 172.21.15.44:0/3653789687 shutdown_connections 2024-04-20T11:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 --2- 172.21.15.44:0/3653789687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfe4109340 0x7fbfe4110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 --2- 172.21.15.44:0/3653789687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104c20 0x7fbfe4108c60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 --2- 172.21.15.44:0/3653789687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfe4104270 0x7fbfe4104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 -- 172.21.15.44:0/3653789687 >> 172.21.15.44:0/3653789687 conn(0x7fbfe40ffa70 msgr2=0x7fbfe4101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 -- 172.21.15.44:0/3653789687 shutdown_connections 2024-04-20T11:31:13.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.544+0000 7fbfeb000640 1 -- 172.21.15.44:0/3653789687 wait complete. 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.545+0000 7fbfeb000640 1 Processor -- start 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.545+0000 7fbfeb000640 1 -- start start 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfeb000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 0x7fbfe41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfeb000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfe4109340 0x7fbfe41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfeb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfe419a7f0 0x7fbfe419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfeb000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4114510 con 0x7fbfe4104270 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfeb000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4114390 con 0x7fbfe4109340 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfeb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4114690 con 0x7fbfe419a7f0 2024-04-20T11:31:13.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 0x7fbfe41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 0x7fbfe41a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43036/0 (socket says 172.21.15.44:43036) 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 -- 172.21.15.44:0/2651836555 learned_addr learned my addr 172.21.15.44:0/2651836555 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9200640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfe4109340 0x7fbfe41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfea600640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfe419a7f0 0x7fbfe419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 -- 172.21.15.44:0/2651836555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfe419a7f0 msgr2=0x7fbfe419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfe419a7f0 0x7fbfe419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 -- 172.21.15.44:0/2651836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfe4109340 msgr2=0x7fbfe41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfe4109340 0x7fbfe41a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.546+0000 7fbfe9c00640 1 -- 172.21.15.44:0/2651836555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe419b380 con 0x7fbfe4104270 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.547+0000 7fbfe9c00640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 0x7fbfe41a0680 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800e9c0 tx=0x7fbfd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.547+0000 7fbfe2000640 1 -- 172.21.15.44:0/2651836555 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800cde0 con 0x7fbfe4104270 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.547+0000 7fbfe2000640 1 -- 172.21.15.44:0/2651836555 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbfd80035f0 con 0x7fbfe4104270 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.547+0000 7fbfe2000640 1 -- 172.21.15.44:0/2651836555 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8010670 con 0x7fbfe4104270 2024-04-20T11:31:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.547+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe419b670 con 0x7fbfe4104270 2024-04-20T11:31:13.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.547+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe41a7870 con 0x7fbfe4104270 2024-04-20T11:31:13.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 Processor -- start 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 -- start start 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfe2000640 1 -- 172.21.15.44:0/2651836555 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbfd80037a0 con 0x7fbfe4104270 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866170 0x55becc866550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866a90 0x55becc86f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc871a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55becc7cb050 con 0x55becc86f670 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55becc771490 con 0x55becc866170 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55becc7c7010 con 0x55becc866a90 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc871a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfea600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866170 0x55becc866550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfe9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866a90 0x55becc86f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc871a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43042/0 (socket says 172.21.15.44:43042) 2024-04-20T11:31:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.551+0000 7fbfe9200640 1 -- 172.21.15.44:0/3756023310 learned_addr learned my addr 172.21.15.44:0/3756023310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfea600640 1 -- 172.21.15.44:0/3756023310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866a90 msgr2=0x55becc86f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfea600640 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866a90 0x55becc86f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfea600640 1 -- 172.21.15.44:0/3756023310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 msgr2=0x55becc871a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfea600640 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc871a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfea600640 1 -- 172.21.15.44:0/3756023310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becc779070 con 0x55becc866170 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfea600640 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866170 0x55becc866550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009e50 tx=0x7fbfdc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0fc77e0da5c795 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfd6a00640 1 -- 172.21.15.44:0/3756023310 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc003b70 con 0x55becc866170 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfd6a00640 1 -- 172.21.15.44:0/3756023310 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbfdc015070 con 0x55becc866170 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.552+0000 7fbfd6a00640 1 -- 172.21.15.44:0/3756023310 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc011860 con 0x55becc866170 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfe2000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbfcc0a3da0 0x7fbfcc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfe2000640 1 -- 172.21.15.44:0/2651836555 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fbfd80d0b60 con 0x7fbfe4104270 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfe9c00640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbfcc0a3da0 0x7fbfcc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3756023310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866170 msgr2=0x55becc866550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866170 0x55becc866550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009e50 tx=0x7fbfdc0107a0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3756023310 shutdown_connections 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc871a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866a90 0x55becc86f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3756023310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866170 0x55becc866550 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3756023310 >> 172.21.15.44:0/3756023310 conn(0x55becc8608a0 msgr2=0x55becc860c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3756023310 shutdown_connections 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.553+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3756023310 wait complete. 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.554+0000 7fbfe9c00640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbfcc0a3da0 0x7fbfcc0a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55becc865c40 tx=0x7fbfd80d4610 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.554+0000 7fbfef3bc740 1 -- start start 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.554+0000 7fbfef3bc740 1 Processor -- start 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.554+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 0x55becc86c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866a90 0x55becc86cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc86f670 0x55becc86d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55becc872460 con 0x55becc866170 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55becc784860 con 0x55becc866a90 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55becc8725e0 con 0x55becc86f670 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfef3bc740 10 client.0 _ll_register_callbacks cb 0x55becc752fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfea600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 0x55becc86c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfea600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 0x55becc86c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43050/0 (socket says 172.21.15.44:43050) 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfea600640 1 -- 172.21.15.44:0/552636238 learned_addr learned my addr 172.21.15.44:0/552636238 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfe9200640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866a90 0x55becc86cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfea600640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc86f670 0x55becc86d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfe9200640 1 -- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc86f670 msgr2=0x55becc86d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfe9200640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc86f670 0x55becc86d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfe9200640 1 -- 172.21.15.44:0/552636238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 msgr2=0x55becc86c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfe9200640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 0x55becc86c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfe9200640 1 -- 172.21.15.44:0/552636238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becc788950 con 0x55becc866a90 2024-04-20T11:31:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.555+0000 7fbfea600640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 0x55becc86c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfe9200640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866a90 0x55becc86cbf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbfd002c180 tx=0x7fbfd0005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2dc6b8b3ec2e19b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfcb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfcb400640 1 -- 172.21.15.44:0/552636238 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0043070 con 0x55becc866a90 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfcb400640 1 -- 172.21.15.44:0/552636238 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fbfd0004450 con 0x55becc866a90 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfdc015070 con 0x55becc866a90 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfcb400640 1 -- 172.21.15.44:0/552636238 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbfd004f050 con 0x55becc866a90 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfef3bc740 10 client.27776 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.556+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55becc756820 con 0x55becc866a90 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfcb400640 1 -- 172.21.15.44:0/552636238 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbfdc015070 con 0x55becc866a90 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfcb400640 1 -- 172.21.15.44:0/552636238 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fbfd0032640 con 0x55becc866a90 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfef3bc740 10 client.27776 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfef3bc740 10 client.27776 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfef3bc740 20 client.27776 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc87f5d0 0x55becc881a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfef3bc740 4 client.27776 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55becc7d8a70 con 0x55becc87f5d0 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.557+0000 7fbfea600640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc87f5d0 0x55becc881a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.558+0000 7fbfea600640 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc87f5d0 0x55becc881a00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfd1eaffddf3cd3 server_cookie=cfb26dadd7b723d5 in_seq=0 out_seq=0 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.558+0000 7fbfcb400640 10 client.27776 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfcb400640 1 -- 172.21.15.44:0/552636238 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55becc7d8a70 con 0x55becc87f5d0 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfcb400640 10 client.27776 handle_command_reply: tid=0 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 client.27776 shutdown 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866a90 msgr2=0x55becc86cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866a90 0x55becc86cbf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbfd002c180 tx=0x7fbfd0005af0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 shutdown_connections 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc87f5d0 0x55becc881a00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc86f670 0x55becc86d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc866a90 0x55becc86cbf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/552636238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc866170 0x55becc86c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfcb400640 0 client.27776 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 >> 172.21.15.44:0/552636238 conn(0x55becc8608a0 msgr2=0x55becc8625f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 shutdown_connections 2024-04-20T11:31:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.559+0000 7fbfef3bc740 1 -- 172.21.15.44:0/552636238 wait complete. 2024-04-20T11:31:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.560+0000 7fbfef3bc740 20 client.27776 trim_cache size 0 max 16384 2024-04-20T11:31:13.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.574+0000 7fbfef3bc740 1 Processor -- start 2024-04-20T11:31:13.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.574+0000 7fbfef3bc740 1 -- start start 2024-04-20T11:31:13.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc86ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc87b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc8872d0 0x55becc887730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41afd20 con 0x55becc86f670 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1860 con 0x55becc8872d0 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b03f0 con 0x55becc866170 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc87b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc87b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43052/0 (socket says 172.21.15.44:43052) 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 -- 172.21.15.44:0/3752898574 learned_addr learned my addr 172.21.15.44:0/3752898574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfea600640 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc8872d0 0x55becc887730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfea600640 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc86ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 -- 172.21.15.44:0/3752898574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 msgr2=0x55becc86ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc86ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 -- 172.21.15.44:0/3752898574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc8872d0 msgr2=0x55becc887730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc8872d0 0x55becc887730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfe9200640 1 -- 172.21.15.44:0/3752898574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becc4dd3f0 con 0x55becc86f670 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.575+0000 7fbfea600640 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc8872d0 0x55becc887730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.576+0000 7fbfe9200640 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc87b320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0005e00 tx=0x7fbfd002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715b9826a3576b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.576+0000 7fbfd6000640 1 -- 172.21.15.44:0/3752898574 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0043050 con 0x55becc86f670 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.576+0000 7fbfd6000640 1 -- 172.21.15.44:0/3752898574 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbfd0032840 con 0x55becc86f670 2024-04-20T11:31:13.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.576+0000 7fbfd6000640 1 -- 172.21.15.44:0/3752898574 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0039040 con 0x55becc86f670 2024-04-20T11:31:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3752898574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 msgr2=0x55becc87b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc87b320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0005e00 tx=0x7fbfd002bb70 comp rx=0 tx=0).stop 2024-04-20T11:31:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3752898574 shutdown_connections 2024-04-20T11:31:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc8872d0 0x55becc887730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc86f670 0x55becc87b320 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/3752898574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc86ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3752898574 >> 172.21.15.44:0/3752898574 conn(0x55becc8608a0 msgr2=0x55becc866e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3752898574 shutdown_connections 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 -- 172.21.15.44:0/3752898574 wait complete. 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 -- start start 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.577+0000 7fbfef3bc740 1 Processor -- start 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc869780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 0x55becc869cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc8872d0 0x55becc8e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b1190 con 0x55becc8872d0 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41b0ac0 con 0x55becc86f670 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41add10 con 0x55becc866170 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfef3bc740 10 client.0 _ll_register_callbacks cb 0x7fbfd0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 0x55becc869cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfe9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 0x55becc869cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42026/0 (socket says 172.21.15.44:42026) 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfe9200640 1 -- 172.21.15.44:0/1896901284 learned_addr learned my addr 172.21.15.44:0/1896901284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfea600640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc869780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.578+0000 7fbfea600640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc8872d0 0x55becc8e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfe9200640 1 -- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 msgr2=0x55becc869780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfe9200640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc869780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfe9200640 1 -- 172.21.15.44:0/1896901284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc8872d0 msgr2=0x55becc8e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfe9200640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc8872d0 0x55becc8e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfe9200640 1 -- 172.21.15.44:0/1896901284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becc5a77d0 con 0x55becc86f670 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfea600640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc869780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfe9200640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 0x55becc869cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00057c0 tx=0x7fbfd0004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e9e55bfbfb1a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfcb400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfcb400640 1 -- 172.21.15.44:0/1896901284 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0061060 con 0x55becc86f670 2024-04-20T11:31:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd0032840 con 0x55becc86f670 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfcb400640 1 -- 172.21.15.44:0/1896901284 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fbfd0034650 con 0x55becc86f670 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfcb400640 1 -- 172.21.15.44:0/1896901284 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbfd0039070 con 0x55becc86f670 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfef3bc740 10 client.27782 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.579+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55becc8e5ff0 con 0x55becc86f670 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfcb400640 1 -- 172.21.15.44:0/1896901284 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbfd0032840 con 0x55becc86f670 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfcb400640 1 -- 172.21.15.44:0/1896901284 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fbfd002c510 con 0x55becc86f670 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfef3bc740 10 client.27782 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfef3bc740 10 client.27782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfef3bc740 20 client.27782 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc8e6700 0x55becc8e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfef3bc740 4 client.27782 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55becc7d28e0 con 0x55becc8e6700 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.580+0000 7fbfea600640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc8e6700 0x55becc8e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfea600640 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc8e6700 0x55becc8e8b20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=636824b006e25916 server_cookie=d7d81c8c7b179401 in_seq=0 out_seq=0 2024-04-20T11:31:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfcb400640 10 client.27782 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfcb400640 1 -- 172.21.15.44:0/1896901284 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55becc7d28e0 con 0x55becc8e6700 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfcb400640 10 client.27782 handle_command_reply: tid=0 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 client.27782 shutdown 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 msgr2=0x55becc869cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 0x55becc869cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00057c0 tx=0x7fbfd0004500 comp rx=0 tx=0).stop 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 shutdown_connections 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55becc8e6700 0x55becc8e8b20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55becc8872d0 0x55becc8e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfcb400640 0 client.27782 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55becc86f670 0x55becc869cc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 --2- 172.21.15.44:0/1896901284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55becc866170 0x55becc869780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.581+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 >> 172.21.15.44:0/1896901284 conn(0x55becc8608a0 msgr2=0x55becc86c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 shutdown_connections 2024-04-20T11:31:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfef3bc740 1 -- 172.21.15.44:0/1896901284 wait complete. 2024-04-20T11:31:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfef3bc740 20 client.27782 trim_cache size 0 max 16384 2024-04-20T11:31:13.587 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbfcc0a3da0 msgr2=0x7fbfcc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfeb000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbfcc0a3da0 0x7fbfcc0a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55becc865c40 tx=0x7fbfd80d4610 comp rx=0 tx=0).stop 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 msgr2=0x7fbfe41a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.582+0000 7fbfeb000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 0x7fbfe41a0680 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800e9c0 tx=0x7fbfd800ee80 comp rx=0 tx=0).stop 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 shutdown_connections 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfe419a7f0 0x7fbfe419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfe4109340 0x7fbfe41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fbfcc0a3da0 0x7fbfcc0a6260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 --2- 172.21.15.44:0/2651836555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfe4104270 0x7fbfe41a0680 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 >> 172.21.15.44:0/2651836555 conn(0x7fbfe40ffa70 msgr2=0x7fbfe4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 shutdown_connections 2024-04-20T11:31:13.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.583+0000 7fbfeb000640 1 -- 172.21.15.44:0/2651836555 wait complete. 2024-04-20T11:31:13.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:31:13.660 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:13 smithi044 ceph-mon[16855]: pgmap v266: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:13.660 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:13 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4219945077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:31:13.660 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:13 smithi044 ceph-mon[16855]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T11:31:13.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.799+0000 7fa071a00640 1 -- 172.21.15.44:0/886810357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c104270 msgr2=0x7fa06c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.799+0000 7fa071a00640 1 --2- 172.21.15.44:0/886810357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c104270 0x7fa06c104650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009900 tx=0x7fa05c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:31:13.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.799+0000 7fa071a00640 1 -- 172.21.15.44:0/886810357 shutdown_connections 2024-04-20T11:31:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.799+0000 7fa071a00640 1 --2- 172.21.15.44:0/886810357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109340 0x7fa06c110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.799+0000 7fa071a00640 1 --2- 172.21.15.44:0/886810357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa06c104c20 0x7fa06c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.799+0000 7fa071a00640 1 --2- 172.21.15.44:0/886810357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c104270 0x7fa06c104650 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.800+0000 7fa071a00640 1 -- 172.21.15.44:0/886810357 >> 172.21.15.44:0/886810357 conn(0x7fa06c0ffad0 msgr2=0x7fa06c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.800+0000 7fa071a00640 1 -- 172.21.15.44:0/886810357 shutdown_connections 2024-04-20T11:31:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.800+0000 7fa071a00640 1 -- 172.21.15.44:0/886810357 wait complete. 2024-04-20T11:31:13.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 Processor -- start 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 -- start start 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c104270 0x7fa06c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa06c109340 0x7fa06c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c19a840 0x7fa06c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1144c0 con 0x7fa06c104270 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c114340 con 0x7fa06c109340 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa071a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c114640 con 0x7fa06c19a840 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa06be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c104270 0x7fa06c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.801+0000 7fa06be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c104270 0x7fa06c1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43062/0 (socket says 172.21.15.44:43062) 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa06be00640 1 -- 172.21.15.44:0/1559995564 learned_addr learned my addr 172.21.15.44:0/1559995564 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c19a840 0x7fa06c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa063400640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa06c109340 0x7fa06c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 -- 172.21.15.44:0/1559995564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa06c109340 msgr2=0x7fa06c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa06c109340 0x7fa06c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 -- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c104270 msgr2=0x7fa06c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c104270 0x7fa06c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 -- 172.21.15.44:0/1559995564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c19b3d0 con 0x7fa06c19a840 2024-04-20T11:31:13.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.802+0000 7fa071000640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c19a840 0x7fa06c19acf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa064009fd0 tx=0x7fa06400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.803+0000 7fa069600640 1 -- 172.21.15.44:0/1559995564 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400ce20 con 0x7fa06c19a840 2024-04-20T11:31:13.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.803+0000 7fa069600640 1 -- 172.21.15.44:0/1559995564 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa064004b20 con 0x7fa06c19a840 2024-04-20T11:31:13.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.803+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c19b6c0 con 0x7fa06c19a840 2024-04-20T11:31:13.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.803+0000 7fa069600640 1 -- 172.21.15.44:0/1559995564 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064010790 con 0x7fa06c19a840 2024-04-20T11:31:13.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.803+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c1a7850 con 0x7fa06c19a840 2024-04-20T11:31:13.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.806+0000 7fa075d66740 1 Processor -- start 2024-04-20T11:31:13.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.806+0000 7fa069600640 1 -- 172.21.15.44:0/1559995564 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa064005020 con 0x7fa06c19a840 2024-04-20T11:31:13.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.806+0000 7fa075d66740 1 -- start start 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa075d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d4d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa075d66740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa075d66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d58a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa075d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5557a8cb2050 con 0x5557a8d4d170 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa075d66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5557a8c58490 con 0x5557a8d56670 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa075d66740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5557a8cae010 con 0x5557a8d4da90 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa063400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d4d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa06be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa063400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d4d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43076/0 (socket says 172.21.15.44:43076) 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa063400640 1 -- 172.21.15.44:0/2422936564 learned_addr learned my addr 172.21.15.44:0/2422936564 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.807+0000 7fa063400640 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d58a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa06be00640 1 -- 172.21.15.44:0/2422936564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 msgr2=0x5557a8d58a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa06be00640 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d58a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa06be00640 1 -- 172.21.15.44:0/2422936564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 msgr2=0x5557a8d4d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa06be00640 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d4d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa06be00640 1 -- 172.21.15.44:0/2422936564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a8c60070 con 0x5557a8d4da90 2024-04-20T11:31:13.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa06be00640 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d56130 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005ec0 tx=0x7fa05c0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88657160d6597ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa061600640 1 -- 172.21.15.44:0/2422936564 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c02c790 con 0x5557a8d4da90 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa069600640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0500a3da0 0x7fa0500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa061600640 1 -- 172.21.15.44:0/2422936564 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c004040 con 0x5557a8d4da90 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.808+0000 7fa061600640 1 -- 172.21.15.44:0/2422936564 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0046c0 con 0x5557a8d4da90 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa069600640 1 -- 172.21.15.44:0/1559995564 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fa0640d0e90 con 0x7fa06c19a840 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa063400640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0500a3da0 0x7fa0500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 -- 172.21.15.44:0/2422936564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 msgr2=0x5557a8d56130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d56130 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005ec0 tx=0x7fa05c0095f0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 -- 172.21.15.44:0/2422936564 shutdown_connections 2024-04-20T11:31:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d58a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d56130 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 --2- 172.21.15.44:0/2422936564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d4d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 -- 172.21.15.44:0/2422936564 >> 172.21.15.44:0/2422936564 conn(0x5557a8d478a0 msgr2=0x5557a8d47c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 -- 172.21.15.44:0/2422936564 shutdown_connections 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa075d66740 1 -- 172.21.15.44:0/2422936564 wait complete. 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.809+0000 7fa063400640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0500a3da0 0x7fa0500a6260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5557a8d48a30 tx=0x7fa054015040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 -- start start 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 Processor -- start 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 0x5557a8d536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4da90 0x5557a8d53be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d54120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5557a8d59400 con 0x5557a8d56670 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5557a8c6b860 con 0x5557a8d4da90 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.810+0000 7fa075d66740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5557a8d59580 con 0x5557a8d4d170 2024-04-20T11:31:13.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa075d66740 10 client.0 _ll_register_callbacks cb 0x5557a8c39fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa06be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 0x5557a8d536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa063400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4da90 0x5557a8d53be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa06be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 0x5557a8d536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58728/0 (socket says 172.21.15.44:58728) 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa06be00640 1 -- 172.21.15.44:0/643259764 learned_addr learned my addr 172.21.15.44:0/643259764 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa071000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d54120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa063400640 1 -- 172.21.15.44:0/643259764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 msgr2=0x5557a8d536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa063400640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 0x5557a8d536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa063400640 1 -- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 msgr2=0x5557a8d54120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa063400640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d54120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa063400640 1 -- 172.21.15.44:0/643259764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a8c6f950 con 0x5557a8d4da90 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa06be00640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 0x5557a8d536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.811+0000 7fa071000640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d54120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa063400640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4da90 0x5557a8d53be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa054009fd0 tx=0x7fa054005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f2d78e54802c479 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa05a000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa05a000640 1 -- 172.21.15.44:0/643259764 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054021ae0 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c004040 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa05a000640 1 -- 172.21.15.44:0/643259764 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fa054021c80 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa05a000640 1 -- 172.21.15.44:0/643259764 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa054034420 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa075d66740 10 client.27794 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.812+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5557a8c3d820 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa05a000640 1 -- 172.21.15.44:0/643259764 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c004040 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa05a000640 1 -- 172.21.15.44:0/643259764 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fa05402a4e0 con 0x5557a8d4da90 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa075d66740 10 client.27794 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa075d66740 10 client.27794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa075d66740 20 client.27794 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa075d66740 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8d66640 0x5557a8d68aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa075d66740 4 client.27794 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a8cbfa70 con 0x5557a8d66640 2024-04-20T11:31:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.813+0000 7fa06be00640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8d66640 0x5557a8d68aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.814+0000 7fa06be00640 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8d66640 0x5557a8d68aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbd5c106ad94357b server_cookie=6c7fc74bbfecf5dc in_seq=0 out_seq=0 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.814+0000 7fa05a000640 10 client.27794 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa05a000640 1 -- 172.21.15.44:0/643259764 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557a8cbfa70 con 0x5557a8d66640 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa05a000640 10 client.27794 handle_command_reply: tid=0 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 client.27794 shutdown 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4da90 msgr2=0x5557a8d53be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4da90 0x5557a8d53be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa054009fd0 tx=0x7fa054005f90 comp rx=0 tx=0).stop 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 shutdown_connections 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8d66640 0x5557a8d68aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d54120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4da90 0x5557a8d53be0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 --2- 172.21.15.44:0/643259764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4d170 0x5557a8d536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa05a000640 0 client.27794 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 >> 172.21.15.44:0/643259764 conn(0x5557a8d478a0 msgr2=0x5557a8d495f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 shutdown_connections 2024-04-20T11:31:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 1 -- 172.21.15.44:0/643259764 wait complete. 2024-04-20T11:31:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.815+0000 7fa075d66740 20 client.27794 trim_cache size 0 max 16384 2024-04-20T11:31:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 Processor -- start 2024-04-20T11:31:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 -- start start 2024-04-20T11:31:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d55dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1afc20 con 0x5557a8d4d170 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b1760 con 0x5557a8d56670 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.830+0000 7fa075d66740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b02f0 con 0x5557a8d4da90 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d55dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d55dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43096/0 (socket says 172.21.15.44:43096) 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa071000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 -- 172.21.15.44:0/3696056431 learned_addr learned my addr 172.21.15.44:0/3696056431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa063400640 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 -- 172.21.15.44:0/3696056431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 msgr2=0x5557a8d622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 -- 172.21.15.44:0/3696056431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 msgr2=0x5557a8d53d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d53d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.831+0000 7fa06be00640 1 -- 172.21.15.44:0/3696056431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a89c43f0 con 0x5557a8d4d170 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.832+0000 7fa06be00640 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d55dd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005bc0 tx=0x7fa05c04a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f003b8a94895e5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.832+0000 7fa060c00640 1 -- 172.21.15.44:0/3696056431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c02c9d0 con 0x5557a8d4d170 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.832+0000 7fa060c00640 1 -- 172.21.15.44:0/3696056431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c02cb70 con 0x5557a8d4d170 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.832+0000 7fa060c00640 1 -- 172.21.15.44:0/3696056431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0344e0 con 0x5557a8d4d170 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.832+0000 7fa075d66740 1 -- 172.21.15.44:0/3696056431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 msgr2=0x5557a8d55dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.832+0000 7fa075d66740 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d55dd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005bc0 tx=0x7fa05c04a040 comp rx=0 tx=0).stop 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 -- 172.21.15.44:0/3696056431 shutdown_connections 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d56670 0x5557a8d53d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 --2- 172.21.15.44:0/3696056431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d4d170 0x5557a8d55dd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 -- 172.21.15.44:0/3696056431 >> 172.21.15.44:0/3696056431 conn(0x5557a8d478a0 msgr2=0x5557a8d53660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 -- 172.21.15.44:0/3696056431 shutdown_connections 2024-04-20T11:31:13.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 -- 172.21.15.44:0/3696056431 wait complete. 2024-04-20T11:31:13.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 -- start start 2024-04-20T11:31:13.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.833+0000 7fa075d66740 1 Processor -- start 2024-04-20T11:31:13.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 0x5557a8d5e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b1090 con 0x5557a8d56670 2024-04-20T11:31:13.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b09c0 con 0x5557a8d4d170 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1adc10 con 0x5557a8d4da90 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa075d66740 10 client.0 _ll_register_callbacks cb 0x7fa05400f880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 0x5557a8d5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa071000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 0x5557a8d5e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42076/0 (socket says 172.21.15.44:42076) 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 -- 172.21.15.44:0/2583234358 learned_addr learned my addr 172.21.15.44:0/2583234358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa063400640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 -- 172.21.15.44:0/2583234358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 msgr2=0x5557a8d5e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d5e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 -- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 msgr2=0x5557a8d598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa06be00640 1 -- 172.21.15.44:0/2583234358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a8a8e7d0 con 0x5557a8d4d170 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa063400640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d5e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.834+0000 7fa071000640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa06be00640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 0x5557a8d5e430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa05c036040 tx=0x7fa05c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff4360b3442f809 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa05a000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa05a000640 1 -- 172.21.15.44:0/2583234358 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c053230 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c02cb70 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa05a000640 1 -- 172.21.15.44:0/2583234358 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fa05c0533d0 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa05a000640 1 -- 172.21.15.44:0/2583234358 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa05c053610 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa075d66740 10 client.27800 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.835+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5557a8dcc400 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa05a000640 1 -- 172.21.15.44:0/2583234358 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa05c02cb70 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa05a000640 1 -- 172.21.15.44:0/2583234358 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fa05c03f430 con 0x5557a8d4d170 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa075d66740 10 client.27800 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa075d66740 10 client.27800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa075d66740 20 client.27800 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa075d66740 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8dcc5d0 0x5557a8dce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa075d66740 4 client.27800 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557a8cb98e0 con 0x5557a8dcc5d0 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.836+0000 7fa063400640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8dcc5d0 0x5557a8dce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa063400640 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8dcc5d0 0x5557a8dce9c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d98a7422bb7c0bc server_cookie=9cf5dfc6b097f112 in_seq=0 out_seq=0 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa05a000640 10 client.27800 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa05a000640 1 -- 172.21.15.44:0/2583234358 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557a8cb98e0 con 0x5557a8dcc5d0 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa05a000640 10 client.27800 handle_command_reply: tid=0 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 client.27800 shutdown 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 msgr2=0x5557a8d5e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 0x5557a8d5e430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa05c036040 tx=0x7fa05c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 shutdown_connections 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x5557a8dcc5d0 0x5557a8dce9c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5557a8d56670 0x5557a8d598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557a8d4da90 0x5557a8d5e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 --2- 172.21.15.44:0/2583234358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557a8d4d170 0x5557a8d5e430 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa05a000640 0 client.27800 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:13.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.837+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 >> 172.21.15.44:0/2583234358 conn(0x5557a8d478a0 msgr2=0x5557a8d46960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 shutdown_connections 2024-04-20T11:31:13.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa075d66740 1 -- 172.21.15.44:0/2583234358 wait complete. 2024-04-20T11:31:13.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa075d66740 20 client.27800 trim_cache size 0 max 16384 2024-04-20T11:31:13.842 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0500a3da0 msgr2=0x7fa0500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0500a3da0 0x7fa0500a6260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5557a8d48a30 tx=0x7fa054015040 comp rx=0 tx=0).stop 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c19a840 msgr2=0x7fa06c19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c19a840 0x7fa06c19acf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa064009fd0 tx=0x7fa06400ef10 comp rx=0 tx=0).stop 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 shutdown_connections 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa06c19a840 0x7fa06c19acf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa06c109340 0x7fa06c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa0500a3da0 0x7fa0500a6260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 --2- 172.21.15.44:0/1559995564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c104270 0x7fa06c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.838+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 >> 172.21.15.44:0/1559995564 conn(0x7fa06c0ffad0 msgr2=0x7fa06c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.839+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 shutdown_connections 2024-04-20T11:31:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:13.839+0000 7fa071a00640 1 -- 172.21.15.44:0/1559995564 wait complete. 2024-04-20T11:31:13.857 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:31:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:13 smithi078 ceph-mon[21464]: pgmap v266: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:13 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4219945077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:31:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:13 smithi078 ceph-mon[21464]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T11:31:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:13 smithi165 ceph-mon[21975]: pgmap v266: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:13 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4219945077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:31:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:13 smithi165 ceph-mon[21975]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T11:31:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 -- 172.21.15.44:0/1439869752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 msgr2=0x7f721c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 --2- 172.21.15.44:0/1439869752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1047f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f720c009900 tx=0x7f720c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:31:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 -- 172.21.15.44:0/1439869752 shutdown_connections 2024-04-20T11:31:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 --2- 172.21.15.44:0/1439869752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721c1094e0 0x7f721c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 --2- 172.21.15.44:0/1439869752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c104dc0 0x7f721c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 --2- 172.21.15.44:0/1439869752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1047f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.058+0000 7f7221800640 1 -- 172.21.15.44:0/1439869752 >> 172.21.15.44:0/1439869752 conn(0x7f721c0ffcf0 msgr2=0x7f721c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.059+0000 7f7221800640 1 -- 172.21.15.44:0/1439869752 shutdown_connections 2024-04-20T11:31:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.059+0000 7f7221800640 1 -- 172.21.15.44:0/1439869752 wait complete. 2024-04-20T11:31:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 Processor -- start 2024-04-20T11:31:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 -- start start 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1094e0 0x7f721c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721c19aa80 0x7f721c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f721c114680 con 0x7f721c19aa80 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f721c114500 con 0x7f721c104410 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.060+0000 7f7221800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f721c114800 con 0x7f721c1094e0 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1094e0 0x7f721c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42092/0 (socket says 172.21.15.44:42092) 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721aa00640 1 -- 172.21.15.44:0/2888817890 learned_addr learned my addr 172.21.15.44:0/2888817890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721b400640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721c19aa80 0x7f721c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 -- 172.21.15.44:0/2888817890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 msgr2=0x7f721c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 -- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721c19aa80 msgr2=0x7f721c19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721c19aa80 0x7f721c19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 -- 172.21.15.44:0/2888817890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c19b5a0 con 0x7f721c1094e0 2024-04-20T11:31:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.061+0000 7f721a000640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1094e0 0x7f721c1a0dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7204005fd0 tx=0x7f720400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.062+0000 7f7213400640 1 -- 172.21.15.44:0/2888817890 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204004b20 con 0x7f721c1094e0 2024-04-20T11:31:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.062+0000 7f7213400640 1 -- 172.21.15.44:0/2888817890 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7204004cc0 con 0x7f721c1094e0 2024-04-20T11:31:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.062+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c19b890 con 0x7f721c1094e0 2024-04-20T11:31:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.062+0000 7f7213400640 1 -- 172.21.15.44:0/2888817890 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040107e0 con 0x7f721c1094e0 2024-04-20T11:31:14.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.062+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c1a7a40 con 0x7f721c1094e0 2024-04-20T11:31:14.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.065+0000 7f72245de740 1 Processor -- start 2024-04-20T11:31:14.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.065+0000 7f72245de740 1 -- start start 2024-04-20T11:31:14.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f72245de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb3d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f7213400640 1 -- 172.21.15.44:0/2888817890 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7204005020 con 0x7f721c1094e0 2024-04-20T11:31:14.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f72245de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3d8d0 0x556a7cb45f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f72245de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb464b0 0x556a7cb488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f72245de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556a7caa2050 con 0x556a7cb3d8d0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f72245de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a7ca48490 con 0x556a7cb464b0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f72245de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556a7ca9e010 con 0x556a7cb3cfb0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f721b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb3d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f721aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3d8d0 0x556a7cb45f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f721b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb3d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58784/0 (socket says 172.21.15.44:58784) 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f721b400640 1 -- 172.21.15.44:0/2805352385 learned_addr learned my addr 172.21.15.44:0/2805352385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.066+0000 7f721a000640 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb464b0 0x556a7cb488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721aa00640 1 -- 172.21.15.44:0/2805352385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 msgr2=0x556a7cb3d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721aa00640 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb3d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721aa00640 1 -- 172.21.15.44:0/2805352385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb464b0 msgr2=0x556a7cb488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721aa00640 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb464b0 0x556a7cb488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721aa00640 1 -- 172.21.15.44:0/2805352385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7ca50070 con 0x556a7cb3d8d0 2024-04-20T11:31:14.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721b400640 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb3d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:14.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f721aa00640 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3d8d0 0x556a7cb45f70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f720c005ec0 tx=0x7f720c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f44028fd6fadc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f720be00640 1 -- 172.21.15.44:0/2805352385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c02c790 con 0x556a7cb3d8d0 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f720be00640 1 -- 172.21.15.44:0/2805352385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f720c004040 con 0x556a7cb3d8d0 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.067+0000 7f720be00640 1 -- 172.21.15.44:0/2805352385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c0055c0 con 0x556a7cb3d8d0 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f7213400640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71fc0a3d20 0x7f71fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f7213400640 1 -- 172.21.15.44:0/2888817890 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f72040d0fe0 con 0x7f721c1094e0 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f721b400640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71fc0a3d20 0x7f71fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 -- 172.21.15.44:0/2805352385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3d8d0 msgr2=0x556a7cb45f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3d8d0 0x556a7cb45f70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f720c005ec0 tx=0x7f720c0095f0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 -- 172.21.15.44:0/2805352385 shutdown_connections 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb464b0 0x556a7cb488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3d8d0 0x556a7cb45f70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 --2- 172.21.15.44:0/2805352385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb3d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 -- 172.21.15.44:0/2805352385 >> 172.21.15.44:0/2805352385 conn(0x556a7cb376e0 msgr2=0x556a7cb37ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.068+0000 7f72245de740 1 -- 172.21.15.44:0/2805352385 shutdown_connections 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.069+0000 7f72245de740 1 -- 172.21.15.44:0/2805352385 wait complete. 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.069+0000 7f721b400640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71fc0a3d20 0x7f71fc0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7214009f50 tx=0x7f7214005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.069+0000 7f72245de740 1 -- start start 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.069+0000 7f72245de740 1 Processor -- start 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb43500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb43a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb464b0 0x556a7cb43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556a7cb49240 con 0x556a7cb464b0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a7ca5b860 con 0x556a7cb3d8d0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556a7cb493c0 con 0x556a7cb3cfb0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f72245de740 10 client.0 _ll_register_callbacks cb 0x556a7ca29fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f721a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb43a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f721b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb464b0 0x556a7cb43f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f721a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb43a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42114/0 (socket says 172.21.15.44:42114) 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f721a000640 1 -- 172.21.15.44:0/916699263 learned_addr learned my addr 172.21.15.44:0/916699263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.070+0000 7f721aa00640 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb43500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f721a000640 1 -- 172.21.15.44:0/916699263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 msgr2=0x556a7cb43500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f721a000640 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb43500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f721a000640 1 -- 172.21.15.44:0/916699263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb464b0 msgr2=0x556a7cb43f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f721a000640 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb464b0 0x556a7cb43f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f721a000640 1 -- 172.21.15.44:0/916699263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7ca5f950 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f721a000640 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb43a40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f72040ce7b0 tx=0x7f720400ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a657f84099797bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f7208c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f7208c00640 1 -- 172.21.15.44:0/916699263 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720409cae0 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f7208c00640 1 -- 172.21.15.44:0/916699263 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f72040d30d0 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720c004040 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f7208c00640 1 -- 172.21.15.44:0/916699263 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f720409c900 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f72245de740 10 client.27812 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.071+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556a7ca2d820 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f7208c00640 1 -- 172.21.15.44:0/916699263 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f720c004040 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f7208c00640 1 -- 172.21.15.44:0/916699263 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f72040ce9d0 con 0x556a7cb3d8d0 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f72245de740 10 client.27812 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f72245de740 10 client.27812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:14.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f72245de740 20 client.27812 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f72245de740 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cb564c0 0x556a7cb588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f72245de740 4 client.27812 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a7caafa70 con 0x556a7cb564c0 2024-04-20T11:31:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.072+0000 7f721aa00640 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cb564c0 0x556a7cb588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.073+0000 7f721aa00640 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cb564c0 0x556a7cb588b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a1ebaaea6962cd server_cookie=cb48ebb993b77121 in_seq=0 out_seq=0 2024-04-20T11:31:14.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.073+0000 7f7208c00640 10 client.27812 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f7208c00640 1 -- 172.21.15.44:0/916699263 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a7caafa70 con 0x556a7cb564c0 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f7208c00640 10 client.27812 handle_command_reply: tid=0 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 client.27812 shutdown 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 msgr2=0x556a7cb43a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb43a40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f72040ce7b0 tx=0x7f720400ce70 comp rx=0 tx=0).stop 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 shutdown_connections 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cb564c0 0x556a7cb588b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb464b0 0x556a7cb43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb43a40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 --2- 172.21.15.44:0/916699263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb3cfb0 0x556a7cb43500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f7208c00640 0 client.27812 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:14.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 >> 172.21.15.44:0/916699263 conn(0x556a7cb376e0 msgr2=0x556a7cb39430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 shutdown_connections 2024-04-20T11:31:14.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 1 -- 172.21.15.44:0/916699263 wait complete. 2024-04-20T11:31:14.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.074+0000 7f72245de740 20 client.27812 trim_cache size 0 max 16384 2024-04-20T11:31:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.089+0000 7f72245de740 1 Processor -- start 2024-04-20T11:31:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 -- start start 2024-04-20T11:31:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb45c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb52100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb43bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1afde0 con 0x556a7cb3cfb0 2024-04-20T11:31:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1b1920 con 0x556a7cb3d8d0 2024-04-20T11:31:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f72245de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1b04b0 con 0x556a7cb464b0 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f721aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb45c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f721a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f721aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb45c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43134/0 (socket says 172.21.15.44:43134) 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f721aa00640 1 -- 172.21.15.44:0/374986303 learned_addr learned my addr 172.21.15.44:0/374986303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.090+0000 7f721b400640 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb43bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721a000640 1 -- 172.21.15.44:0/374986303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 msgr2=0x556a7cb43bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721a000640 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb43bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721a000640 1 -- 172.21.15.44:0/374986303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 msgr2=0x556a7cb45c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721a000640 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb45c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721a000640 1 -- 172.21.15.44:0/374986303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7c7b43f0 con 0x556a7cb3d8d0 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721aa00640 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb45c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f721a000640 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb52100 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7204099400 tx=0x7f7204005520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22b713dfb84ddfe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f720b400640 1 -- 172.21.15.44:0/374986303 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204099240 con 0x556a7cb3d8d0 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f720b400640 1 -- 172.21.15.44:0/374986303 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72040b5020 con 0x556a7cb3d8d0 2024-04-20T11:31:14.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.091+0000 7f720b400640 1 -- 172.21.15.44:0/374986303 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040055b0 con 0x556a7cb3d8d0 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 -- 172.21.15.44:0/374986303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 msgr2=0x556a7cb52100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb52100 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7204099400 tx=0x7f7204005520 comp rx=0 tx=0).stop 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 -- 172.21.15.44:0/374986303 shutdown_connections 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb43bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb52100 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 --2- 172.21.15.44:0/374986303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb45c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 -- 172.21.15.44:0/374986303 >> 172.21.15.44:0/374986303 conn(0x556a7cb376e0 msgr2=0x556a7cb434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 -- 172.21.15.44:0/374986303 shutdown_connections 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.092+0000 7f72245de740 1 -- 172.21.15.44:0/374986303 wait complete. 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 -- start start 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 Processor -- start 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb40000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1b1250 con 0x556a7cb3cfb0 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1b0b80 con 0x556a7cb3d8d0 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1addd0 con 0x556a7cb464b0 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f72245de740 10 client.0 _ll_register_callbacks cb 0x7f72040d47c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f721aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.093+0000 7f721aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb4eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43140/0 (socket says 172.21.15.44:43140) 2024-04-20T11:31:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 -- 172.21.15.44:0/3786888617 learned_addr learned my addr 172.21.15.44:0/3786888617 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721a000640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721b400640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 -- 172.21.15.44:0/3786888617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 msgr2=0x556a7cb406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 -- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 msgr2=0x556a7cb40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb40000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 -- 172.21.15.44:0/3786888617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7c87e7d0 con 0x556a7cb3cfb0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721a000640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb40000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f721aa00640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb4eb80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f720c006010 tx=0x7f720c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbf2ec38e32413c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f7208c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.094+0000 7f7208c00640 1 -- 172.21.15.44:0/3786888617 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c058060 con 0x556a7cb3cfb0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f7208c00640 1 -- 172.21.15.44:0/3786888617 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f720c02c9d0 con 0x556a7cb3cfb0 2024-04-20T11:31:14.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72040b5020 con 0x556a7cb3cfb0 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f7208c00640 1 -- 172.21.15.44:0/3786888617 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f720c02cbd0 con 0x556a7cb3cfb0 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 10 client.17700 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x556a7cbbca20 con 0x556a7cb3cfb0 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f7208c00640 1 -- 172.21.15.44:0/3786888617 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72040b5020 con 0x556a7cb3cfb0 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f7208c00640 1 -- 172.21.15.44:0/3786888617 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f720c0344e0 con 0x556a7cb3cfb0 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 10 client.17700 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 20 client.17700 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cbbcd80 0x556a7cbbf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 4 client.17700 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:14.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.095+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a7caa98e0 con 0x556a7cbbcd80 2024-04-20T11:31:14.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.096+0000 7f721a000640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cbbcd80 0x556a7cbbf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f721a000640 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cbbcd80 0x556a7cbbf1a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fa4c5c177096ac server_cookie=d39352778bcc668e in_seq=0 out_seq=0 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f7208c00640 10 client.17700 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f7208c00640 1 -- 172.21.15.44:0/3786888617 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a7caa98e0 con 0x556a7cbbcd80 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f7208c00640 10 client.17700 handle_command_reply: tid=0 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 client.17700 shutdown 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 msgr2=0x556a7cb4eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb4eb80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f720c006010 tx=0x7f720c005e00 comp rx=0 tx=0).stop 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 shutdown_connections 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x556a7cbbcd80 0x556a7cbbf1a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a7cb464b0 0x556a7cb406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f7208c00640 0 client.17700 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7cb3d8d0 0x556a7cb40000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 --2- 172.21.15.44:0/3786888617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556a7cb3cfb0 0x556a7cb4eb80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 >> 172.21.15.44:0/3786888617 conn(0x556a7cb376e0 msgr2=0x556a7cb43d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 shutdown_connections 2024-04-20T11:31:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 1 -- 172.21.15.44:0/3786888617 wait complete. 2024-04-20T11:31:14.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.097+0000 7f72245de740 20 client.17700 trim_cache size 0 max 16384 2024-04-20T11:31:14.102 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71fc0a3d20 msgr2=0x7f71fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71fc0a3d20 0x7f71fc0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7214009f50 tx=0x7f7214005f90 comp rx=0 tx=0).stop 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1094e0 msgr2=0x7f721c1a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1094e0 0x7f721c1a0dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7204005fd0 tx=0x7f720400ef90 comp rx=0 tx=0).stop 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 shutdown_connections 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71fc0a3d20 0x7f71fc0a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f721c19aa80 0x7f721c19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1094e0 0x7f721c1a0dc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 --2- 172.21.15.44:0/2888817890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f721c104410 0x7f721c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 >> 172.21.15.44:0/2888817890 conn(0x7f721c0ffcf0 msgr2=0x7f721c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:14.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 shutdown_connections 2024-04-20T11:31:14.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:14.098+0000 7f7221800640 1 -- 172.21.15.44:0/2888817890 wait complete. 2024-04-20T11:31:15.117 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:31:15.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:15.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:31:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 -- 172.21.15.44:0/17304583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc1009b0 msgr2=0x7f12dc100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 --2- 172.21.15.44:0/17304583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc1009b0 0x7f12dc100d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009900 tx=0x7f12cc02bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 -- 172.21.15.44:0/17304583 shutdown_connections 2024-04-20T11:31:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 --2- 172.21.15.44:0/17304583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1071c0 0x7f12dc10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 --2- 172.21.15.44:0/17304583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc101360 0x7f12dc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 --2- 172.21.15.44:0/17304583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc1009b0 0x7f12dc100d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 -- 172.21.15.44:0/17304583 >> 172.21.15.44:0/17304583 conn(0x7f12dc075ff0 msgr2=0x7f12dc076400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 -- 172.21.15.44:0/17304583 shutdown_connections 2024-04-20T11:31:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.346+0000 7f12db200640 1 -- 172.21.15.44:0/17304583 wait complete. 2024-04-20T11:31:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 Processor -- start 2024-04-20T11:31:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 -- start start 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 0x7f12dc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 0x7f12dc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc19a860 0x7f12dc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc112340 con 0x7f12dc1071c0 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1121c0 con 0x7f12dc19a860 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.348+0000 7f12db200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1124c0 con 0x7f12dc1009b0 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 0x7f12dc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 0x7f12dc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 0x7f12dc1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43158/0 (socket says 172.21.15.44:43158) 2024-04-20T11:31:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9400640 1 -- 172.21.15.44:0/2652440996 learned_addr learned my addr 172.21.15.44:0/2652440996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 0x7f12dc1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58824/0 (socket says 172.21.15.44:58824) 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12da800640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc19a860 0x7f12dc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 -- 172.21.15.44:0/2652440996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc19a860 msgr2=0x7f12dc19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc19a860 0x7f12dc19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 -- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 msgr2=0x7f12dc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 0x7f12dc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9e00640 1 -- 172.21.15.44:0/2652440996 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc19b380 con 0x7f12dc1009b0 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12d9400640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 0x7f12dc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.349+0000 7f12da800640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc19a860 0x7f12dc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.350+0000 7f12d9e00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 0x7f12dc1a06b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f12cc036040 tx=0x7f12cc02dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.350+0000 7f12d2a00640 1 -- 172.21.15.44:0/2652440996 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc043020 con 0x7f12dc1009b0 2024-04-20T11:31:15.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.350+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc19b610 con 0x7f12dc1009b0 2024-04-20T11:31:15.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.350+0000 7f12d2a00640 1 -- 172.21.15.44:0/2652440996 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12cc004e50 con 0x7f12dc1009b0 2024-04-20T11:31:15.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.350+0000 7f12d2a00640 1 -- 172.21.15.44:0/2652440996 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc003970 con 0x7f12dc1009b0 2024-04-20T11:31:15.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.350+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1a78b0 con 0x7f12dc1009b0 2024-04-20T11:31:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.353+0000 7f12e3d68740 1 Processor -- start 2024-04-20T11:31:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.353+0000 7f12e3d68740 1 -- start start 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2ca10 0x557b95b350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12e3d68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b95a90d20 con 0x557b95b2ca10 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12e3d68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b95a373e0 con 0x557b95b2c0f0 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12e3d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b95a8cce0 con 0x557b95b355f0 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12d2a00640 1 -- 172.21.15.44:0/2652440996 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f12cc046020 con 0x7f12dc1009b0 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12da800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12da800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b2c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42154/0 (socket says 172.21.15.44:42154) 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12da800640 1 -- 172.21.15.44:0/1804980797 learned_addr learned my addr 172.21.15.44:0/1804980797 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.354+0000 7f12d9e00640 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2ca10 0x557b95b350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12da800640 1 -- 172.21.15.44:0/1804980797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 msgr2=0x557b95b379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12da800640 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12da800640 1 -- 172.21.15.44:0/1804980797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2ca10 msgr2=0x557b95b350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12da800640 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2ca10 0x557b95b350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12da800640 1 -- 172.21.15.44:0/1804980797 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b95a56980 con 0x557b95b2c0f0 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12d9400640 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12da800640 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b2c4d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009fd0 tx=0x7f12d4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6444484ded3642 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12c7400640 1 -- 172.21.15.44:0/1804980797 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011700 con 0x557b95b2c0f0 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12c7400640 1 -- 172.21.15.44:0/1804980797 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d4011da0 con 0x557b95b2c0f0 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.355+0000 7f12c7400640 1 -- 172.21.15.44:0/1804980797 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400e620 con 0x557b95b2c0f0 2024-04-20T11:31:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12d2a00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3da0 0x7f12c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12d2a00640 1 -- 172.21.15.44:0/2652440996 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f12cc0f25d0 con 0x7f12dc1009b0 2024-04-20T11:31:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 -- 172.21.15.44:0/1804980797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 msgr2=0x557b95b2c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b2c4d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009fd0 tx=0x7f12d4010860 comp rx=0 tx=0).stop 2024-04-20T11:31:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12d9e00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3da0 0x7f12c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 -- 172.21.15.44:0/1804980797 shutdown_connections 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2ca10 0x557b95b350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1804980797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b2c4d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 -- 172.21.15.44:0/1804980797 >> 172.21.15.44:0/1804980797 conn(0x557b95b26820 msgr2=0x557b95b26c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 -- 172.21.15.44:0/1804980797 shutdown_connections 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.356+0000 7f12e3d68740 1 -- 172.21.15.44:0/1804980797 wait complete. 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.357+0000 7f12d9e00640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3da0 0x7f12c00a6260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x557b95b2bbc0 tx=0x7f12cc02bc40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.357+0000 7f12e3d68740 1 -- start start 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.357+0000 7f12e3d68740 1 Processor -- start 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b355f0 0x557b95b33100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b95b383e0 con 0x557b95b355f0 2024-04-20T11:31:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b95a4a7a0 con 0x557b95b2c0f0 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b95b38560 con 0x557b95b2ca10 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12e3d68740 10 client.0 _ll_register_callbacks cb 0x557b95a4cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b355f0 0x557b95b33100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12da800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b32bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58836/0 (socket says 172.21.15.44:58836) 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 -- 172.21.15.44:0/299361225 learned_addr learned my addr 172.21.15.44:0/299361225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 -- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 msgr2=0x557b95b32680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b32680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 -- 172.21.15.44:0/299361225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b355f0 msgr2=0x557b95b33100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b355f0 0x557b95b33100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12d9400640 1 -- 172.21.15.44:0/299361225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b9586b7e0 con 0x557b95b2ca10 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.358+0000 7f12da800640 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b32680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.359+0000 7f12d9400640 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b32bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12c8012d20 tx=0x7f12c800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e77066d16b2f997 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.359+0000 7f12bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.359+0000 7f12bbe00640 1 -- 172.21.15.44:0/299361225 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c80120d0 con 0x557b95b2ca10 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.359+0000 7f12bbe00640 1 -- 172.21.15.44:0/299361225 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f12c80047a0 con 0x557b95b2ca10 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.359+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d4011da0 con 0x557b95b2ca10 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12bbe00640 1 -- 172.21.15.44:0/299361225 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12c8012270 con 0x557b95b2ca10 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 10 client.27835 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557b958cbf70 con 0x557b95b2ca10 2024-04-20T11:31:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12bbe00640 1 -- 172.21.15.44:0/299361225 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d4011da0 con 0x557b95b2ca10 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12bbe00640 1 -- 172.21.15.44:0/299361225 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f12c801a5a0 con 0x557b95b2ca10 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 10 client.27835 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 10 client.27835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 20 client.27835 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95b45b30 0x557b95b47f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 4 client.27835 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.360+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b95a9e740 con 0x557b95b45b30 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.361+0000 7f12da800640 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95b45b30 0x557b95b47f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.361+0000 7f12da800640 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95b45b30 0x557b95b47f50 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85097b2bcec73114 server_cookie=9a2a29b6a38b1d94 in_seq=0 out_seq=0 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.361+0000 7f12bbe00640 10 client.27835 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12bbe00640 1 -- 172.21.15.44:0/299361225 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b95a9e740 con 0x557b95b45b30 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12bbe00640 10 client.27835 handle_command_reply: tid=0 2024-04-20T11:31:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 client.27835 shutdown 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 msgr2=0x557b95b32bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b32bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12c8012d20 tx=0x7f12c800f790 comp rx=0 tx=0).stop 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 shutdown_connections 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95b45b30 0x557b95b47f50 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b355f0 0x557b95b33100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b32bc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 --2- 172.21.15.44:0/299361225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2c0f0 0x557b95b32680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 >> 172.21.15.44:0/299361225 conn(0x557b95b26820 msgr2=0x557b95b28570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.362+0000 7f12bbe00640 0 client.27835 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.363+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 shutdown_connections 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.363+0000 7f12e3d68740 1 -- 172.21.15.44:0/299361225 wait complete. 2024-04-20T11:31:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.363+0000 7f12e3d68740 20 client.27835 trim_cache size 0 max 16384 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.382+0000 7f12e3d68740 1 Processor -- start 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.382+0000 7f12e3d68740 1 -- start start 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2ca10 0x557b95b41300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12e3d68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1afdd0 con 0x557b95b2c0f0 2024-04-20T11:31:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12e3d68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1b1910 con 0x557b95b2ca10 2024-04-20T11:31:15.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12e3d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1b04a0 con 0x557b95b355f0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b34db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b34db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43186/0 (socket says 172.21.15.44:43186) 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 -- 172.21.15.44:0/719821280 learned_addr learned my addr 172.21.15.44:0/719821280 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12d9400640 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2ca10 0x557b95b41300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 -- 172.21.15.44:0/719821280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 msgr2=0x557b95b4d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b4d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 -- 172.21.15.44:0/719821280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2ca10 msgr2=0x557b95b41300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.383+0000 7f12da800640 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2ca10 0x557b95b41300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.384+0000 7f12da800640 1 -- 172.21.15.44:0/719821280 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b95b2c840 con 0x557b95b2c0f0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.384+0000 7f12da800640 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b34db0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f12d401bba0 tx=0x7f12d4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e114fe294e242bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.384+0000 7f12c6a00640 1 -- 172.21.15.44:0/719821280 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d40049c0 con 0x557b95b2c0f0 2024-04-20T11:31:15.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.384+0000 7f12c6a00640 1 -- 172.21.15.44:0/719821280 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d4004b60 con 0x557b95b2c0f0 2024-04-20T11:31:15.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.384+0000 7f12c6a00640 1 -- 172.21.15.44:0/719821280 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d402f320 con 0x557b95b2c0f0 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 -- 172.21.15.44:0/719821280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 msgr2=0x557b95b34db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b34db0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f12d401bba0 tx=0x7f12d4009500 comp rx=0 tx=0).stop 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 -- 172.21.15.44:0/719821280 shutdown_connections 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b355f0 0x557b95b4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b2ca10 0x557b95b41300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 --2- 172.21.15.44:0/719821280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b34db0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 -- 172.21.15.44:0/719821280 >> 172.21.15.44:0/719821280 conn(0x557b95b26820 msgr2=0x557b95b284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 -- 172.21.15.44:0/719821280 shutdown_connections 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 -- 172.21.15.44:0/719821280 wait complete. 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.385+0000 7f12e3d68740 1 -- start start 2024-04-20T11:31:15.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 Processor -- start 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b355f0 0x557b95b3a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1b1240 con 0x557b95b2c0f0 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1b0b70 con 0x557b95b355f0 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1addc0 con 0x557b95b2ca10 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12e3d68740 10 client.0 _ll_register_callbacks cb 0x7f12c8009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b3d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43192/0 (socket says 172.21.15.44:43192) 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12da800640 1 -- 172.21.15.44:0/1998073035 learned_addr learned my addr 172.21.15.44:0/1998073035 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.386+0000 7f12d9400640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b355f0 0x557b95b3a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 -- 172.21.15.44:0/1998073035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 msgr2=0x557b95b3dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b3dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 -- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b355f0 msgr2=0x557b95b3a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b355f0 0x557b95b3a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 -- 172.21.15.44:0/1998073035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b95763170 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12da800640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b3d810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f12d4010d90 tx=0x7f12d401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3539d2a0797fbdca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12bbe00640 1 -- 172.21.15.44:0/1998073035 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d402f2c0 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12bbe00640 1 -- 172.21.15.44:0/1998073035 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f12d402f460 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.387+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d4004b60 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12bbe00640 1 -- 172.21.15.44:0/1998073035 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12d4041050 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 10 client.17718 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557b95bacac0 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12bbe00640 1 -- 172.21.15.44:0/1998073035 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d4004b60 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12bbe00640 1 -- 172.21.15.44:0/1998073035 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f12d4029040 con 0x557b95b2c0f0 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 10 client.17718 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:15.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 20 client.17718 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:15.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95bacf40 0x557b95baf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 4 client.17718 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:31:15.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.388+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557b95a985b0 con 0x557b95bacf40 2024-04-20T11:31:15.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.389+0000 7f12d9400640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95bacf40 0x557b95baf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.389+0000 7f12d9400640 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95bacf40 0x557b95baf330 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dff1b4d76b74fcb server_cookie=7a9504d5308e6c33 in_seq=0 out_seq=0 2024-04-20T11:31:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.389+0000 7f12bbe00640 10 client.17718 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12bbe00640 1 -- 172.21.15.44:0/1998073035 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557b95a985b0 con 0x557b95bacf40 2024-04-20T11:31:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12bbe00640 10 client.17718 handle_command_reply: tid=0 2024-04-20T11:31:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 client.17718 shutdown 2024-04-20T11:31:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 msgr2=0x557b95b3d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b3d810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f12d4010d90 tx=0x7f12d401ba30 comp rx=0 tx=0).stop 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 shutdown_connections 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557b95bacf40 0x557b95baf330 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b95b355f0 0x557b95b3a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b95b2ca10 0x557b95b3dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 --2- 172.21.15.44:0/1998073035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b95b2c0f0 0x557b95b3d810 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12bbe00640 0 client.17718 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 >> 172.21.15.44:0/1998073035 conn(0x557b95b26820 msgr2=0x557b95b27c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 shutdown_connections 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.391+0000 7f12e3d68740 1 -- 172.21.15.44:0/1998073035 wait complete. 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12e3d68740 20 client.17718 trim_cache size 0 max 16384 2024-04-20T11:31:15.395 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:31:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:31:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "5cd7d1ea-c56e-47ff-a5ba-6df547f72974", 2024-04-20T11:31:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:31:15.396 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:31:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3da0 msgr2=0x7f12c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3da0 0x7f12c00a6260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x557b95b2bbc0 tx=0x7f12cc02bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 msgr2=0x7f12dc1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 0x7f12dc1a06b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f12cc036040 tx=0x7f12cc02dc00 comp rx=0 tx=0).stop 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 shutdown_connections 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12dc19a860 0x7f12dc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3da0 0x7f12c00a6260 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12dc1071c0 0x7f12dc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 --2- 172.21.15.44:0/2652440996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12dc1009b0 0x7f12dc1a06b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 >> 172.21.15.44:0/2652440996 conn(0x7f12dc075ff0 msgr2=0x7f12dc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.392+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 shutdown_connections 2024-04-20T11:31:15.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.393+0000 7f12db200640 1 -- 172.21.15.44:0/2652440996 wait complete. 2024-04-20T11:31:15.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:31:15.411 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:31:15.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 -- 172.21.15.44:0/2483095772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc1043b0 msgr2=0x7f16fc104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 --2- 172.21.15.44:0/2483095772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc1043b0 0x7f16fc104790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009ca0 tx=0x7f16ec02c0f0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 -- 172.21.15.44:0/2483095772 shutdown_connections 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 --2- 172.21.15.44:0/2483095772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc109480 0x7f16fc111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 --2- 172.21.15.44:0/2483095772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc104d60 0x7f16fc108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 --2- 172.21.15.44:0/2483095772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc1043b0 0x7f16fc104790 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 -- 172.21.15.44:0/2483095772 >> 172.21.15.44:0/2483095772 conn(0x7f16fc0ffbd0 msgr2=0x7f16fc101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 -- 172.21.15.44:0/2483095772 shutdown_connections 2024-04-20T11:31:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.606+0000 7f1704200640 1 -- 172.21.15.44:0/2483095772 wait complete. 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 Processor -- start 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 -- start start 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc1043b0 0x7f16fc1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc109480 0x7f16fc1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 0x7f16fc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc114670 con 0x7f16fc19a990 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1144f0 con 0x7f16fc109480 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.607+0000 7f1704200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1147f0 con 0x7f16fc1043b0 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 0x7f16fc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1701800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc1043b0 0x7f16fc1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 0x7f16fc19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43212/0 (socket says 172.21.15.44:43212) 2024-04-20T11:31:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 -- 172.21.15.44:0/1141443870 learned_addr learned my addr 172.21.15.44:0/1141443870 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1700e00640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc109480 0x7f16fc1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 -- 172.21.15.44:0/1141443870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc1043b0 msgr2=0x7f16fc1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc1043b0 0x7f16fc1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 -- 172.21.15.44:0/1141443870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc109480 msgr2=0x7f16fc1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc109480 0x7f16fc1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 -- 172.21.15.44:0/1141443870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc19b4b0 con 0x7f16fc19a990 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1701800640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc1043b0 0x7f16fc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1700e00640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc109480 0x7f16fc1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.608+0000 7f1702200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 0x7f16fc19ae40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e970 tx=0x7f16f400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.609+0000 7f16fa000640 1 -- 172.21.15.44:0/1141443870 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400ccc0 con 0x7f16fc19a990 2024-04-20T11:31:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.609+0000 7f16fa000640 1 -- 172.21.15.44:0/1141443870 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16f400ce60 con 0x7f16fc19a990 2024-04-20T11:31:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.609+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc19b7a0 con 0x7f16fc19a990 2024-04-20T11:31:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.609+0000 7f16fa000640 1 -- 172.21.15.44:0/1141443870 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4010690 con 0x7f16fc19a990 2024-04-20T11:31:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.609+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc1a79a0 con 0x7f16fc19a990 2024-04-20T11:31:15.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.612+0000 7f1706fab740 1 Processor -- start 2024-04-20T11:31:15.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.612+0000 7f16fa000640 1 -- 172.21.15.44:0/1141443870 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f16f40046c0 con 0x7f16fc19a990 2024-04-20T11:31:15.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 -- start start 2024-04-20T11:31:15.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125def580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125df8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125df86a0 0x565125dfaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565125d55150 con 0x565125def1a0 2024-04-20T11:31:15.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565125cfb370 con 0x565125df86a0 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1706fab740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565125d51110 con 0x565125defac0 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1701800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125df8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1700e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125def580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1701800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125df8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58882/0 (socket says 172.21.15.44:58882) 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.613+0000 7f1701800640 1 -- 172.21.15.44:0/1218818762 learned_addr learned my addr 172.21.15.44:0/1218818762 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1700e00640 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125df86a0 0x565125dfaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1701800640 1 -- 172.21.15.44:0/1218818762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125df86a0 msgr2=0x565125dfaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1701800640 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125df86a0 0x565125dfaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1701800640 1 -- 172.21.15.44:0/1218818762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 msgr2=0x565125def580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1701800640 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125def580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1701800640 1 -- 172.21.15.44:0/1218818762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565125b317d0 con 0x565125defac0 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1700e00640 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125def580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f16fa000640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16e00a3d20 0x7f16e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f16fa000640 1 -- 172.21.15.44:0/1141443870 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f16f40d0c20 con 0x7f16fc19a990 2024-04-20T11:31:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1701800640 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125df8160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f16ec02c630 tx=0x7f16ec035eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba3738c396053f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.614+0000 7f1700e00640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16e00a3d20 0x7f16e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.615+0000 7f16f2a00640 1 -- 172.21.15.44:0/1218818762 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec043070 con 0x565125defac0 2024-04-20T11:31:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.615+0000 7f16f2a00640 1 -- 172.21.15.44:0/1218818762 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16ec032340 con 0x565125defac0 2024-04-20T11:31:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.615+0000 7f16f2a00640 1 -- 172.21.15.44:0/1218818762 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec039040 con 0x565125defac0 2024-04-20T11:31:15.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.615+0000 7f1706fab740 1 -- 172.21.15.44:0/1218818762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 msgr2=0x565125df8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125df8160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f16ec02c630 tx=0x7f16ec035eb0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.615+0000 7f1700e00640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16e00a3d20 0x7f16e00a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x565125deaa60 tx=0x7f16e8009680 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 -- 172.21.15.44:0/1218818762 shutdown_connections 2024-04-20T11:31:15.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125df86a0 0x565125dfaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125df8160 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 --2- 172.21.15.44:0/1218818762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125def580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 -- 172.21.15.44:0/1218818762 >> 172.21.15.44:0/1218818762 conn(0x565125de98d0 msgr2=0x565125de9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 -- 172.21.15.44:0/1218818762 shutdown_connections 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 -- 172.21.15.44:0/1218818762 wait complete. 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 -- start start 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.616+0000 7f1706fab740 1 Processor -- start 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125df56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125df5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125df86a0 0x565125df6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565125dfb430 con 0x565125def1a0 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565125d0e740 con 0x565125defac0 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565125dfb5b0 con 0x565125df86a0 2024-04-20T11:31:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1706fab740 10 client.0 _ll_register_callbacks cb 0x565125d09b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1701800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125df56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1701800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125df56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43232/0 (socket says 172.21.15.44:43232) 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1701800640 1 -- 172.21.15.44:0/597255694 learned_addr learned my addr 172.21.15.44:0/597255694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1700e00640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125df5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.617+0000 7f1702200640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125df86a0 0x565125df6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1701800640 1 -- 172.21.15.44:0/597255694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125df86a0 msgr2=0x565125df6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1701800640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125df86a0 0x565125df6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1701800640 1 -- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 msgr2=0x565125df5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1701800640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125df5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1701800640 1 -- 172.21.15.44:0/597255694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565125a673f0 con 0x565125def1a0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1701800640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125df56e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f16ec006010 tx=0x7f16ec0324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f14c26dfb84b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f16e7400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f16e7400640 1 -- 172.21.15.44:0/597255694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec04c030 con 0x565125def1a0 2024-04-20T11:31:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f16e7400640 1 -- 172.21.15.44:0/597255694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f16ec032880 con 0x565125def1a0 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.618+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ec032340 con 0x565125def1a0 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f16e7400640 1 -- 172.21.15.44:0/597255694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f16ec039070 con 0x565125def1a0 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 10 client.17730 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x565125a270f0 con 0x565125def1a0 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f16e7400640 1 -- 172.21.15.44:0/597255694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16ec032340 con 0x565125def1a0 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f16e7400640 1 -- 172.21.15.44:0/597255694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f16ec0040a0 con 0x565125def1a0 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 10 client.17730 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 20 client.17730 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e08870 0x565125e0ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 4 client.17730 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.619+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565125d62b70 con 0x565125e08870 2024-04-20T11:31:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.620+0000 7f1700e00640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e08870 0x565125e0ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.620+0000 7f1700e00640 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e08870 0x565125e0ac60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2d0440223257e1 server_cookie=1a99e54ef0d412c6 in_seq=0 out_seq=0 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.620+0000 7f16e7400640 10 client.17730 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.621+0000 7f16e7400640 1 -- 172.21.15.44:0/597255694 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565125d62b70 con 0x565125e08870 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.621+0000 7f16e7400640 10 client.17730 handle_command_reply: tid=0 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.621+0000 7f1706fab740 1 client.17730 shutdown 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.621+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 msgr2=0x565125df56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.621+0000 7f1706fab740 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125df56e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f16ec006010 tx=0x7f16ec0324f0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 shutdown_connections 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e08870 0x565125e0ac60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125df86a0 0x565125df6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125df5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f16e7400640 0 client.17730 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 --2- 172.21.15.44:0/597255694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125def1a0 0x565125df56e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 >> 172.21.15.44:0/597255694 conn(0x565125de98d0 msgr2=0x565125deb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 shutdown_connections 2024-04-20T11:31:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 1 -- 172.21.15.44:0/597255694 wait complete. 2024-04-20T11:31:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.622+0000 7f1706fab740 20 client.17730 trim_cache size 0 max 16384 2024-04-20T11:31:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 Processor -- start 2024-04-20T11:31:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 -- start start 2024-04-20T11:31:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125def1a0 0x565125df7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125e042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125df4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1aff60 con 0x565125df86a0 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b1aa0 con 0x565125defac0 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.641+0000 7f1706fab740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b0630 con 0x565125def1a0 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125df4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125df4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43234/0 (socket says 172.21.15.44:43234) 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 -- 172.21.15.44:0/590169285 learned_addr learned my addr 172.21.15.44:0/590169285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1701800640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125def1a0 0x565125df7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1700e00640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125e042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 -- 172.21.15.44:0/590169285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125def1a0 msgr2=0x565125df7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125def1a0 0x565125df7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 -- 172.21.15.44:0/590169285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 msgr2=0x565125e042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.642+0000 7f1702200640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125e042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f1702200640 1 -- 172.21.15.44:0/590169285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565125def8f0 con 0x565125df86a0 2024-04-20T11:31:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f1701800640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125def1a0 0x565125df7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f1700e00640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125e042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f1702200640 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125df4840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f16f40ce3b0 tx=0x7f16f409c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1066eb556c1951 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f16f2000640 1 -- 172.21.15.44:0/590169285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40a5810 con 0x565125df86a0 2024-04-20T11:31:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f16f2000640 1 -- 172.21.15.44:0/590169285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16f409aea0 con 0x565125df86a0 2024-04-20T11:31:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.643+0000 7f16f2000640 1 -- 172.21.15.44:0/590169285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40a5380 con 0x565125df86a0 2024-04-20T11:31:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 -- 172.21.15.44:0/590169285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 msgr2=0x565125df4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125df4840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f16f40ce3b0 tx=0x7f16f409c5c0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 -- 172.21.15.44:0/590169285 shutdown_connections 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125df4840 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125defac0 0x565125e042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 --2- 172.21.15.44:0/590169285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125def1a0 0x565125df7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 -- 172.21.15.44:0/590169285 >> 172.21.15.44:0/590169285 conn(0x565125de98d0 msgr2=0x565125de8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 -- 172.21.15.44:0/590169285 shutdown_connections 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 -- 172.21.15.44:0/590169285 wait complete. 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.644+0000 7f1706fab740 1 -- start start 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 Processor -- start 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 0x565125dfc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125dfc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125dfccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b13d0 con 0x565125df86a0 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b0d00 con 0x565125def1a0 2024-04-20T11:31:15.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1adf50 con 0x565125defac0 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1706fab740 10 client.0 _ll_register_callbacks cb 0x7f16f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1702200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125dfccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1701800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 0x565125dfc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1701800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 0x565125dfc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:42248/0 (socket says 172.21.15.44:42248) 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1701800640 1 -- 172.21.15.44:0/2022048043 learned_addr learned my addr 172.21.15.44:0/2022048043 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.645+0000 7f1700e00640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125dfc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1701800640 1 -- 172.21.15.44:0/2022048043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 msgr2=0x565125dfc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1701800640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125dfc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1701800640 1 -- 172.21.15.44:0/2022048043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 msgr2=0x565125dfccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1701800640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125dfccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1701800640 1 -- 172.21.15.44:0/2022048043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565125e6daf0 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1702200640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125dfccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1700e00640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125dfc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1701800640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 0x565125dfc270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f16ec004af0 tx=0x7f16ec000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e108433a48af2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f16e7400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f16e7400640 1 -- 172.21.15.44:0/2022048043 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec04c070 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f16e7400640 1 -- 172.21.15.44:0/2022048043 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f16ec0357e0 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f409aea0 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f16e7400640 1 -- 172.21.15.44:0/2022048043 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f16ec043040 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1706fab740 10 client.27854 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.646+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x565125e6ee00 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f16e7400640 1 -- 172.21.15.44:0/2022048043 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16f409aea0 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f16e7400640 1 -- 172.21.15.44:0/2022048043 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f16ec004010 con 0x565125def1a0 2024-04-20T11:31:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1706fab740 10 client.27854 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1706fab740 10 client.27854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1706fab740 20 client.27854 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1706fab740 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e6f5a0 0x565125e719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1706fab740 4 client.27854 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565125d5c9e0 con 0x565125e6f5a0 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.647+0000 7f1700e00640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e6f5a0 0x565125e719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1700e00640 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e6f5a0 0x565125e719c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38345c3b0a88e0c server_cookie=4272e24e7f04e0f3 in_seq=0 out_seq=0 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f16e7400640 10 client.27854 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f16e7400640 1 -- 172.21.15.44:0/2022048043 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565125d5c9e0 con 0x565125e6f5a0 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f16e7400640 10 client.27854 handle_command_reply: tid=0 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1706fab740 1 client.27854 shutdown 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 msgr2=0x565125dfc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1706fab740 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 0x565125dfc270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f16ec004af0 tx=0x7f16ec000f40 comp rx=0 tx=0).stop 2024-04-20T11:31:15.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 shutdown_connections 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1706fab740 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x565125e6f5a0 0x565125e719c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.648+0000 7f1706fab740 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565125df86a0 0x565125dfccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1706fab740 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565125defac0 0x565125dfc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f16e7400640 0 client.27854 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1706fab740 1 --2- 172.21.15.44:0/2022048043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565125def1a0 0x565125dfc270 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 >> 172.21.15.44:0/2022048043 conn(0x565125de98d0 msgr2=0x565125df46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 shutdown_connections 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1706fab740 1 -- 172.21.15.44:0/2022048043 wait complete. 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1706fab740 20 client.27854 trim_cache size 0 max 16384 2024-04-20T11:31:15.656 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: "5cd7d1ea-c56e-47ff-a5ba-6df547f72974": { 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "5cd7d1ea-c56e-47ff-a5ba-6df547f72974", 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:31:15.657 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:31:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16e00a3d20 msgr2=0x7f16e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1704200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16e00a3d20 0x7f16e00a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x565125deaa60 tx=0x7f16e8009680 comp rx=0 tx=0).stop 2024-04-20T11:31:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 msgr2=0x7f16fc19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.649+0000 7f1704200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 0x7f16fc19ae40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e970 tx=0x7f16f400ed70 comp rx=0 tx=0).stop 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 shutdown_connections 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f16e00a3d20 0x7f16e00a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16fc19a990 0x7f16fc19ae40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16fc109480 0x7f16fc1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 --2- 172.21.15.44:0/1141443870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc1043b0 0x7f16fc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 >> 172.21.15.44:0/1141443870 conn(0x7f16fc0ffbd0 msgr2=0x7f16fc101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 shutdown_connections 2024-04-20T11:31:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:15.650+0000 7f1704200640 1 -- 172.21.15.44:0/1141443870 wait complete. 2024-04-20T11:31:15.669 INFO:tasks.cephfs.filesystem:scrub status for tag:5cd7d1ea-c56e-47ff-a5ba-6df547f72974 - {'path': '/', 'tag': '5cd7d1ea-c56e-47ff-a5ba-6df547f72974', 'options': 'recursive,force'} 2024-04-20T11:31:15.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:15 smithi078 ceph-mon[21464]: pgmap v268: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:31:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:15 smithi044 ceph-mon[16855]: pgmap v268: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:31:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:15 smithi165 ceph-mon[21975]: pgmap v268: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:31:16.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:16 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:31:16.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:16 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:16.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:16 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:31:16.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:16 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:16.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:16 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:31:16.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:16 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:16.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:16 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:31:16.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:16 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:16 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:31:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:16 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:16 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:31:16.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:16 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:17.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:17 smithi078 ceph-mon[21464]: pgmap v269: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 115 B/s wr, 5 op/s 2024-04-20T11:31:17.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:17 smithi044 ceph-mon[16855]: pgmap v269: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 115 B/s wr, 5 op/s 2024-04-20T11:31:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:17 smithi165 ceph-mon[21975]: pgmap v269: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 115 B/s wr, 5 op/s 2024-04-20T11:31:18.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:18 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:31:18.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:18 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:31:18.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:18 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:31:19.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:19 smithi078 ceph-mon[21464]: pgmap v270: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:19.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:19 smithi044 ceph-mon[16855]: pgmap v270: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:19 smithi165 ceph-mon[21975]: pgmap v270: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:21.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:21 smithi078 ceph-mon[21464]: pgmap v271: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:21 smithi044 ceph-mon[16855]: pgmap v271: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:21 smithi165 ceph-mon[21975]: pgmap v271: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:31:23.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:23 smithi078 ceph-mon[21464]: pgmap v272: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:31:23.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:23 smithi044 ceph-mon[16855]: pgmap v272: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:31:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:23 smithi165 ceph-mon[21975]: pgmap v272: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:31:25.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:25 smithi078 ceph-mon[21464]: pgmap v273: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:31:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:25 smithi044 ceph-mon[16855]: pgmap v273: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:31:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:25 smithi165 ceph-mon[21975]: pgmap v273: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:31:27.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:27 smithi078 ceph-mon[21464]: pgmap v274: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:31:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:27 smithi044 ceph-mon[16855]: pgmap v274: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:31:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:27 smithi165 ceph-mon[21975]: pgmap v274: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:31:29.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:29 smithi078 ceph-mon[21464]: pgmap v275: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:31:29.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:29 smithi044 ceph-mon[16855]: pgmap v275: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:31:29.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:29 smithi165 ceph-mon[21975]: pgmap v275: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:31:31.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:31 smithi078 ceph-mon[21464]: pgmap v276: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:31.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:31 smithi044 ceph-mon[16855]: pgmap v276: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:31.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:31 smithi165 ceph-mon[21975]: pgmap v276: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:33.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:33 smithi078 ceph-mon[21464]: pgmap v277: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:33.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:33 smithi044 ceph-mon[16855]: pgmap v277: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:33.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:33 smithi165 ceph-mon[21975]: pgmap v277: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:35.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:35 smithi078 ceph-mon[21464]: pgmap v278: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:35.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:35 smithi044 ceph-mon[16855]: pgmap v278: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:35.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:35 smithi165 ceph-mon[21975]: pgmap v278: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:37 smithi078 ceph-mon[21464]: pgmap v279: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:37.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:37 smithi044 ceph-mon[16855]: pgmap v279: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:37.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:37 smithi165 ceph-mon[21975]: pgmap v279: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:39.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:39 smithi078 ceph-mon[21464]: pgmap v280: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:39 smithi044 ceph-mon[16855]: pgmap v280: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:39 smithi165 ceph-mon[21975]: pgmap v280: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:41 smithi078 ceph-mon[21464]: pgmap v281: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:41.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:41 smithi044 ceph-mon[16855]: pgmap v281: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:41 smithi165 ceph-mon[21975]: pgmap v281: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:43.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:43 smithi078 ceph-mon[21464]: pgmap v282: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:43.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:43 smithi044 ceph-mon[16855]: pgmap v282: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:43.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:43 smithi165 ceph-mon[21975]: pgmap v282: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:45.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:31:45.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 -- 172.21.15.44:0/259794975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828102b50 msgr2=0x7fe828106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 --2- 172.21.15.44:0/259794975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828102b50 0x7fe828106b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe814009900 tx=0x7fe81402bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 -- 172.21.15.44:0/259794975 shutdown_connections 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 --2- 172.21.15.44:0/259794975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe828107270 0x7fe82810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 --2- 172.21.15.44:0/259794975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828102b50 0x7fe828106b90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 --2- 172.21.15.44:0/259794975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8281021a0 0x7fe828102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 -- 172.21.15.44:0/259794975 >> 172.21.15.44:0/259794975 conn(0x7fe8280793b0 msgr2=0x7fe8280797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 -- 172.21.15.44:0/259794975 shutdown_connections 2024-04-20T11:31:45.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.896+0000 7fe82fe00640 1 -- 172.21.15.44:0/259794975 wait complete. 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.897+0000 7fe82fe00640 1 Processor -- start 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.897+0000 7fe82fe00640 1 -- start start 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 0x7fe828074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828107270 0x7fe8280747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe828074ce0 0x7fe8281a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe828112440 con 0x7fe828107270 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281122c0 con 0x7fe828074ce0 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281125c0 con 0x7fe8281021a0 2024-04-20T11:31:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 0x7fe828074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 0x7fe828074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:32940/0 (socket says 172.21.15.44:32940) 2024-04-20T11:31:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82d400640 1 -- 172.21.15.44:0/24693906 learned_addr learned my addr 172.21.15.44:0/24693906 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82ca00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828107270 0x7fe8280747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.898+0000 7fe82de00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe828074ce0 0x7fe8281a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe82d400640 1 -- 172.21.15.44:0/24693906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe828074ce0 msgr2=0x7fe8281a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe82d400640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe828074ce0 0x7fe8281a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe82d400640 1 -- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828107270 msgr2=0x7fe8280747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe82d400640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828107270 0x7fe8280747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe82d400640 1 -- 172.21.15.44:0/24693906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8281a71f0 con 0x7fe8281021a0 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe82d400640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 0x7fe828074260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe81800ebc0 tx=0x7fe81800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.899+0000 7fe826000640 1 -- 172.21.15.44:0/24693906 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818010630 con 0x7fe8281021a0 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.900+0000 7fe826000640 1 -- 172.21.15.44:0/24693906 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe818010cd0 con 0x7fe8281021a0 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.900+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8281986d0 con 0x7fe8281021a0 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.900+0000 7fe826000640 1 -- 172.21.15.44:0/24693906 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818018cf0 con 0x7fe8281021a0 2024-04-20T11:31:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.900+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe828198c40 con 0x7fe8281021a0 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe832a01740 1 Processor -- start 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe826000640 1 -- 172.21.15.44:0/24693906 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe8180107d0 con 0x7fe8281021a0 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe832a01740 1 -- start start 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe832a01740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d21a0 0x557e713d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe832a01740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 0x557e713db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe832a01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.903+0000 7fe832a01740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557e71338150 con 0x557e713d2ac0 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe832a01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557e712de370 con 0x557e713db6a0 2024-04-20T11:31:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe832a01740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557e71334110 con 0x557e713d21a0 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 0x557e713db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d21a0 0x557e713d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 0x557e713db160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50472/0 (socket says 172.21.15.44:50472) 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82d400640 1 -- 172.21.15.44:0/3808213186 learned_addr learned my addr 172.21.15.44:0/3808213186 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82ca00640 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82de00640 1 -- 172.21.15.44:0/3808213186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 msgr2=0x557e713dda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82de00640 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82de00640 1 -- 172.21.15.44:0/3808213186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 msgr2=0x557e713db160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82de00640 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 0x557e713db160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82de00640 1 -- 172.21.15.44:0/3808213186 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e711147d0 con 0x557e713d21a0 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.904+0000 7fe82d400640 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 0x557e713db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe82de00640 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d21a0 0x557e713d2580 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe820004530 tx=0x7fe82000ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98fb8832ef96c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe826000640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe80c0a3dd0 0x7fe80c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe81e600640 1 -- 172.21.15.44:0/3808213186 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820003ad0 con 0x557e713d21a0 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe81e600640 1 -- 172.21.15.44:0/3808213186 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe820015070 con 0x557e713d21a0 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe81e600640 1 -- 172.21.15.44:0/3808213186 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820011640 con 0x557e713d21a0 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe826000640 1 -- 172.21.15.44:0/24693906 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fe8180d0df0 con 0x7fe8281021a0 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.905+0000 7fe82d400640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe80c0a3dd0 0x7fe80c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 -- 172.21.15.44:0/3808213186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d21a0 msgr2=0x557e713d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d21a0 0x557e713d2580 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe820004530 tx=0x7fe82000ba50 comp rx=0 tx=0).stop 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 -- 172.21.15.44:0/3808213186 shutdown_connections 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d2ac0 0x557e713db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 --2- 172.21.15.44:0/3808213186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d21a0 0x557e713d2580 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 -- 172.21.15.44:0/3808213186 >> 172.21.15.44:0/3808213186 conn(0x557e713cc8d0 msgr2=0x557e713cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 -- 172.21.15.44:0/3808213186 shutdown_connections 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe82d400640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe80c0a3dd0 0x7fe80c0a6290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe81809cb20 tx=0x7fe8180d48a0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.906+0000 7fe832a01740 1 -- 172.21.15.44:0/3808213186 wait complete. 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 -- start start 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 Processor -- start 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557e713de490 con 0x557e713d21a0 2024-04-20T11:31:45.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557e712f1740 con 0x557e713d2ac0 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557e713de610 con 0x557e713db6a0 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.907+0000 7fe832a01740 10 client.0 _ll_register_callbacks cb 0x557e712ecb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713d8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50474/0 (socket says 172.21.15.44:50474) 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82de00640 1 -- 172.21.15.44:0/2899234841 learned_addr learned my addr 172.21.15.44:0/2899234841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82de00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 -- 172.21.15.44:0/2899234841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 msgr2=0x557e713d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 -- 172.21.15.44:0/2899234841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 msgr2=0x557e713d8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713d8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 -- 172.21.15.44:0/2899234841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7104a3f0 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82de00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe82ca00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713d8cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe814036040 tx=0x7fe81402dda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2526a31682a05169 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.908+0000 7fe812a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe812a00640 1 -- 172.21.15.44:0/2899234841 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814046070 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe812a00640 1 -- 172.21.15.44:0/2899234841 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fe814004310 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe820015070 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe812a00640 1 -- 172.21.15.44:0/2899234841 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe81404f050 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe832a01740 10 client.27866 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557e7100a0f0 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.909+0000 7fe812a00640 1 -- 172.21.15.44:0/2899234841 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe820015070 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe812a00640 1 -- 172.21.15.44:0/2899234841 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe814049020 con 0x557e713d2ac0 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe832a01740 10 client.27866 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:45.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe832a01740 10 client.27866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe832a01740 20 client.27866 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe832a01740 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e713ebab0 0x557e713edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe832a01740 4 client.27866 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e71345b70 con 0x557e713ebab0 2024-04-20T11:31:45.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.910+0000 7fe82de00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e713ebab0 0x557e713edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.911+0000 7fe82de00640 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e713ebab0 0x557e713edee0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e939ed1a0f1698e2 server_cookie=6d30eff09aef2544 in_seq=0 out_seq=0 2024-04-20T11:31:45.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.911+0000 7fe812a00640 10 client.27866 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe812a00640 1 -- 172.21.15.44:0/2899234841 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e71345b70 con 0x557e713ebab0 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe812a00640 10 client.27866 handle_command_reply: tid=0 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 client.27866 shutdown 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 msgr2=0x557e713d8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713d8cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe814036040 tx=0x7fe81402dda0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 shutdown_connections 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e713ebab0 0x557e713edee0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713d8cb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 --2- 172.21.15.44:0/2899234841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713d8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 >> 172.21.15.44:0/2899234841 conn(0x557e713cc8d0 msgr2=0x557e713ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe812a00640 0 client.27866 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 shutdown_connections 2024-04-20T11:31:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 1 -- 172.21.15.44:0/2899234841 wait complete. 2024-04-20T11:31:45.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.912+0000 7fe832a01740 20 client.27866 trim_cache size 0 max 16384 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 Processor -- start 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 -- start start 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281b4380 con 0x557e713d21a0 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281b5ec0 con 0x557e713d2ac0 2024-04-20T11:31:45.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe832a01740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281b4a50 con 0x557e713db6a0 2024-04-20T11:31:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.932+0000 7fe82de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713daea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50480/0 (socket says 172.21.15.44:50480) 2024-04-20T11:31:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82de00640 1 -- 172.21.15.44:0/1758340492 learned_addr learned my addr 172.21.15.44:0/1758340492 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82de00640 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 -- 172.21.15.44:0/1758340492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 msgr2=0x557e713d8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 -- 172.21.15.44:0/1758340492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 msgr2=0x557e713daea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713daea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 -- 172.21.15.44:0/1758340492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e713cc400 con 0x557e713d2ac0 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.933+0000 7fe82ca00640 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713e7350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe814009850 tx=0x7fe814005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c0de57a387a982 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe81dc00640 1 -- 172.21.15.44:0/1758340492 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814046070 con 0x557e713d2ac0 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe81dc00640 1 -- 172.21.15.44:0/1758340492 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe814004310 con 0x557e713d2ac0 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe81dc00640 1 -- 172.21.15.44:0/1758340492 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814003bd0 con 0x557e713d2ac0 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 -- 172.21.15.44:0/1758340492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 msgr2=0x557e713e7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713e7350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe814009850 tx=0x7fe814005fd0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 -- 172.21.15.44:0/1758340492 shutdown_connections 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713db6a0 0x557e713d8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713d2ac0 0x557e713e7350 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 --2- 172.21.15.44:0/1758340492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713daea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 -- 172.21.15.44:0/1758340492 >> 172.21.15.44:0/1758340492 conn(0x557e713cc8d0 msgr2=0x557e713d8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 -- 172.21.15.44:0/1758340492 shutdown_connections 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.934+0000 7fe832a01740 1 -- 172.21.15.44:0/1758340492 wait complete. 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 -- start start 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 Processor -- start 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d2ac0 0x557e713df820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281b57f0 con 0x557e713d21a0 2024-04-20T11:31:45.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281b5120 con 0x557e713db6a0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281b2370 con 0x557e713d2ac0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.935+0000 7fe832a01740 10 client.0 _ll_register_callbacks cb 0x7fe814009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713df2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50488/0 (socket says 172.21.15.44:50488) 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 -- 172.21.15.44:0/3167586920 learned_addr learned my addr 172.21.15.44:0/3167586920 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82ca00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d2ac0 0x557e713df820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 -- 172.21.15.44:0/3167586920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d2ac0 msgr2=0x557e713df820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d2ac0 0x557e713df820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 -- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 msgr2=0x557e713dfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.936+0000 7fe82de00640 1 -- 172.21.15.44:0/3167586920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e71451070 con 0x557e713d21a0 2024-04-20T11:31:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe82de00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713df2e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe82001acb0 tx=0x7fe820005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acb5cfd0f5f17bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:45.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe812a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:45.940 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:45 smithi078 ceph-mon[21464]: pgmap v283: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:45.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe812a00640 1 -- 172.21.15.44:0/3167586920 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820032020 con 0x557e713d21a0 2024-04-20T11:31:45.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe812a00640 1 -- 172.21.15.44:0/3167586920 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fe82002c580 con 0x557e713d21a0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe814004310 con 0x557e713d21a0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe812a00640 1 -- 172.21.15.44:0/3167586920 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe82003d050 con 0x557e713d21a0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe832a01740 10 client.17754 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557e71452380 con 0x557e713d21a0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.937+0000 7fe812a00640 1 -- 172.21.15.44:0/3167586920 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe814004310 con 0x557e713d21a0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe812a00640 1 -- 172.21.15.44:0/3167586920 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe820011560 con 0x557e713d21a0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe832a01740 10 client.17754 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe832a01740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe832a01740 20 client.17754 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe832a01740 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e714529c0 0x557e71454de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe832a01740 4 client.17754 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e7133f9e0 con 0x557e714529c0 2024-04-20T11:31:45.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.938+0000 7fe82ca00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e714529c0 0x557e71454de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe82ca00640 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e714529c0 0x557e71454de0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45bba3d42d142568 server_cookie=a20ff721041605f7 in_seq=0 out_seq=0 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe812a00640 10 client.17754 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe812a00640 1 -- 172.21.15.44:0/3167586920 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e7133f9e0 con 0x557e714529c0 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe812a00640 10 client.17754 handle_command_reply: tid=0 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 client.17754 shutdown 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 msgr2=0x557e713df2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713df2e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe82001acb0 tx=0x7fe820005fb0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 shutdown_connections 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x557e714529c0 0x557e71454de0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e713db6a0 0x557e713dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e713d2ac0 0x557e713df820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 --2- 172.21.15.44:0/3167586920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557e713d21a0 0x557e713df2e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe812a00640 0 client.17754 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 >> 172.21.15.44:0/3167586920 conn(0x557e713cc8d0 msgr2=0x557e713cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 shutdown_connections 2024-04-20T11:31:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 1 -- 172.21.15.44:0/3167586920 wait complete. 2024-04-20T11:31:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.939+0000 7fe832a01740 20 client.17754 trim_cache size 0 max 16384 2024-04-20T11:31:45.946 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:31:45.946 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:31:45.946 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:31:45.946 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe80c0a3dd0 msgr2=0x7fe80c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe80c0a3dd0 0x7fe80c0a6290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe81809cb20 tx=0x7fe8180d48a0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 msgr2=0x7fe828074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 0x7fe828074260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe81800ebc0 tx=0x7fe81800c6a0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 shutdown_connections 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe828074ce0 0x7fe8281a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe80c0a3dd0 0x7fe80c0a6290 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe828107270 0x7fe8280747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 --2- 172.21.15.44:0/24693906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8281021a0 0x7fe828074260 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:45.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 >> 172.21.15.44:0/24693906 conn(0x7fe8280793b0 msgr2=0x7fe8280fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:45.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 shutdown_connections 2024-04-20T11:31:45.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:45.940+0000 7fe82fe00640 1 -- 172.21.15.44:0/24693906 wait complete. 2024-04-20T11:31:45.948 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:45 smithi044 ceph-mon[16855]: pgmap v283: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:45.960 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:31:45.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:31:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:45 smithi165 ceph-mon[21975]: pgmap v283: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 -- 172.21.15.44:0/564217477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b0102b50 msgr2=0x7fc2b0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/564217477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b0102b50 0x7fc2b0106b90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009900 tx=0x7fc29c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 -- 172.21.15.44:0/564217477 shutdown_connections 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/564217477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0107270 0x7fc2b010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/564217477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b0102b50 0x7fc2b0106b90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/564217477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b01021a0 0x7fc2b0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 -- 172.21.15.44:0/564217477 >> 172.21.15.44:0/564217477 conn(0x7fc2b0079320 msgr2=0x7fc2b0079730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 -- 172.21.15.44:0/564217477 shutdown_connections 2024-04-20T11:31:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.177+0000 7fc2b4e00640 1 -- 172.21.15.44:0/564217477 wait complete. 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.178+0000 7fc2b4e00640 1 Processor -- start 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 -- start start 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 0x7fc2b01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b0107270 0x7fc2b01a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b019a990 0x7fc2b019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01123f0 con 0x7fc2b0107270 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0112270 con 0x7fc2b019a990 2024-04-20T11:31:46.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2b4e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0112570 con 0x7fc2b01021a0 2024-04-20T11:31:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2ade00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 0x7fc2b01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.179+0000 7fc2ad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b0107270 0x7fc2b01a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 0x7fc2b01a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:32994/0 (socket says 172.21.15.44:32994) 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 -- 172.21.15.44:0/4167407905 learned_addr learned my addr 172.21.15.44:0/4167407905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ae800640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b019a990 0x7fc2b019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 -- 172.21.15.44:0/4167407905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b019a990 msgr2=0x7fc2b019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b019a990 0x7fc2b019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 -- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b0107270 msgr2=0x7fc2b01a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b0107270 0x7fc2b01a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 -- 172.21.15.44:0/4167407905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b019b4b0 con 0x7fc2b01021a0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.180+0000 7fc2ade00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 0x7fc2b01a07e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000ebc0 tx=0x7fc2a000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.181+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0010630 con 0x7fc2b01021a0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.181+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc2a0010cd0 con 0x7fc2b01021a0 2024-04-20T11:31:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.181+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b019b7a0 con 0x7fc2b01021a0 2024-04-20T11:31:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.181+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0018cf0 con 0x7fc2b01021a0 2024-04-20T11:31:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.181+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b01a79f0 con 0x7fc2b01021a0 2024-04-20T11:31:46.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.182+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b00691d0 con 0x7fc2b01021a0 2024-04-20T11:31:46.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.184+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc2a00107d0 con 0x7fc2b01021a0 2024-04-20T11:31:46.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.186+0000 7fc2a6a00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc2940a3d20 0x7fc2940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.186+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fc2a00d0f80 con 0x7fc2b01021a0 2024-04-20T11:31:46.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.186+0000 7fc2ad400640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc2940a3d20 0x7fc2940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.187+0000 7fc2ad400640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc2940a3d20 0x7fc2940a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc29c000c00 tx=0x7fc29c005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.188+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc2a009cfb0 con 0x7fc2b01021a0 2024-04-20T11:31:46.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.438+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2b0102c60 con 0x7fc2b01021a0 2024-04-20T11:31:46.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.440+0000 7fc2a6a00640 1 -- 172.21.15.44:0/4167407905 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fc2a00a12b0 con 0x7fc2b01021a0 2024-04-20T11:31:46.440 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:31:46.440 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:31:46.440 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:31:46.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc2940a3d20 msgr2=0x7fc2940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc2940a3d20 0x7fc2940a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc29c000c00 tx=0x7fc29c005fd0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 msgr2=0x7fc2b01a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 0x7fc2b01a07e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000ebc0 tx=0x7fc2a000c6a0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 shutdown_connections 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b019a990 0x7fc2b019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc2940a3d20 0x7fc2940a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2b0107270 0x7fc2b01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 --2- 172.21.15.44:0/4167407905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b01021a0 0x7fc2b01a07e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 >> 172.21.15.44:0/4167407905 conn(0x7fc2b0079320 msgr2=0x7fc2b010d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 shutdown_connections 2024-04-20T11:31:46.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.443+0000 7fc2b4e00640 1 -- 172.21.15.44:0/4167407905 wait complete. 2024-04-20T11:31:46.464 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:31:46.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.678+0000 7f65bb200640 1 -- 172.21.15.44:0/3557573829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc112e20 msgr2=0x7f65bc115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.678+0000 7f65bb200640 1 --2- 172.21.15.44:0/3557573829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc112e20 0x7f65bc115210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009900 tx=0x7f65b402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.678+0000 7f65bb200640 1 -- 172.21.15.44:0/3557573829 shutdown_connections 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.679+0000 7f65bb200640 1 --2- 172.21.15.44:0/3557573829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc112e20 0x7f65bc115210 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.679+0000 7f65bb200640 1 --2- 172.21.15.44:0/3557573829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074c20 0x7f65bc1128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.679+0000 7f65bb200640 1 --2- 172.21.15.44:0/3557573829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc074300 0x7f65bc0746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.679+0000 7f65bb200640 1 -- 172.21.15.44:0/3557573829 >> 172.21.15.44:0/3557573829 conn(0x7f65bc0fd900 msgr2=0x7f65bc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.679+0000 7f65bb200640 1 -- 172.21.15.44:0/3557573829 shutdown_connections 2024-04-20T11:31:46.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.679+0000 7f65bb200640 1 -- 172.21.15.44:0/3557573829 wait complete. 2024-04-20T11:31:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 Processor -- start 2024-04-20T11:31:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 -- start start 2024-04-20T11:31:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074300 0x7f65bc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 0x7f65bc19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc112e20 0x7f65bc1a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc118940 con 0x7f65bc074c20 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1187c0 con 0x7f65bc074300 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65bb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc118ac0 con 0x7f65bc112e20 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.680+0000 7f65b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 0x7f65bc19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc112e20 0x7f65bc1a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074300 0x7f65bc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 0x7f65bc19f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50522/0 (socket says 172.21.15.44:50522) 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc112e20 0x7f65bc1a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33012/0 (socket says 172.21.15.44:33012) 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b1400640 1 -- 172.21.15.44:0/1681377482 learned_addr learned my addr 172.21.15.44:0/1681377482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 -- 172.21.15.44:0/1681377482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc112e20 msgr2=0x7f65bc1a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc112e20 0x7f65bc1a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 -- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 msgr2=0x7f65bc19f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 0x7f65bc19f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 -- 172.21.15.44:0/1681377482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc1a3da0 con 0x7f65bc074300 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b1400640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 0x7f65bc19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.681+0000 7f65b9e00640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074300 0x7f65bc19ee00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00ec80 tx=0x7f65ac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.682+0000 7f65b3400640 1 -- 172.21.15.44:0/1681377482 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac003b70 con 0x7f65bc074300 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.682+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc1a4090 con 0x7f65bc074300 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.682+0000 7f65b3400640 1 -- 172.21.15.44:0/1681377482 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65ac003d10 con 0x7f65bc074300 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.682+0000 7f65b3400640 1 -- 172.21.15.44:0/1681377482 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac010a90 con 0x7f65bc074300 2024-04-20T11:31:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.682+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1ab920 con 0x7f65bc074300 2024-04-20T11:31:46.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.685+0000 7f65b3400640 1 -- 172.21.15.44:0/1681377482 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f65ac003ec0 con 0x7f65bc074300 2024-04-20T11:31:46.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.685+0000 7f65c397f740 1 Processor -- start 2024-04-20T11:31:46.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.685+0000 7f65c397f740 1 -- start start 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65c397f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 0x55823cbef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65c397f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbefa90 0x55823cbf8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65c397f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbf8670 0x55823cbfaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65c397f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55823cb54050 con 0x55823cbefa90 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65c397f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55823cafa490 con 0x55823cbf8670 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65c397f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55823cb50010 con 0x55823cbef170 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbefa90 0x55823cbf8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 0x55823cbef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65b1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbf8670 0x55823cbfaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 0x55823cbef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33020/0 (socket says 172.21.15.44:33020) 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65ba800640 1 -- 172.21.15.44:0/1810702069 learned_addr learned my addr 172.21.15.44:0/1810702069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.686+0000 7f65ba800640 1 -- 172.21.15.44:0/1810702069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbf8670 msgr2=0x55823cbfaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ba800640 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbf8670 0x55823cbfaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ba800640 1 -- 172.21.15.44:0/1810702069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbefa90 msgr2=0x55823cbf8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ba800640 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbefa90 0x55823cbf8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ba800640 1 -- 172.21.15.44:0/1810702069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823cb02070 con 0x55823cbef170 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65b1400640 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbf8670 0x55823cbfaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65b3400640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f65a00a3d20 0x7f65a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ba800640 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 0x55823cbef550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005ec0 tx=0x7f65b4009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43d0694d404ec24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65b3400640 1 -- 172.21.15.44:0/1681377482 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f65ac0d0f30 con 0x7f65bc074300 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65b9e00640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f65a00a3d20 0x7f65a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ab000640 1 -- 172.21.15.44:0/1810702069 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b402c7a0 con 0x55823cbef170 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ab000640 1 -- 172.21.15.44:0/1810702069 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65b4004040 con 0x55823cbef170 2024-04-20T11:31:46.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.687+0000 7f65ab000640 1 -- 172.21.15.44:0/1810702069 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40055e0 con 0x55823cbef170 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 -- 172.21.15.44:0/1810702069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 msgr2=0x55823cbef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65b9e00640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f65a00a3d20 0x7f65a00a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55823cbeec40 tx=0x7f65ac0d49e0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 0x55823cbef550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005ec0 tx=0x7f65b4009560 comp rx=0 tx=0).stop 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 -- 172.21.15.44:0/1810702069 shutdown_connections 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbf8670 0x55823cbfaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbefa90 0x55823cbf8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 --2- 172.21.15.44:0/1810702069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbef170 0x55823cbef550 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 -- 172.21.15.44:0/1810702069 >> 172.21.15.44:0/1810702069 conn(0x55823cbe98a0 msgr2=0x55823cbe9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 -- 172.21.15.44:0/1810702069 shutdown_connections 2024-04-20T11:31:46.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.688+0000 7f65c397f740 1 -- 172.21.15.44:0/1810702069 wait complete. 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.689+0000 7f65c397f740 1 -- start start 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.689+0000 7f65c397f740 1 Processor -- start 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.689+0000 7f65c397f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbf56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.689+0000 7f65c397f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbf5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65c397f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbf6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65c397f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55823cbfb400 con 0x55823cbef170 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65c397f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55823cb0d860 con 0x55823cbefa90 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65c397f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55823cbfb580 con 0x55823cbf8670 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65c397f740 10 client.0 _ll_register_callbacks cb 0x55823cadbfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbf56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbf56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50544/0 (socket says 172.21.15.44:50544) 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65ba800640 1 -- 172.21.15.44:0/3725342204 learned_addr learned my addr 172.21.15.44:0/3725342204 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65b1400640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbf5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65ba800640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbf6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65b1400640 1 -- 172.21.15.44:0/3725342204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 msgr2=0x55823cbf6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65b1400640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbf6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65b1400640 1 -- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 msgr2=0x55823cbf56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65b1400640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbf56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65b1400640 1 -- 172.21.15.44:0/3725342204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823cb11950 con 0x55823cbefa90 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.690+0000 7f65ba800640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbf56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f65b1400640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbf5be0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009620 tx=0x7f65a400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9289c418db2ddc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f659f400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f659f400640 1 -- 172.21.15.44:0/3725342204 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40120d0 con 0x55823cbefa90 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f659f400640 1 -- 172.21.15.44:0/3725342204 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f65a4004030 con 0x55823cbefa90 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4004040 con 0x55823cbefa90 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f659f400640 1 -- 172.21.15.44:0/3725342204 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f65a4012270 con 0x55823cbefa90 2024-04-20T11:31:46.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f65c397f740 10 client.27890 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.691+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55823cadf820 con 0x55823cbefa90 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f659f400640 1 -- 172.21.15.44:0/3725342204 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65b4004040 con 0x55823cbefa90 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f659f400640 1 -- 172.21.15.44:0/3725342204 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f65a401c4f0 con 0x55823cbefa90 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65c397f740 10 client.27890 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65c397f740 10 client.27890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65c397f740 20 client.27890 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65c397f740 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc08690 0x55823cc0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65c397f740 4 client.27890 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55823cb61a70 con 0x55823cc08690 2024-04-20T11:31:46.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.692+0000 7f65ba800640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc08690 0x55823cc0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.693+0000 7f65ba800640 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc08690 0x55823cc0aa80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d25871e10fda764a server_cookie=bb4ecc4ff756cc72 in_seq=0 out_seq=0 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.693+0000 7f659f400640 10 client.27890 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f659f400640 1 -- 172.21.15.44:0/3725342204 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55823cb61a70 con 0x55823cc08690 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f659f400640 10 client.27890 handle_command_reply: tid=0 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 client.27890 shutdown 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 msgr2=0x55823cbf5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbf5be0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009620 tx=0x7f65a400f790 comp rx=0 tx=0).stop 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 shutdown_connections 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbf6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc08690 0x55823cc0aa80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbf5be0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 --2- 172.21.15.44:0/3725342204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbf56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 >> 172.21.15.44:0/3725342204 conn(0x55823cbe98a0 msgr2=0x55823cbeb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f659f400640 0 client.27890 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 shutdown_connections 2024-04-20T11:31:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 1 -- 172.21.15.44:0/3725342204 wait complete. 2024-04-20T11:31:46.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.694+0000 7f65c397f740 20 client.27890 trim_cache size 0 max 16384 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.709+0000 7f65c397f740 1 Processor -- start 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.709+0000 7f65c397f740 1 -- start start 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65c397f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 0x55823cbf7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65c397f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbefa90 0x55823cc042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65c397f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbf8670 0x55823cc10370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65c397f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b4030 con 0x55823cbf8670 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65c397f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b5b70 con 0x55823cbef170 2024-04-20T11:31:46.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65c397f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b4700 con 0x55823cbefa90 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbf8670 0x55823cc10370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65ba800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 0x55823cbf7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65ba800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 0x55823cbf7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48034/0 (socket says 172.21.15.44:48034) 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65ba800640 1 -- 172.21.15.44:0/586072633 learned_addr learned my addr 172.21.15.44:0/586072633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.710+0000 7f65b1400640 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbefa90 0x55823cc042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65ba800640 1 -- 172.21.15.44:0/586072633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbefa90 msgr2=0x55823cc042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65ba800640 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbefa90 0x55823cc042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65ba800640 1 -- 172.21.15.44:0/586072633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbf8670 msgr2=0x55823cc10370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65ba800640 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbf8670 0x55823cc10370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65ba800640 1 -- 172.21.15.44:0/586072633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823cc0adf0 con 0x55823cbef170 2024-04-20T11:31:46.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65ba800640 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 0x55823cbf7dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a44174aaf9adbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65aa600640 1 -- 172.21.15.44:0/586072633 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b402ca60 con 0x55823cbef170 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65aa600640 1 -- 172.21.15.44:0/586072633 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65b402cc00 con 0x55823cbef170 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.711+0000 7f65aa600640 1 -- 172.21.15.44:0/586072633 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4034730 con 0x55823cbef170 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 -- 172.21.15.44:0/586072633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 msgr2=0x55823cbf7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 0x55823cbf7dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b4005a20 comp rx=0 tx=0).stop 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 -- 172.21.15.44:0/586072633 shutdown_connections 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbf8670 0x55823cc10370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbefa90 0x55823cc042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 --2- 172.21.15.44:0/586072633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbef170 0x55823cbf7dd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 -- 172.21.15.44:0/586072633 >> 172.21.15.44:0/586072633 conn(0x55823cbe98a0 msgr2=0x55823cbe9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 -- 172.21.15.44:0/586072633 shutdown_connections 2024-04-20T11:31:46.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.712+0000 7f65c397f740 1 -- 172.21.15.44:0/586072633 wait complete. 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 -- start start 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 Processor -- start 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbfc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbfc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbfcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b54a0 con 0x55823cbef170 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b4dd0 con 0x55823cbefa90 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1b2020 con 0x55823cbf8670 2024-04-20T11:31:46.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.713+0000 7f65c397f740 10 client.0 _ll_register_callbacks cb 0x7f65a4012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbfc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbfc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48050/0 (socket says 172.21.15.44:48050) 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 -- 172.21.15.44:0/2478248808 learned_addr learned my addr 172.21.15.44:0/2478248808 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbfc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65ba800640 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbfcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 -- 172.21.15.44:0/2478248808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 msgr2=0x55823cbfcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbfcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 -- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 msgr2=0x55823cbfc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbfc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 -- 172.21.15.44:0/2478248808 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823c8663f0 con 0x55823cbefa90 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f65b1400640 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbfc820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55823cbf4f00 tx=0x7f65a401c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9562f859d3b43ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.714+0000 7f659f400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f659f400640 1 -- 172.21.15.44:0/2478248808 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4029040 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f659f400640 1 -- 172.21.15.44:0/2478248808 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f65a4021020 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b402cc00 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f659f400640 1 -- 172.21.15.44:0/2478248808 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f65a4035050 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f65c397f740 10 client.27896 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55823c9307d0 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f659f400640 1 -- 172.21.15.44:0/2478248808 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65b402cc00 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.715+0000 7f659f400640 1 -- 172.21.15.44:0/2478248808 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f65a4003ad0 con 0x55823cbefa90 2024-04-20T11:31:46.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65c397f740 10 client.27896 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65c397f740 10 client.27896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65c397f740 20 client.27896 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65c397f740 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc6f3d0 0x55823cc717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65c397f740 4 client.27896 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55823cb5b8e0 con 0x55823cc6f3d0 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.716+0000 7f65ba800640 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc6f3d0 0x55823cc717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65ba800640 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc6f3d0 0x55823cc717f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2992ef3fb4e754cc server_cookie=54a78408b0fe2309 in_seq=0 out_seq=0 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f659f400640 10 client.27896 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f659f400640 1 -- 172.21.15.44:0/2478248808 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55823cb5b8e0 con 0x55823cc6f3d0 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f659f400640 10 client.27896 handle_command_reply: tid=0 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 client.27896 shutdown 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 msgr2=0x55823cbfc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbfc820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55823cbf4f00 tx=0x7f65a401c710 comp rx=0 tx=0).stop 2024-04-20T11:31:46.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 shutdown_connections 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55823cbf8670 0x55823cbfcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x55823cc6f3d0 0x55823cc717f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55823cbefa90 0x55823cbfc820 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 --2- 172.21.15.44:0/2478248808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55823cbef170 0x55823cbfc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 >> 172.21.15.44:0/2478248808 conn(0x55823cbe98a0 msgr2=0x55823cbe80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f659f400640 0 client.27896 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 shutdown_connections 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 1 -- 172.21.15.44:0/2478248808 wait complete. 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.717+0000 7f65c397f740 20 client.27896 trim_cache size 0 max 16384 2024-04-20T11:31:46.721 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:46.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f65a00a3d20 msgr2=0x7f65a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f65a00a3d20 0x7f65a00a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55823cbeec40 tx=0x7f65ac0d49e0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074300 msgr2=0x7f65bc19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074300 0x7f65bc19ee00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00ec80 tx=0x7f65ac00c6a0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 shutdown_connections 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65bc112e20 0x7f65bc1a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f65a00a3d20 0x7f65a00a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65bc074c20 0x7f65bc19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 --2- 172.21.15.44:0/1681377482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65bc074300 0x7f65bc19ee00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 >> 172.21.15.44:0/1681377482 conn(0x7f65bc0fd900 msgr2=0x7f65bc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 shutdown_connections 2024-04-20T11:31:46.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.718+0000 7f65bb200640 1 -- 172.21.15.44:0/1681377482 wait complete. 2024-04-20T11:31:46.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:46 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4167407905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:31:46.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:31:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:46 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4167407905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 -- 172.21.15.44:0/3840281690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7164109350 msgr2=0x7f7164110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 --2- 172.21.15.44:0/3840281690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7164109350 0x7f7164110f20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f715c009900 tx=0x7f715c02bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 -- 172.21.15.44:0/3840281690 shutdown_connections 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 --2- 172.21.15.44:0/3840281690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7164109350 0x7f7164110f20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 --2- 172.21.15.44:0/3840281690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7164104c30 0x7f7164108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 --2- 172.21.15.44:0/3840281690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7164104280 0x7f7164104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 -- 172.21.15.44:0/3840281690 >> 172.21.15.44:0/3840281690 conn(0x7f71640ffaa0 msgr2=0x7f7164101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 -- 172.21.15.44:0/3840281690 shutdown_connections 2024-04-20T11:31:46.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.942+0000 7f7169800640 1 -- 172.21.15.44:0/3840281690 wait complete. 2024-04-20T11:31:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.943+0000 7f7169800640 1 Processor -- start 2024-04-20T11:31:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.943+0000 7f7169800640 1 -- start start 2024-04-20T11:31:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7169800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7164104280 0x7f71641a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7169800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7164109350 0x7f71641a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7169800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 0x7f716419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7169800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7164114520 con 0x7f7164109350 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7169800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71641143a0 con 0x7f716419a7e0 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7169800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71641146a0 con 0x7f7164104280 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7168e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 0x7f716419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f715b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7164109350 0x7f71641a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7168e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 0x7f716419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48078/0 (socket says 172.21.15.44:48078) 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7168e00640 1 -- 172.21.15.44:0/2159073264 learned_addr learned my addr 172.21.15.44:0/2159073264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.944+0000 7f7163e00640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7164104280 0x7f71641a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7168e00640 1 -- 172.21.15.44:0/2159073264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7164104280 msgr2=0x7f71641a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7168e00640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7164104280 0x7f71641a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7168e00640 1 -- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7164109350 msgr2=0x7f71641a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7168e00640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7164109350 0x7f71641a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7168e00640 1 -- 172.21.15.44:0/2159073264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716419b370 con 0x7f716419a7e0 2024-04-20T11:31:46.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7168e00640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 0x7f716419ac90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f715c02c180 tx=0x7f715c02dc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7161600640 1 -- 172.21.15.44:0/2159073264 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c043020 con 0x7f716419a7e0 2024-04-20T11:31:46.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7161600640 1 -- 172.21.15.44:0/2159073264 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f715c004250 con 0x7f716419a7e0 2024-04-20T11:31:46.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7161600640 1 -- 172.21.15.44:0/2159073264 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c0039e0 con 0x7f716419a7e0 2024-04-20T11:31:46.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716419b600 con 0x7f716419a7e0 2024-04-20T11:31:46.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.945+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71641a7790 con 0x7f716419a7e0 2024-04-20T11:31:46.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 Processor -- start 2024-04-20T11:31:46.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f7161600640 1 -- 172.21.15.44:0/2159073264 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f715c046020 con 0x7f716419a7e0 2024-04-20T11:31:46.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 -- start start 2024-04-20T11:31:46.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5170 0x555809fb5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5a90 0x555809fbe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555809f1a050 con 0x555809fbe670 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555809ec0490 con 0x555809fb5a90 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.949+0000 7f716da5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555809f16010 con 0x555809fb5170 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f715b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5170 0x555809fb5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7163e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5a90 0x555809fbe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f715b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5170 0x555809fb5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33070/0 (socket says 172.21.15.44:33070) 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f715b400640 1 -- 172.21.15.44:0/4044538123 learned_addr learned my addr 172.21.15.44:0/4044538123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f715b400640 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7163e00640 1 -- 172.21.15.44:0/4044538123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5170 msgr2=0x555809fb5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7163e00640 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5170 0x555809fb5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7163e00640 1 -- 172.21.15.44:0/4044538123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 msgr2=0x555809fc0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7163e00640 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7163e00640 1 -- 172.21.15.44:0/4044538123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555809ec8070 con 0x555809fb5a90 2024-04-20T11:31:46.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.950+0000 7f7161600640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71480a3d20 0x7f71480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.951+0000 7f7163e00640 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5a90 0x555809fbe130 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7154009ee0 tx=0x7f715400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ad5d17fac29603 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.951+0000 7f7161600640 1 -- 172.21.15.44:0/2159073264 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f715c0f27f0 con 0x7f716419a7e0 2024-04-20T11:31:46.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.951+0000 7f715b400640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71480a3d20 0x7f71480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.951+0000 7f7159600640 1 -- 172.21.15.44:0/4044538123 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154011640 con 0x555809fb5a90 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.951+0000 7f7159600640 1 -- 172.21.15.44:0/4044538123 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7154011ce0 con 0x555809fb5a90 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.951+0000 7f7159600640 1 -- 172.21.15.44:0/4044538123 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154010da0 con 0x555809fb5a90 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 -- 172.21.15.44:0/4044538123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5a90 msgr2=0x555809fbe130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f715b400640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71480a3d20 0x7f71480a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x555809fb0a30 tx=0x7f714c005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5a90 0x555809fbe130 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7154009ee0 tx=0x7f715400f730 comp rx=0 tx=0).stop 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 -- 172.21.15.44:0/4044538123 shutdown_connections 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5a90 0x555809fbe130 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 --2- 172.21.15.44:0/4044538123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5170 0x555809fb5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 -- 172.21.15.44:0/4044538123 >> 172.21.15.44:0/4044538123 conn(0x555809faf8a0 msgr2=0x555809fafc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 -- 172.21.15.44:0/4044538123 shutdown_connections 2024-04-20T11:31:46.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 -- 172.21.15.44:0/4044538123 wait complete. 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.952+0000 7f716da5a740 1 -- start start 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 Processor -- start 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 0x555809fbb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fbbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fbe670 0x555809fbc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555809fc1460 con 0x555809fb5170 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555809ed3860 con 0x555809fbe670 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555809fc15e0 con 0x555809fb5a90 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f716da5a740 10 client.0 _ll_register_callbacks cb 0x555809ea1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.953+0000 7f7163e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 0x555809fbb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 0x555809fbb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50606/0 (socket says 172.21.15.44:50606) 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 -- 172.21.15.44:0/628555502 learned_addr learned my addr 172.21.15.44:0/628555502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f715b400640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fbbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7168e00640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fbe670 0x555809fbc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 -- 172.21.15.44:0/628555502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 msgr2=0x555809fbbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fbbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 -- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fbe670 msgr2=0x555809fbc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fbe670 0x555809fbc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 -- 172.21.15.44:0/628555502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555809ed7950 con 0x555809fb5170 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7163e00640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 0x555809fbb700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7154006010 tx=0x7f7154010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22518cada6b7d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.954+0000 7f7152000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f7152000640 1 -- 172.21.15.44:0/628555502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715402a060 con 0x555809fb5170 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f7152000640 1 -- 172.21.15.44:0/628555502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f7154010d30 con 0x555809fb5170 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7154011ce0 con 0x555809fb5170 2024-04-20T11:31:46.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f7152000640 1 -- 172.21.15.44:0/628555502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7154009ae0 con 0x555809fb5170 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f716da5a740 10 client.17790 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555809ea5820 con 0x555809fb5170 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.955+0000 7f7152000640 1 -- 172.21.15.44:0/628555502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7154011ce0 con 0x555809fb5170 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f7152000640 1 -- 172.21.15.44:0/628555502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f7154005040 con 0x555809fb5170 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f716da5a740 10 client.17790 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f716da5a740 10 client.17790 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f716da5a740 20 client.17790 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f716da5a740 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555809fce5b0 0x555809fd09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f716da5a740 4 client.17790 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555809f27a70 con 0x555809fce5b0 2024-04-20T11:31:46.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.956+0000 7f715b400640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555809fce5b0 0x555809fd09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.957+0000 7f715b400640 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555809fce5b0 0x555809fd09a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=549aef96bd64e408 server_cookie=564b216763d0fd88 in_seq=0 out_seq=0 2024-04-20T11:31:46.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.957+0000 7f7152000640 10 client.17790 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f7152000640 1 -- 172.21.15.44:0/628555502 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555809f27a70 con 0x555809fce5b0 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f7152000640 10 client.17790 handle_command_reply: tid=0 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 client.17790 shutdown 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 msgr2=0x555809fbb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 0x555809fbb700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7154006010 tx=0x7f7154010740 comp rx=0 tx=0).stop 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 shutdown_connections 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fbe670 0x555809fbc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fbbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x555809fce5b0 0x555809fd09a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 --2- 172.21.15.44:0/628555502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fb5170 0x555809fbb700 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f7152000640 0 client.17790 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 >> 172.21.15.44:0/628555502 conn(0x555809faf8a0 msgr2=0x555809fb15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 shutdown_connections 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 1 -- 172.21.15.44:0/628555502 wait complete. 2024-04-20T11:31:46.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.958+0000 7f716da5a740 20 client.17790 trim_cache size 0 max 16384 2024-04-20T11:31:46.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.973+0000 7f716da5a740 1 Processor -- start 2024-04-20T11:31:46.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 -- start start 2024-04-20T11:31:46.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fbde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fd6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71641afbe0 con 0x555809fbe670 2024-04-20T11:31:46.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71641b1720 con 0x555809fb5170 2024-04-20T11:31:46.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f716da5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71641b02b0 con 0x555809fb5a90 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f7163e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fbde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f7168e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fd6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f7163e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fbde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48108/0 (socket says 172.21.15.44:48108) 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f7163e00640 1 -- 172.21.15.44:0/3925909865 learned_addr learned my addr 172.21.15.44:0/3925909865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.974+0000 7f715b400640 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7163e00640 1 -- 172.21.15.44:0/3925909865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 msgr2=0x555809fca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7163e00640 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7163e00640 1 -- 172.21.15.44:0/3925909865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 msgr2=0x555809fd6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7163e00640 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fd6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7163e00640 1 -- 172.21.15.44:0/3925909865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555809c2c3f0 con 0x555809fb5170 2024-04-20T11:31:46.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7163e00640 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fbde30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f71540055b0 tx=0x7f71540021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127306a2b1a5670c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7158c00640 1 -- 172.21.15.44:0/3925909865 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715401f070 con 0x555809fb5170 2024-04-20T11:31:46.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7158c00640 1 -- 172.21.15.44:0/3925909865 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71540057b0 con 0x555809fb5170 2024-04-20T11:31:46.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.975+0000 7f7158c00640 1 -- 172.21.15.44:0/3925909865 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715402a050 con 0x555809fb5170 2024-04-20T11:31:46.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 -- 172.21.15.44:0/3925909865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 msgr2=0x555809fbde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fbde30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f71540055b0 tx=0x7f71540021d0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 -- 172.21.15.44:0/3925909865 shutdown_connections 2024-04-20T11:31:46.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fd6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 --2- 172.21.15.44:0/3925909865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fbde30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 -- 172.21.15.44:0/3925909865 >> 172.21.15.44:0/3925909865 conn(0x555809faf8a0 msgr2=0x555809fafc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 -- 172.21.15.44:0/3925909865 shutdown_connections 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.976+0000 7f716da5a740 1 -- 172.21.15.44:0/3925909865 wait complete. 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 -- start start 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 Processor -- start 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fc6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fc1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71641b1050 con 0x555809fbe670 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71641b0980 con 0x555809fb5170 2024-04-20T11:31:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71641adbd0 con 0x555809fb5a90 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.977+0000 7f716da5a740 10 client.0 _ll_register_callbacks cb 0x7f714c0099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fc1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fc1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33094/0 (socket says 172.21.15.44:33094) 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 -- 172.21.15.44:0/1088624965 learned_addr learned my addr 172.21.15.44:0/1088624965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f7168e00640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f7163e00640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fc6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 -- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 msgr2=0x555809fc6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fc6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 -- 172.21.15.44:0/1088624965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 msgr2=0x555809fc23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 -- 172.21.15.44:0/1088624965 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555809cf67d0 con 0x555809fb5a90 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f7168e00640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f715b400640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fc1eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f714c009fd0 tx=0x7f714c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643da26303eb48af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.978+0000 7f7152000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f7152000640 1 -- 172.21.15.44:0/1088624965 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c011d50 con 0x555809fb5a90 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71540057b0 con 0x555809fb5a90 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f7152000640 1 -- 172.21.15.44:0/1088624965 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f714c02e070 con 0x555809fb5a90 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f7152000640 1 -- 172.21.15.44:0/1088624965 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f714c03e050 con 0x555809fb5a90 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f716da5a740 10 client.27919 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55580a03cec0 con 0x555809fb5a90 2024-04-20T11:31:46.982 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:46 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4167407905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:31:46.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.979+0000 7f7152000640 1 -- 172.21.15.44:0/1088624965 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71540057b0 con 0x555809fb5a90 2024-04-20T11:31:46.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f7152000640 1 -- 172.21.15.44:0/1088624965 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f714c020930 con 0x555809fb5a90 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f716da5a740 10 client.27919 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f716da5a740 10 client.27919 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f716da5a740 20 client.27919 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f716da5a740 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55580a03d620 0x55580a03fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f716da5a740 4 client.27919 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555809f218e0 con 0x55580a03d620 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f7163e00640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55580a03d620 0x55580a03fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f7163e00640 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55580a03d620 0x55580a03fa40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd6023b1b011db05 server_cookie=53c0e16b691c8815 in_seq=0 out_seq=0 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.980+0000 7f7152000640 10 client.27919 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f7152000640 1 -- 172.21.15.44:0/1088624965 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555809f218e0 con 0x55580a03d620 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f7152000640 10 client.27919 handle_command_reply: tid=0 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 client.27919 shutdown 2024-04-20T11:31:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 msgr2=0x555809fc1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fc1eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f714c009fd0 tx=0x7f714c004730 comp rx=0 tx=0).stop 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 shutdown_connections 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55580a03d620 0x55580a03fa40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555809fbe670 0x555809fc23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555809fb5a90 0x555809fc1eb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 --2- 172.21.15.44:0/1088624965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555809fb5170 0x555809fc6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f7152000640 0 client.27919 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 >> 172.21.15.44:0/1088624965 conn(0x555809faf8a0 msgr2=0x555809fb0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 shutdown_connections 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 1 -- 172.21.15.44:0/1088624965 wait complete. 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.981+0000 7f716da5a740 20 client.27919 trim_cache size 0 max 16384 2024-04-20T11:31:46.985 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:46.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71480a3d20 msgr2=0x7f71480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71480a3d20 0x7f71480a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x555809fb0a30 tx=0x7f714c005f90 comp rx=0 tx=0).stop 2024-04-20T11:31:46.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 msgr2=0x7f716419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:46.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 0x7f716419ac90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f715c02c180 tx=0x7f715c02dc70 comp rx=0 tx=0).stop 2024-04-20T11:31:46.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 shutdown_connections 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f716419a7e0 0x7f716419ac90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f71480a3d20 0x7f71480a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7164109350 0x7f71641a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 --2- 172.21.15.44:0/2159073264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7164104280 0x7f71641a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 >> 172.21.15.44:0/2159073264 conn(0x7f71640ffaa0 msgr2=0x7f71641005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 shutdown_connections 2024-04-20T11:31:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:46.982+0000 7f7169800640 1 -- 172.21.15.44:0/2159073264 wait complete. 2024-04-20T11:31:46.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:31:47.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 -- 172.21.15.44:0/1318683761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 msgr2=0x7f2914104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 --2- 172.21.15.44:0/1318683761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f2914104660 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2904009be0 tx=0x7f290402bef0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 -- 172.21.15.44:0/1318683761 shutdown_connections 2024-04-20T11:31:47.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 --2- 172.21.15.44:0/1318683761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2914109350 0x7f2914110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 --2- 172.21.15.44:0/1318683761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2914104c30 0x7f2914108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 --2- 172.21.15.44:0/1318683761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f2914104660 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 -- 172.21.15.44:0/1318683761 >> 172.21.15.44:0/1318683761 conn(0x7f29140ffaa0 msgr2=0x7f2914101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.211+0000 7f2913e00640 1 -- 172.21.15.44:0/1318683761 shutdown_connections 2024-04-20T11:31:47.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.212+0000 7f2913e00640 1 -- 172.21.15.44:0/1318683761 wait complete. 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 Processor -- start 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 -- start start 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f29141a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2914109350 0x7f29141a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291419a840 0x7f291419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2914114520 con 0x7f291419a840 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29141143a0 con 0x7f2914104280 2024-04-20T11:31:47.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.213+0000 7f2913e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29141146a0 con 0x7f2914109350 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f29141a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2913400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291419a840 0x7f291419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2914109350 0x7f29141a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f29141a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48130/0 (socket says 172.21.15.44:48130) 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 -- 172.21.15.44:0/1272496907 learned_addr learned my addr 172.21.15.44:0/1272496907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 -- 172.21.15.44:0/1272496907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2914109350 msgr2=0x7f29141a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2914109350 0x7f29141a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 -- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291419a840 msgr2=0x7f291419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291419a840 0x7f291419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.214+0000 7f2912a00640 1 -- 172.21.15.44:0/1272496907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291419b3d0 con 0x7f2914104280 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.215+0000 7f2912a00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f29141a06d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2904036040 tx=0x7f2904033720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.215+0000 7f290b400640 1 -- 172.21.15.44:0/1272496907 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904043020 con 0x7f2914104280 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.215+0000 7f290b400640 1 -- 172.21.15.44:0/1272496907 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2904033910 con 0x7f2914104280 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.215+0000 7f290b400640 1 -- 172.21.15.44:0/1272496907 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290403e6b0 con 0x7f2914104280 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.215+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291419b660 con 0x7f2914104280 2024-04-20T11:31:47.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.215+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29141a77f0 con 0x7f2914104280 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.218+0000 7f291c79b740 1 Processor -- start 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 -- start start 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f290b400640 1 -- 172.21.15.44:0/1272496907 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2904046020 con 0x7f2914104280 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abdb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdba90 0x55ae6abe4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abe4670 0x55ae6abe6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ae6ab40050 con 0x55ae6abdba90 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae6aae6490 con 0x55ae6abdb170 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f291c79b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ae6ab3c010 con 0x55ae6abe4670 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f2912a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdba90 0x55ae6abe4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f2913400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abdb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f2913400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abdb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48144/0 (socket says 172.21.15.44:48144) 2024-04-20T11:31:47.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f2913400640 1 -- 172.21.15.44:0/2855370008 learned_addr learned my addr 172.21.15.44:0/2855370008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.219+0000 7f2912000640 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abe4670 0x55ae6abe6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2913400640 1 -- 172.21.15.44:0/2855370008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abe4670 msgr2=0x55ae6abe6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2913400640 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abe4670 0x55ae6abe6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2913400640 1 -- 172.21.15.44:0/2855370008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdba90 msgr2=0x55ae6abe4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2913400640 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdba90 0x55ae6abe4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2913400640 1 -- 172.21.15.44:0/2855370008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6aaee070 con 0x55ae6abdb170 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2913400640 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abdb550 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f290c009c20 tx=0x7f290c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24a189a5405a4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2903e00640 1 -- 172.21.15.44:0/2855370008 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c01e070 con 0x55ae6abdb170 2024-04-20T11:31:47.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2903e00640 1 -- 172.21.15.44:0/2855370008 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f290c019b30 con 0x55ae6abdb170 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.220+0000 7f2903e00640 1 -- 172.21.15.44:0/2855370008 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00eaa0 con 0x55ae6abdb170 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f290b400640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f28f80a3d20 0x7f28f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f290b400640 1 -- 172.21.15.44:0/1272496907 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f29040f2bf0 con 0x7f2914104280 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f2912a00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f28f80a3d20 0x7f28f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 -- 172.21.15.44:0/2855370008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 msgr2=0x55ae6abdb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abdb550 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f290c009c20 tx=0x7f290c011810 comp rx=0 tx=0).stop 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 -- 172.21.15.44:0/2855370008 shutdown_connections 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abe4670 0x55ae6abe6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdba90 0x55ae6abe4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 --2- 172.21.15.44:0/2855370008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abdb550 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 -- 172.21.15.44:0/2855370008 >> 172.21.15.44:0/2855370008 conn(0x55ae6abd58a0 msgr2=0x55ae6abd5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 -- 172.21.15.44:0/2855370008 shutdown_connections 2024-04-20T11:31:47.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.221+0000 7f291c79b740 1 -- 172.21.15.44:0/2855370008 wait complete. 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.222+0000 7f2912a00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f28f80a3d20 0x7f28f80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ae6abdac40 tx=0x7f29040f66a0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.222+0000 7f291c79b740 1 -- start start 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.222+0000 7f291c79b740 1 Processor -- start 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abe1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ae6abe7460 con 0x55ae6abe4670 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae6aaf9860 con 0x55ae6abdb170 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ae6abe75e0 con 0x55ae6abdba90 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f291c79b740 10 client.0 _ll_register_callbacks cb 0x55ae6aac7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2913400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abe1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2913400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abe1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48148/0 (socket says 172.21.15.44:48148) 2024-04-20T11:31:47.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2913400640 1 -- 172.21.15.44:0/1222382184 learned_addr learned my addr 172.21.15.44:0/1222382184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2912000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2913400640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2912000640 1 -- 172.21.15.44:0/1222382184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 msgr2=0x55ae6abe1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2912000640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abe1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2912000640 1 -- 172.21.15.44:0/1222382184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 msgr2=0x55ae6abe2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2912000640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.223+0000 7f2912000640 1 -- 172.21.15.44:0/1222382184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6aafd950 con 0x55ae6abdba90 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f2913400640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f2912000640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe1c50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f291410a920 tx=0x7f28fc011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daed9777dd33a1be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f2900c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f2900c00640 1 -- 172.21.15.44:0/1222382184 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc01e020 con 0x55ae6abdba90 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f2900c00640 1 -- 172.21.15.44:0/1222382184 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f28fc00fb00 con 0x55ae6abdba90 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c019b30 con 0x55ae6abdba90 2024-04-20T11:31:47.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f2900c00640 1 -- 172.21.15.44:0/1222382184 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f28fc00fd00 con 0x55ae6abdba90 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f291c79b740 10 client.27931 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.224+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ae6aacb820 con 0x55ae6abdba90 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f2900c00640 1 -- 172.21.15.44:0/1222382184 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f290c019b30 con 0x55ae6abdba90 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f2900c00640 1 -- 172.21.15.44:0/1222382184 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f28fc019070 con 0x55ae6abdba90 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f291c79b740 10 client.27931 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f291c79b740 10 client.27931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f291c79b740 20 client.27931 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f291c79b740 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6abf4770 0x55ae6abf6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f291c79b740 4 client.27931 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.225+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae6ab4da70 con 0x55ae6abf4770 2024-04-20T11:31:47.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.226+0000 7f2913400640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6abf4770 0x55ae6abf6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.226+0000 7f2913400640 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6abf4770 0x55ae6abf6bb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15ae3c86ed56ebea server_cookie=6902a9bb37a17503 in_seq=0 out_seq=0 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.226+0000 7f2900c00640 10 client.27931 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f2900c00640 1 -- 172.21.15.44:0/1222382184 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae6ab4da70 con 0x55ae6abf4770 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f2900c00640 10 client.27931 handle_command_reply: tid=0 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 client.27931 shutdown 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 msgr2=0x55ae6abe1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe1c50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f291410a920 tx=0x7f28fc011b50 comp rx=0 tx=0).stop 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 shutdown_connections 2024-04-20T11:31:47.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6abf4770 0x55ae6abf6bb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe1c50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f2900c00640 0 client.27931 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 --2- 172.21.15.44:0/1222382184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abe1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.227+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 >> 172.21.15.44:0/1222382184 conn(0x55ae6abd58a0 msgr2=0x55ae6abd75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.228+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 shutdown_connections 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.228+0000 7f291c79b740 1 -- 172.21.15.44:0/1222382184 wait complete. 2024-04-20T11:31:47.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.228+0000 7f291c79b740 20 client.27931 trim_cache size 0 max 16384 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 Processor -- start 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 -- start start 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 0x55ae6abe3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abf0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abe4670 0x55ae6abfc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29141afd20 con 0x55ae6abdb170 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b1860 con 0x55ae6abe4670 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f291c79b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b03f0 con 0x55ae6abdba90 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f2913400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 0x55ae6abe3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f2913400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 0x55ae6abe3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50672/0 (socket says 172.21.15.44:50672) 2024-04-20T11:31:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.243+0000 7f2913400640 1 -- 172.21.15.44:0/448312521 learned_addr learned my addr 172.21.15.44:0/448312521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2912000640 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abf0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abe4670 0x55ae6abfc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 -- 172.21.15.44:0/448312521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 msgr2=0x55ae6abf0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abf0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 -- 172.21.15.44:0/448312521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abe4670 msgr2=0x55ae6abfc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abe4670 0x55ae6abfc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 -- 172.21.15.44:0/448312521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6a8523f0 con 0x55ae6abdb170 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2913400640 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 0x55ae6abe3e40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f290c009c20 tx=0x7f290c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bee2e02f4663ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2903400640 1 -- 172.21.15.44:0/448312521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c01e040 con 0x55ae6abdb170 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2903400640 1 -- 172.21.15.44:0/448312521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f290c01f040 con 0x55ae6abdb170 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.244+0000 7f2903400640 1 -- 172.21.15.44:0/448312521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c02e200 con 0x55ae6abdb170 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 -- 172.21.15.44:0/448312521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 msgr2=0x55ae6abe3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 0x55ae6abe3e40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f290c009c20 tx=0x7f290c005f90 comp rx=0 tx=0).stop 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 -- 172.21.15.44:0/448312521 shutdown_connections 2024-04-20T11:31:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abe4670 0x55ae6abfc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abf0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 --2- 172.21.15.44:0/448312521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abdb170 0x55ae6abe3e40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 -- 172.21.15.44:0/448312521 >> 172.21.15.44:0/448312521 conn(0x55ae6abd58a0 msgr2=0x55ae6abdbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 -- 172.21.15.44:0/448312521 shutdown_connections 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.245+0000 7f291c79b740 1 -- 172.21.15.44:0/448312521 wait complete. 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.246+0000 7f291c79b740 1 -- start start 2024-04-20T11:31:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.246+0000 7f291c79b740 1 Processor -- start 2024-04-20T11:31:47.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.246+0000 7f291c79b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f291c79b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f291c79b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f291c79b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b1190 con 0x55ae6abe4670 2024-04-20T11:31:47.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f291c79b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b0ac0 con 0x55ae6abdb170 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f291c79b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29141add10 con 0x55ae6abdba90 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f291c79b740 10 client.0 _ll_register_callbacks cb 0x7f28fc009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33148/0 (socket says 172.21.15.44:33148) 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 -- 172.21.15.44:0/635221574 learned_addr learned my addr 172.21.15.44:0/635221574 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2913400640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2913400640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 -- 172.21.15.44:0/635221574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 msgr2=0x55ae6abecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abecd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 -- 172.21.15.44:0/635221574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 msgr2=0x55ae6abe87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2912000640 1 -- 172.21.15.44:0/635221574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6a91c7d0 con 0x55ae6abdba90 2024-04-20T11:31:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.247+0000 7f2913400640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f2912000640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe8100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28fc000c00 tx=0x7f28fc005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b7c40a06701fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f2900c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f2900c00640 1 -- 172.21.15.44:0/635221574 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc01e070 con 0x55ae6abdba90 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f2900c00640 1 -- 172.21.15.44:0/635221574 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f28fc00fb00 con 0x55ae6abdba90 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c01f040 con 0x55ae6abdba90 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f2900c00640 1 -- 172.21.15.44:0/635221574 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f28fc005040 con 0x55ae6abdba90 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f291c79b740 10 client.27937 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.248+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ae6ac5acc0 con 0x55ae6abdba90 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f2900c00640 1 -- 172.21.15.44:0/635221574 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f290c01f040 con 0x55ae6abdba90 2024-04-20T11:31:47.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f2900c00640 1 -- 172.21.15.44:0/635221574 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f28fc019040 con 0x55ae6abdba90 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f291c79b740 10 client.27937 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f291c79b740 10 client.27937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f291c79b740 20 client.27937 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f291c79b740 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6ac5b460 0x55ae6ac5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f291c79b740 4 client.27937 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae6ab478e0 con 0x55ae6ac5b460 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.249+0000 7f2913400640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6ac5b460 0x55ae6ac5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f2913400640 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6ac5b460 0x55ae6ac5d880 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ed9eb672e75ccc8 server_cookie=31c628c6ca2654f in_seq=0 out_seq=0 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f2900c00640 10 client.27937 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f2900c00640 1 -- 172.21.15.44:0/635221574 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae6ab478e0 con 0x55ae6ac5b460 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f2900c00640 10 client.27937 handle_command_reply: tid=0 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 client.27937 shutdown 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 msgr2=0x55ae6abe8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe8100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28fc000c00 tx=0x7f28fc005e70 comp rx=0 tx=0).stop 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 shutdown_connections 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55ae6ac5b460 0x55ae6ac5d880 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae6abe4670 0x55ae6abe87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f2900c00640 0 client.27937 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ae6abdba90 0x55ae6abe8100 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 --2- 172.21.15.44:0/635221574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae6abdb170 0x55ae6abecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.250+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 >> 172.21.15.44:0/635221574 conn(0x55ae6abd58a0 msgr2=0x55ae6abd40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 shutdown_connections 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f291c79b740 1 -- 172.21.15.44:0/635221574 wait complete. 2024-04-20T11:31:47.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f291c79b740 20 client.27937 trim_cache size 0 max 16384 2024-04-20T11:31:47.255 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:47.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f28f80a3d20 msgr2=0x7f28f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f28f80a3d20 0x7f28f80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ae6abdac40 tx=0x7f29040f66a0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 msgr2=0x7f29141a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f29141a06d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2904036040 tx=0x7f2904033720 comp rx=0 tx=0).stop 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 shutdown_connections 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f28f80a3d20 0x7f28f80a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f291419a840 0x7f291419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2914109350 0x7f29141a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 --2- 172.21.15.44:0/1272496907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2914104280 0x7f29141a06d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.251+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 >> 172.21.15.44:0/1272496907 conn(0x7f29140ffaa0 msgr2=0x7f2914101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.252+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 shutdown_connections 2024-04-20T11:31:47.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.252+0000 7f2913e00640 1 -- 172.21.15.44:0/1272496907 wait complete. 2024-04-20T11:31:47.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 -- 172.21.15.44:0/2402336312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4102080 msgr2=0x7feae4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 --2- 172.21.15.44:0/2402336312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4102080 0x7feae4102460 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fead8009900 tx=0x7fead802bc40 comp rx=0 tx=0).stop 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 -- 172.21.15.44:0/2402336312 shutdown_connections 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 --2- 172.21.15.44:0/2402336312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4107150 0x7feae410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 --2- 172.21.15.44:0/2402336312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae4102a30 0x7feae4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 --2- 172.21.15.44:0/2402336312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4102080 0x7feae4102460 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 -- 172.21.15.44:0/2402336312 >> 172.21.15.44:0/2402336312 conn(0x7feae40fd900 msgr2=0x7feae40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 -- 172.21.15.44:0/2402336312 shutdown_connections 2024-04-20T11:31:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.488+0000 7feaeac00640 1 -- 172.21.15.44:0/2402336312 wait complete. 2024-04-20T11:31:47.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.489+0000 7feaeac00640 1 Processor -- start 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.489+0000 7feaeac00640 1 -- start start 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.489+0000 7feaeac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4102080 0x7feae419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaeac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 0x7feae419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaeac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae41986c0 0x7feae4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaeac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feae4112320 con 0x7feae4107150 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaeac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feae41121a0 con 0x7feae4102080 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaeac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feae41124a0 con 0x7feae41986c0 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 0x7feae419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 0x7feae419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50698/0 (socket says 172.21.15.44:50698) 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 -- 172.21.15.44:0/3732174809 learned_addr learned my addr 172.21.15.44:0/3732174809 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae9800640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4102080 0x7feae419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaea200640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae41986c0 0x7feae4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 -- 172.21.15.44:0/3732174809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae41986c0 msgr2=0x7feae4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae41986c0 0x7feae4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 -- 172.21.15.44:0/3732174809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4102080 msgr2=0x7feae419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4102080 0x7feae419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae8e00640 1 -- 172.21.15.44:0/3732174809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae41992c0 con 0x7feae4107150 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feaea200640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae41986c0 0x7feae4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.490+0000 7feae9800640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4102080 0x7feae419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.491+0000 7feae8e00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 0x7feae419eb40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fead000d770 tx=0x7fead000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.491+0000 7feae2000640 1 -- 172.21.15.44:0/3732174809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0017070 con 0x7feae4107150 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.491+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41a53a0 con 0x7feae4107150 2024-04-20T11:31:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.491+0000 7feae2000640 1 -- 172.21.15.44:0/3732174809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fead0009c30 con 0x7feae4107150 2024-04-20T11:31:47.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.491+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae41a57a0 con 0x7feae4107150 2024-04-20T11:31:47.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.491+0000 7feae2000640 1 -- 172.21.15.44:0/3732174809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0021060 con 0x7feae4107150 2024-04-20T11:31:47.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 Processor -- start 2024-04-20T11:31:47.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 -- start start 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bc4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87bd4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cd87b2a050 con 0x55cd87bc4b50 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cd87ad0490 con 0x55cd87bc5500 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.495+0000 7feaeee82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cd87b26010 con 0x55cd87bd2a80 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae2000640 1 -- 172.21.15.44:0/3732174809 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fead001a070 con 0x7feae4107150 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feaea200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bc4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae9800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feaea200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bc4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50702/0 (socket says 172.21.15.44:50702) 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feaea200640 1 -- 172.21.15.44:0/476324061 learned_addr learned my addr 172.21.15.44:0/476324061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae8e00640 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87bd4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae9800640 1 -- 172.21.15.44:0/476324061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 msgr2=0x55cd87bd4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae9800640 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87bd4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae9800640 1 -- 172.21.15.44:0/476324061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 msgr2=0x55cd87bc4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae9800640 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bc4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feae9800640 1 -- 172.21.15.44:0/476324061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd87ad8070 con 0x55cd87bc5500 2024-04-20T11:31:47.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.496+0000 7feaea200640 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bc4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.497+0000 7feae9800640 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd2540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fead802c180 tx=0x7fead802d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aef7045cfabaa1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.497+0000 7fead6a00640 1 -- 172.21.15.44:0/476324061 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8045070 con 0x55cd87bc5500 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.497+0000 7fead6a00640 1 -- 172.21.15.44:0/476324061 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fead802dbe0 con 0x55cd87bc5500 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.497+0000 7fead6a00640 1 -- 172.21.15.44:0/476324061 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8005650 con 0x55cd87bc5500 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feae2000640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feacc0a3d20 0x7feacc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feae2000640 1 -- 172.21.15.44:0/3732174809 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fead00d0350 con 0x7feae4107150 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaea200640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feacc0a3d20 0x7feacc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 -- 172.21.15.44:0/476324061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 msgr2=0x55cd87bd2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd2540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fead802c180 tx=0x7fead802d9f0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 -- 172.21.15.44:0/476324061 shutdown_connections 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87bd4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd2540 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 --2- 172.21.15.44:0/476324061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bc4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 -- 172.21.15.44:0/476324061 >> 172.21.15.44:0/476324061 conn(0x55cd87bbf8a0 msgr2=0x55cd87bbfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.498+0000 7feaeee82740 1 -- 172.21.15.44:0/476324061 shutdown_connections 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.499+0000 7feaeee82740 1 -- 172.21.15.44:0/476324061 wait complete. 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.499+0000 7feaea200640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feacc0a3d20 0x7feacc0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feadc005c40 tx=0x7feadc009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.499+0000 7feaeee82740 1 -- start start 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.499+0000 7feaeee82740 1 Processor -- start 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bcfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc5500 0x55cd87bd0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bd2a80 0x55cd87bd0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cd87bd57e0 con 0x55cd87bc4b50 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cd87ae3860 con 0x55cd87bd2a80 2024-04-20T11:31:47.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cd87bd5960 con 0x55cd87bc5500 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaeee82740 10 client.0 _ll_register_callbacks cb 0x55cd87ab1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feae9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bcfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feae9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bcfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50710/0 (socket says 172.21.15.44:50710) 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feae9800640 1 -- 172.21.15.44:0/3872181562 learned_addr learned my addr 172.21.15.44:0/3872181562 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feaea200640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bd2a80 0x55cd87bd0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.500+0000 7feae8e00640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc5500 0x55cd87bd0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feae9800640 1 -- 172.21.15.44:0/3872181562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc5500 msgr2=0x55cd87bd0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feae9800640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc5500 0x55cd87bd0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feae9800640 1 -- 172.21.15.44:0/3872181562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bd2a80 msgr2=0x55cd87bd0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feae9800640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bd2a80 0x55cd87bd0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feae9800640 1 -- 172.21.15.44:0/3872181562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd87ae7950 con 0x55cd87bc4b50 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feae9800640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bcfac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fead8036040 tx=0x7fead80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86464772c6e293df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feacb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:47.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feacb400640 1 -- 172.21.15.44:0/3872181562 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead804d020 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feacb400640 1 -- 172.21.15.44:0/3872181562 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fead80344e0 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.501+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead802dbe0 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feacb400640 1 -- 172.21.15.44:0/3872181562 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fead8045070 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 10 client.17826 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cd87ab5820 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feacb400640 1 -- 172.21.15.44:0/3872181562 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fead802dbe0 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feacb400640 1 -- 172.21.15.44:0/3872181562 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fead8041610 con 0x55cd87bc4b50 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 10 client.17826 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 10 client.17826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 20 client.17826 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87be28d0 0x55cd87be4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 4 client.17826 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.502+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd87b37a70 con 0x55cd87be28d0 2024-04-20T11:31:47.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.503+0000 7feae8e00640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87be28d0 0x55cd87be4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.503+0000 7feae8e00640 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87be28d0 0x55cd87be4cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5987ab324dd3b688 server_cookie=1f21e3f8d774b290 in_seq=0 out_seq=0 2024-04-20T11:31:47.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.503+0000 7feacb400640 10 client.17826 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feacb400640 1 -- 172.21.15.44:0/3872181562 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd87b37a70 con 0x55cd87be28d0 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feacb400640 10 client.17826 handle_command_reply: tid=0 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 client.17826 shutdown 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 msgr2=0x55cd87bcfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bcfac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fead8036040 tx=0x7fead80038a0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 shutdown_connections 2024-04-20T11:31:47.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87be28d0 0x55cd87be4cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bd2a80 0x55cd87bd0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc5500 0x55cd87bd0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 --2- 172.21.15.44:0/3872181562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bcfac0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 >> 172.21.15.44:0/3872181562 conn(0x55cd87bbf8a0 msgr2=0x55cd87bc15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feacb400640 0 client.17826 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.504+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 shutdown_connections 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.505+0000 7feaeee82740 1 -- 172.21.15.44:0/3872181562 wait complete. 2024-04-20T11:31:47.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.505+0000 7feaeee82740 20 client.17826 trim_cache size 0 max 16384 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 Processor -- start 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 -- start start 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc4b50 0x55cd87bd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 0x55cd87bea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feae41ab940 con 0x55cd87bd2a80 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feae41ad480 con 0x55cd87bc5500 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaeee82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feae41ac010 con 0x55cd87bc4b50 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaea200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 0x55cd87bea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaea200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 0x55cd87bea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50718/0 (socket says 172.21.15.44:50718) 2024-04-20T11:31:47.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feaea200640 1 -- 172.21.15.44:0/3715695916 learned_addr learned my addr 172.21.15.44:0/3715695916 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.520+0000 7feae8e00640 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7feaea200640 1 -- 172.21.15.44:0/3715695916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc4b50 msgr2=0x55cd87bd21f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T11:31:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7feaea200640 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc4b50 0x55cd87bd21f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7feaea200640 1 -- 172.21.15.44:0/3715695916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 msgr2=0x55cd87bde6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7feaea200640 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bde6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7feaea200640 1 -- 172.21.15.44:0/3715695916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8783c3f0 con 0x55cd87bd2a80 2024-04-20T11:31:47.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7feaea200640 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 0x55cd87bea320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feadc010960 tx=0x7feadc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b2603fd8d2a37c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7fead6000640 1 -- 172.21.15.44:0/3715695916 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc004b20 con 0x55cd87bd2a80 2024-04-20T11:31:47.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7fead6000640 1 -- 172.21.15.44:0/3715695916 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feadc004cc0 con 0x55cd87bd2a80 2024-04-20T11:31:47.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.521+0000 7fead6000640 1 -- 172.21.15.44:0/3715695916 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc01c3f0 con 0x55cd87bd2a80 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 -- 172.21.15.44:0/3715695916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 msgr2=0x55cd87bea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 0x55cd87bea320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feadc010960 tx=0x7feadc019000 comp rx=0 tx=0).stop 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 -- 172.21.15.44:0/3715695916 shutdown_connections 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bd2a80 0x55cd87bea320 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bde6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 --2- 172.21.15.44:0/3715695916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bc4b50 0x55cd87bd21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 -- 172.21.15.44:0/3715695916 >> 172.21.15.44:0/3715695916 conn(0x55cd87bbf8a0 msgr2=0x55cd87bd0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 -- 172.21.15.44:0/3715695916 shutdown_connections 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 -- 172.21.15.44:0/3715695916 wait complete. 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.522+0000 7feaeee82740 1 -- start start 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 Processor -- start 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bd6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87c47b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feae41acdb0 con 0x55cd87bc4b50 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feae41ac6e0 con 0x55cd87bc5500 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feae41a9930 con 0x55cd87bd2a80 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feaeee82740 10 client.0 _ll_register_callbacks cb 0x7fead8004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feae9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bd6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feae9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bd6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50720/0 (socket says 172.21.15.44:50720) 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.523+0000 7feae9800640 1 -- 172.21.15.44:0/2693625645 learned_addr learned my addr 172.21.15.44:0/2693625645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae8e00640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feaea200640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87c47b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae9800640 1 -- 172.21.15.44:0/2693625645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 msgr2=0x55cd87c47b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae9800640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87c47b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae9800640 1 -- 172.21.15.44:0/2693625645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 msgr2=0x55cd87bd6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae9800640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae9800640 1 -- 172.21.15.44:0/2693625645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd879067d0 con 0x55cd87bc4b50 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feae9800640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bd6960 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fead802c180 tx=0x7fead8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5abb78562b453b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feacb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feacb400640 1 -- 172.21.15.44:0/2693625645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead804d040 con 0x55cd87bc4b50 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feacb400640 1 -- 172.21.15.44:0/2693625645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fead802c9e0 con 0x55cd87bc4b50 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.524+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feadc004cc0 con 0x55cd87bc4b50 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feacb400640 1 -- 172.21.15.44:0/2693625645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fead8041890 con 0x55cd87bc4b50 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 10 client.17832 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cd87c49000 con 0x55cd87bc4b50 2024-04-20T11:31:47.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feacb400640 1 -- 172.21.15.44:0/2693625645 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feadc004cc0 con 0x55cd87bc4b50 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feacb400640 1 -- 172.21.15.44:0/2693625645 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fead8034c40 con 0x55cd87bc4b50 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 10 client.17832 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 10 client.17832 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 20 client.17832 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87c497a0 0x55cd87c4bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 4 client.17832 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.525+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd87b318e0 con 0x55cd87c497a0 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.526+0000 7feae8e00640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87c497a0 0x55cd87c4bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.526+0000 7feae8e00640 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87c497a0 0x55cd87c4bbc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9b34501ea25cad8 server_cookie=c7c677f0cec21a6d in_seq=0 out_seq=0 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.526+0000 7feacb400640 10 client.17832 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feacb400640 1 -- 172.21.15.44:0/2693625645 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd87b318e0 con 0x55cd87c497a0 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feacb400640 10 client.17832 handle_command_reply: tid=0 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 client.17832 shutdown 2024-04-20T11:31:47.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 msgr2=0x55cd87bd6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bd6960 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fead802c180 tx=0x7fead8005a20 comp rx=0 tx=0).stop 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 shutdown_connections 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x55cd87c497a0 0x55cd87c4bbc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd87bd2a80 0x55cd87c47b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd87bc5500 0x55cd87bd6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feacb400640 0 client.17832 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 --2- 172.21.15.44:0/2693625645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cd87bc4b50 0x55cd87bd6960 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 >> 172.21.15.44:0/2693625645 conn(0x55cd87bbf8a0 msgr2=0x55cd87bbe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 shutdown_connections 2024-04-20T11:31:47.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 1 -- 172.21.15.44:0/2693625645 wait complete. 2024-04-20T11:31:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.527+0000 7feaeee82740 20 client.17832 trim_cache size 0 max 16384 2024-04-20T11:31:47.537 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feacc0a3d20 msgr2=0x7feacc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feacc0a3d20 0x7feacc0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feadc005c40 tx=0x7feadc009500 comp rx=0 tx=0).stop 2024-04-20T11:31:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 msgr2=0x7feae419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 0x7feae419eb40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fead000d770 tx=0x7fead000dc30 comp rx=0 tx=0).stop 2024-04-20T11:31:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 shutdown_connections 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae41986c0 0x7feae4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7feacc0a3d20 0x7feacc0a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feae4107150 0x7feae419eb40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 --2- 172.21.15.44:0/3732174809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae4102080 0x7feae419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 >> 172.21.15.44:0/3732174809 conn(0x7feae40fd900 msgr2=0x7feae40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 shutdown_connections 2024-04-20T11:31:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.528+0000 7feaeac00640 1 -- 172.21.15.44:0/3732174809 wait complete. 2024-04-20T11:31:47.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:31:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 -- 172.21.15.44:0/2775050628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101740 msgr2=0x7f3af0105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 --2- 172.21.15.44:0/2775050628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101740 0x7f3af0105c00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009900 tx=0x7f3ad802bd10 comp rx=0 tx=0).stop 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 -- 172.21.15.44:0/2775050628 shutdown_connections 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 --2- 172.21.15.44:0/2775050628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3af0106220 0x7f3af01066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 --2- 172.21.15.44:0/2775050628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0101740 0x7f3af0105c00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 --2- 172.21.15.44:0/2775050628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0100e20 0x7f3af0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.768+0000 7f3af4a00640 1 -- 172.21.15.44:0/2775050628 >> 172.21.15.44:0/2775050628 conn(0x7f3af00fc870 msgr2=0x7f3af00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.769+0000 7f3af4a00640 1 -- 172.21.15.44:0/2775050628 shutdown_connections 2024-04-20T11:31:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.769+0000 7f3af4a00640 1 -- 172.21.15.44:0/2775050628 wait complete. 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.770+0000 7f3af4a00640 1 Processor -- start 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.770+0000 7f3af4a00640 1 -- start start 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3af4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3af0100e20 0x7f3af019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3af4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 0x7f3af019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3af4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0106220 0x7f3af01a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3af4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0118980 con 0x7f3af0106220 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3af4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0118800 con 0x7f3af0101740 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3af4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0118b00 con 0x7f3af0100e20 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3aef200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0106220 0x7f3af01a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 0x7f3af019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 0x7f3af019f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48256/0 (socket says 172.21.15.44:48256) 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3ae5e00640 1 -- 172.21.15.44:0/3494807043 learned_addr learned my addr 172.21.15.44:0/3494807043 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.771+0000 7f3aee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3af0100e20 0x7f3af019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3aef200640 1 -- 172.21.15.44:0/3494807043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3af0100e20 msgr2=0x7f3af019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3aef200640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3af0100e20 0x7f3af019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3aef200640 1 -- 172.21.15.44:0/3494807043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 msgr2=0x7f3af019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3aef200640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 0x7f3af019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3aef200640 1 -- 172.21.15.44:0/3494807043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af01a3e10 con 0x7f3af0106220 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 0x7f3af019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3aef200640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0106220 0x7f3af01a36e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009fd0 tx=0x7f3ae800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3ae7e00640 1 -- 172.21.15.44:0/3494807043 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800ce20 con 0x7f3af0106220 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3ae7e00640 1 -- 172.21.15.44:0/3494807043 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ae8004b20 con 0x7f3af0106220 2024-04-20T11:31:47.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af01a4100 con 0x7f3af0106220 2024-04-20T11:31:47.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3ae7e00640 1 -- 172.21.15.44:0/3494807043 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8010790 con 0x7f3af0106220 2024-04-20T11:31:47.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.772+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af01ab990 con 0x7f3af0106220 2024-04-20T11:31:47.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.777+0000 7f3af801e740 1 Processor -- start 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.777+0000 7f3ae7e00640 1 -- 172.21.15.44:0/3494807043 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3ae8005020 con 0x7f3af0106220 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.777+0000 7f3af801e740 1 -- start start 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.777+0000 7f3af801e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d780550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.777+0000 7f3af801e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 0x55746d789130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.777+0000 7f3af801e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d789670 0x55746d78ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3af801e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55746d6e5050 con 0x55746d780170 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3af801e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55746d68b490 con 0x55746d789670 2024-04-20T11:31:47.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3af801e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55746d6e1010 con 0x55746d780a90 2024-04-20T11:31:47.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d780550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 0x55746d789130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 0x55746d789130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33248/0 (socket says 172.21.15.44:33248) 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 -- 172.21.15.44:0/2659061535 learned_addr learned my addr 172.21.15.44:0/2659061535 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d789670 0x55746d78ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 -- 172.21.15.44:0/2659061535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d789670 msgr2=0x55746d78ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d789670 0x55746d78ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 -- 172.21.15.44:0/2659061535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 msgr2=0x55746d780550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d780550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3aee800640 1 -- 172.21.15.44:0/2659061535 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746d693070 con 0x55746d780a90 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.778+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d780550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3ae7e00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ad40a3da0 0x7f3ad40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3aee800640 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 0x55746d789130 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009cf0 tx=0x7f3ae0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=349ef0008ac25aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3ae7e00640 1 -- 172.21.15.44:0/3494807043 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f3ae80d0e90 con 0x7f3af0106220 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ad40a3da0 0x7f3ad40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3adf400640 1 -- 172.21.15.44:0/2659061535 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0004230 con 0x55746d780a90 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3adf400640 1 -- 172.21.15.44:0/2659061535 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ae00043d0 con 0x55746d780a90 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.779+0000 7f3adf400640 1 -- 172.21.15.44:0/2659061535 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00199d0 con 0x55746d780a90 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 -- 172.21.15.44:0/2659061535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 msgr2=0x55746d789130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ad40a3da0 0x7f3ad40a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8000c00 tx=0x7f3ad8005b50 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 0x55746d789130 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009cf0 tx=0x7f3ae0011810 comp rx=0 tx=0).stop 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 -- 172.21.15.44:0/2659061535 shutdown_connections 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d789670 0x55746d78ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780a90 0x55746d789130 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 --2- 172.21.15.44:0/2659061535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d780550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 -- 172.21.15.44:0/2659061535 >> 172.21.15.44:0/2659061535 conn(0x55746d77a8a0 msgr2=0x55746d77ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 -- 172.21.15.44:0/2659061535 shutdown_connections 2024-04-20T11:31:47.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.780+0000 7f3af801e740 1 -- 172.21.15.44:0/2659061535 wait complete. 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 -- start start 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 Processor -- start 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 0x55746d7865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780a90 0x55746d786b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d787070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55746d78c460 con 0x55746d780a90 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55746d69e860 con 0x55746d780170 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55746d78c5e0 con 0x55746d789670 2024-04-20T11:31:47.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.781+0000 7f3af801e740 10 client.0 _ll_register_callbacks cb 0x55746d66cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 0x55746d7865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780a90 0x55746d786b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 0x55746d7865f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48260/0 (socket says 172.21.15.44:48260) 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d787070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 -- 172.21.15.44:0/4153944642 learned_addr learned my addr 172.21.15.44:0/4153944642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 -- 172.21.15.44:0/4153944642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 msgr2=0x55746d787070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d787070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 -- 172.21.15.44:0/4153944642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780a90 msgr2=0x55746d786b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780a90 0x55746d786b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3aee800640 1 -- 172.21.15.44:0/4153944642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746d6a2950 con 0x55746d780170 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.782+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780a90 0x55746d786b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3aee800640 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 0x55746d7865f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0019630 tx=0x7f3ae00052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b5e0fed0baca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3ad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:47.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3ad3e00640 1 -- 172.21.15.44:0/4153944642 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0027070 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae00043d0 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3ad3e00640 1 -- 172.21.15.44:0/4153944642 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f3ae00037f0 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3ad3e00640 1 -- 172.21.15.44:0/4153944642 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0033050 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3af801e740 10 client.27962 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.783+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55746d670820 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3ad3e00640 1 -- 172.21.15.44:0/4153944642 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ae00043d0 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3ad3e00640 1 -- 172.21.15.44:0/4153944642 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3ae0003a80 con 0x55746d780170 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3af801e740 10 client.27962 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3af801e740 10 client.27962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3af801e740 20 client.27962 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3af801e740 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d799540 0x55746d79b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3af801e740 4 client.27962 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55746d6f2a70 con 0x55746d799540 2024-04-20T11:31:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.784+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d799540 0x55746d79b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.785+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d799540 0x55746d79b930 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b74560e17abc083 server_cookie=df7ce15e816ff68c in_seq=0 out_seq=0 2024-04-20T11:31:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.785+0000 7f3ad3e00640 10 client.27962 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3ad3e00640 1 -- 172.21.15.44:0/4153944642 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55746d6f2a70 con 0x55746d799540 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3ad3e00640 10 client.27962 handle_command_reply: tid=0 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 client.27962 shutdown 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 msgr2=0x55746d7865f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 0x55746d7865f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0019630 tx=0x7f3ae00052f0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 shutdown_connections 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d799540 0x55746d79b930 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d787070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780a90 0x55746d786b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3ad3e00640 0 client.27962 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 --2- 172.21.15.44:0/4153944642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780170 0x55746d7865f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 >> 172.21.15.44:0/4153944642 conn(0x55746d77a8a0 msgr2=0x55746d77c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 shutdown_connections 2024-04-20T11:31:47.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.786+0000 7f3af801e740 1 -- 172.21.15.44:0/4153944642 wait complete. 2024-04-20T11:31:47.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.787+0000 7f3af801e740 20 client.27962 trim_cache size 0 max 16384 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.801+0000 7f3af801e740 1 Processor -- start 2024-04-20T11:31:47.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.801+0000 7f3af801e740 1 -- start start 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3af801e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780170 0x55746d788d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3af801e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d795320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3af801e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 0x55746d785750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3af801e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b4040 con 0x55746d789670 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3af801e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b5b80 con 0x55746d780a90 2024-04-20T11:31:47.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3af801e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b4710 con 0x55746d780170 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 0x55746d785750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780170 0x55746d788d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 0x55746d785750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50760/0 (socket says 172.21.15.44:50760) 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 -- 172.21.15.44:0/616506932 learned_addr learned my addr 172.21.15.44:0/616506932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d795320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 -- 172.21.15.44:0/616506932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780170 msgr2=0x55746d788d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780170 0x55746d788d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 -- 172.21.15.44:0/616506932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 msgr2=0x55746d795320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d795320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aef200640 1 -- 172.21.15.44:0/616506932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746d3f73f0 con 0x55746d789670 2024-04-20T11:31:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.802+0000 7f3aee800640 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780170 0x55746d788d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.803+0000 7f3aef200640 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 0x55746d785750 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80997e0 tx=0x7f3ae809d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cceb5ff1b1ebb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.803+0000 7f3adea00640 1 -- 172.21.15.44:0/616506932 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80b5070 con 0x55746d789670 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.803+0000 7f3adea00640 1 -- 172.21.15.44:0/616506932 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ae80a5a80 con 0x55746d789670 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.803+0000 7f3adea00640 1 -- 172.21.15.44:0/616506932 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8099480 con 0x55746d789670 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 -- 172.21.15.44:0/616506932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 msgr2=0x55746d785750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 0x55746d785750 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80997e0 tx=0x7f3ae809d620 comp rx=0 tx=0).stop 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 -- 172.21.15.44:0/616506932 shutdown_connections 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d789670 0x55746d785750 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d795320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 --2- 172.21.15.44:0/616506932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d780170 0x55746d788d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 -- 172.21.15.44:0/616506932 >> 172.21.15.44:0/616506932 conn(0x55746d77a8a0 msgr2=0x55746d7790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 -- 172.21.15.44:0/616506932 shutdown_connections 2024-04-20T11:31:47.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 -- 172.21.15.44:0/616506932 wait complete. 2024-04-20T11:31:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 -- start start 2024-04-20T11:31:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.804+0000 7f3af801e740 1 Processor -- start 2024-04-20T11:31:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d78ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d78cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d7fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b54b0 con 0x55746d780170 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b4de0 con 0x55746d780a90 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b2030 con 0x55746d789670 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3af801e740 10 client.0 _ll_register_callbacks cb 0x7f3ae0003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d78cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3aee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d78ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d78cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:48286/0 (socket says 172.21.15.44:48286) 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3aee800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d78ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50776/0 (socket says 172.21.15.44:50776) 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 -- 172.21.15.44:0/978607250 learned_addr learned my addr 172.21.15.44:0/978607250 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3aef200640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d7fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 -- 172.21.15.44:0/978607250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 msgr2=0x55746d7fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d7fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 -- 172.21.15.44:0/978607250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 msgr2=0x55746d78ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.805+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d78ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3ae5e00640 1 -- 172.21.15.44:0/978607250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746d4c17d0 con 0x55746d780a90 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3aee800640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d78ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3ae5e00640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d78cf80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005c40 tx=0x7f3ad8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75e2d28128296 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3ad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3ad3e00640 1 -- 172.21.15.44:0/978607250 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8004d20 con 0x55746d780a90 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae80a5a80 con 0x55746d780a90 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.806+0000 7f3ad3e00640 1 -- 172.21.15.44:0/978607250 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f3ad80035f0 con 0x55746d780a90 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3ad3e00640 1 -- 172.21.15.44:0/978607250 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3ad80037f0 con 0x55746d780a90 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 10 client.27968 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55746d7ff140 con 0x55746d780a90 2024-04-20T11:31:47.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3ad3e00640 1 -- 172.21.15.44:0/978607250 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ae80a5a80 con 0x55746d780a90 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3ad3e00640 1 -- 172.21.15.44:0/978607250 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3ad805c070 con 0x55746d780a90 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 10 client.27968 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 10 client.27968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 20 client.27968 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d7ff870 0x55746d801c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 4 client.27968 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.807+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55746d6ec8e0 con 0x55746d7ff870 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3aee800640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d7ff870 0x55746d801c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3aee800640 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d7ff870 0x55746d801c90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e4e888af2c6855 server_cookie=ef48c2b0e6aca1bf in_seq=0 out_seq=0 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3ad3e00640 10 client.27968 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3ad3e00640 1 -- 172.21.15.44:0/978607250 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55746d6ec8e0 con 0x55746d7ff870 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3ad3e00640 10 client.27968 handle_command_reply: tid=0 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3af801e740 1 client.27968 shutdown 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.808+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 msgr2=0x55746d78cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d78cf80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005c40 tx=0x7f3ad8009260 comp rx=0 tx=0).stop 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 shutdown_connections 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55746d7ff870 0x55746d801c90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55746d789670 0x55746d7fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55746d780a90 0x55746d78cf80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 --2- 172.21.15.44:0/978607250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55746d780170 0x55746d78ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3ad3e00640 0 client.27968 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 >> 172.21.15.44:0/978607250 conn(0x55746d77a8a0 msgr2=0x55746d779960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 shutdown_connections 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 1 -- 172.21.15.44:0/978607250 wait complete. 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af801e740 20 client.27968 trim_cache size 0 max 16384 2024-04-20T11:31:47.813 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:31:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ad40a3da0 msgr2=0x7f3ad40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af4a00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ad40a3da0 0x7f3ad40a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8000c00 tx=0x7f3ad8005b50 comp rx=0 tx=0).stop 2024-04-20T11:31:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0106220 msgr2=0x7f3af01a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.809+0000 7f3af4a00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0106220 0x7f3af01a36e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009fd0 tx=0x7f3ae800ef10 comp rx=0 tx=0).stop 2024-04-20T11:31:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 shutdown_connections 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f3ad40a3da0 0x7f3ad40a6260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3af0106220 0x7f3af01a36e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0101740 0x7f3af019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 --2- 172.21.15.44:0/3494807043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3af0100e20 0x7f3af019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 >> 172.21.15.44:0/3494807043 conn(0x7f3af00fc870 msgr2=0x7f3af00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 shutdown_connections 2024-04-20T11:31:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:47.810+0000 7f3af4a00640 1 -- 172.21.15.44:0/3494807043 wait complete. 2024-04-20T11:31:47.825 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:47 smithi044 ceph-mon[16855]: pgmap v284: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:47.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:47 smithi078 ceph-mon[21464]: pgmap v284: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:47.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:47 smithi165 ceph-mon[21975]: pgmap v284: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:48.829 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:31:48.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:31:48.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:31:49.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 -- 172.21.15.44:0/3661965623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104c20 msgr2=0x7fd690108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 --2- 172.21.15.44:0/3661965623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104c20 0x7fd690108c60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c02bc30 comp rx=0 tx=0).stop 2024-04-20T11:31:49.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 -- 172.21.15.44:0/3661965623 shutdown_connections 2024-04-20T11:31:49.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 --2- 172.21.15.44:0/3661965623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd690110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 --2- 172.21.15.44:0/3661965623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104c20 0x7fd690108c60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 --2- 172.21.15.44:0/3661965623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd690104270 0x7fd690104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 -- 172.21.15.44:0/3661965623 >> 172.21.15.44:0/3661965623 conn(0x7fd6900ffa70 msgr2=0x7fd690101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 -- 172.21.15.44:0/3661965623 shutdown_connections 2024-04-20T11:31:49.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.060+0000 7fd696e00640 1 -- 172.21.15.44:0/3661965623 wait complete. 2024-04-20T11:31:49.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.061+0000 7fd696e00640 1 Processor -- start 2024-04-20T11:31:49.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 -- start start 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104270 0x7fd6901a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd6901a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd69019a7a0 0x7fd69019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901144c0 con 0x7fd690109340 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd690114340 con 0x7fd69019a7a0 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd696e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd690114640 con 0x7fd690104270 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd695000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd6901a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd695a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104270 0x7fd6901a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd695000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd6901a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50870/0 (socket says 172.21.15.44:50870) 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.062+0000 7fd695000640 1 -- 172.21.15.44:0/4081117972 learned_addr learned my addr 172.21.15.44:0/4081117972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd696400640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd69019a7a0 0x7fd69019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd695000640 1 -- 172.21.15.44:0/4081117972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104270 msgr2=0x7fd6901a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd695000640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104270 0x7fd6901a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd695000640 1 -- 172.21.15.44:0/4081117972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd69019a7a0 msgr2=0x7fd69019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd695000640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd69019a7a0 0x7fd69019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd695000640 1 -- 172.21.15.44:0/4081117972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69019b330 con 0x7fd690109340 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.063+0000 7fd695000640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd6901a0b70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009230 tx=0x7fd67c004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.064+0000 7fd68e000640 1 -- 172.21.15.44:0/4081117972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c043020 con 0x7fd690109340 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.064+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69019b5c0 con 0x7fd690109340 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.064+0000 7fd68e000640 1 -- 172.21.15.44:0/4081117972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd67c004e50 con 0x7fd690109340 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.064+0000 7fd68e000640 1 -- 172.21.15.44:0/4081117972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c003a10 con 0x7fd690109340 2024-04-20T11:31:49.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.064+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6901a77a0 con 0x7fd690109340 2024-04-20T11:31:49.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.067+0000 7fd68e000640 1 -- 172.21.15.44:0/4081117972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd67c046020 con 0x7fd690109340 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.067+0000 7fd69b06f740 1 Processor -- start 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.067+0000 7fd69b06f740 1 -- start start 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 0x559cad7344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad73f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd69b06f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559cad698d20 con 0x559cad73d5f0 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd69b06f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559cad63f3e0 con 0x559cad7340f0 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd69b06f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559cad694ce0 con 0x559cad734a10 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd696400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 0x559cad7344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd695000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad73f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd696400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 0x559cad7344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37238/0 (socket says 172.21.15.44:37238) 2024-04-20T11:31:49.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd696400640 1 -- 172.21.15.44:0/3998748932 learned_addr learned my addr 172.21.15.44:0/3998748932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.068+0000 7fd695a00640 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd696400640 1 -- 172.21.15.44:0/3998748932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 msgr2=0x559cad73d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd696400640 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd696400640 1 -- 172.21.15.44:0/3998748932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 msgr2=0x559cad73f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd696400640 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad73f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd696400640 1 -- 172.21.15.44:0/3998748932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cad65e980 con 0x559cad7340f0 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd695000640 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad73f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd696400640 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 0x559cad7344d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd688009f50 tx=0x7fd6880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48be169fcc8779d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd68e000640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6780a3d20 0x7fd6780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd682a00640 1 -- 172.21.15.44:0/3998748932 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688004d20 con 0x559cad7340f0 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd682a00640 1 -- 172.21.15.44:0/3998748932 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd688004050 con 0x559cad7340f0 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd68e000640 1 -- 172.21.15.44:0/4081117972 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fd67c0f2590 con 0x7fd690109340 2024-04-20T11:31:49.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd682a00640 1 -- 172.21.15.44:0/3998748932 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688005220 con 0x559cad7340f0 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.069+0000 7fd695a00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6780a3d20 0x7fd6780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd695a00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6780a3d20 0x7fd6780a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x559cad733bc0 tx=0x7fd684005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 -- 172.21.15.44:0/3998748932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 msgr2=0x559cad7344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 0x559cad7344d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd688009f50 tx=0x7fd6880118d0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 -- 172.21.15.44:0/3998748932 shutdown_connections 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad73f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 --2- 172.21.15.44:0/3998748932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad7340f0 0x559cad7344d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 -- 172.21.15.44:0/3998748932 >> 172.21.15.44:0/3998748932 conn(0x559cad72e820 msgr2=0x559cad72ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 -- 172.21.15.44:0/3998748932 shutdown_connections 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.070+0000 7fd69b06f740 1 -- 172.21.15.44:0/3998748932 wait complete. 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.071+0000 7fd69b06f740 1 -- start start 2024-04-20T11:31:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.071+0000 7fd69b06f740 1 Processor -- start 2024-04-20T11:31:49.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad73a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad73d5f0 0x559cad73b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559cad7403e0 con 0x559cad7340f0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559cad6527a0 con 0x559cad73d5f0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559cad740560 con 0x559cad734a10 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd69b06f740 10 client.0 _ll_register_callbacks cb 0x559cad654b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd695a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad73a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd696400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad73d5f0 0x559cad73b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd695a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad73a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50890/0 (socket says 172.21.15.44:50890) 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd695a00640 1 -- 172.21.15.44:0/1352258369 learned_addr learned my addr 172.21.15.44:0/1352258369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd695000640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd696400640 1 -- 172.21.15.44:0/1352258369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 msgr2=0x559cad73ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd696400640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd696400640 1 -- 172.21.15.44:0/1352258369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 msgr2=0x559cad73a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd696400640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad73a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd696400640 1 -- 172.21.15.44:0/1352258369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cad4737e0 con 0x559cad73d5f0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.072+0000 7fd695000640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd696400640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad73d5f0 0x559cad73b0d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd688012fd0 tx=0x7fd688004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3f82b18a3df09e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd677400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd677400640 1 -- 172.21.15.44:0/1352258369 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688029070 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd688004050 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd677400640 1 -- 172.21.15.44:0/1352258369 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fd688003e40 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd677400640 1 -- 172.21.15.44:0/1352258369 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd688035050 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd69b06f740 10 client.27980 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.073+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559cad4d3f70 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd677400640 1 -- 172.21.15.44:0/1352258369 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd688004050 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd677400640 1 -- 172.21.15.44:0/1352258369 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd6880050c0 con 0x559cad73d5f0 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd69b06f740 10 client.27980 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd69b06f740 10 client.27980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd69b06f740 20 client.27980 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd69b06f740 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad74d8e0 0x559cad74fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd69b06f740 4 client.27980 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:49.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cad6a6740 con 0x559cad74d8e0 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.074+0000 7fd695a00640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad74d8e0 0x559cad74fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.075+0000 7fd695a00640 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad74d8e0 0x559cad74fcd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490492b47266172c server_cookie=cc319e206a47012 in_seq=0 out_seq=0 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.075+0000 7fd677400640 10 client.27980 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd677400640 1 -- 172.21.15.44:0/1352258369 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559cad6a6740 con 0x559cad74d8e0 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd677400640 10 client.27980 handle_command_reply: tid=0 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 client.27980 shutdown 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad73d5f0 msgr2=0x559cad73b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad73d5f0 0x559cad73b0d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd688012fd0 tx=0x7fd688004430 comp rx=0 tx=0).stop 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 shutdown_connections 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad74d8e0 0x559cad74fcd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad73d5f0 0x559cad73b0d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad734a10 0x559cad73ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 --2- 172.21.15.44:0/1352258369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad73a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 >> 172.21.15.44:0/1352258369 conn(0x559cad72e820 msgr2=0x559cad730570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd677400640 0 client.27980 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.076+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 shutdown_connections 2024-04-20T11:31:49.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.077+0000 7fd69b06f740 1 -- 172.21.15.44:0/1352258369 wait complete. 2024-04-20T11:31:49.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.077+0000 7fd69b06f740 20 client.27980 trim_cache size 0 max 16384 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 Processor -- start 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 -- start start 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad7340f0 0x559cad7492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad73aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad72d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901afd30 con 0x559cad73d5f0 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901b1870 con 0x559cad734a10 2024-04-20T11:31:49.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.096+0000 7fd69b06f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901b0400 con 0x559cad7340f0 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd696400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad72d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad7340f0 0x559cad7492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd696400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad72d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50898/0 (socket says 172.21.15.44:50898) 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd696400640 1 -- 172.21.15.44:0/2240087254 learned_addr learned my addr 172.21.15.44:0/2240087254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695000640 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad73aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695a00640 1 -- 172.21.15.44:0/2240087254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 msgr2=0x559cad73aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695a00640 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad73aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695a00640 1 -- 172.21.15.44:0/2240087254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 msgr2=0x559cad72d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695a00640 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad72d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd695a00640 1 -- 172.21.15.44:0/2240087254 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cad36b170 con 0x559cad7340f0 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.097+0000 7fd696400640 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad72d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:49.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.098+0000 7fd695a00640 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad7340f0 0x559cad7492a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x559cad740540 tx=0x7fd68401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c5f0ad7f620c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.098+0000 7fd682000640 1 -- 172.21.15.44:0/2240087254 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684003a40 con 0x559cad7340f0 2024-04-20T11:31:49.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.098+0000 7fd682000640 1 -- 172.21.15.44:0/2240087254 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd68402b920 con 0x559cad7340f0 2024-04-20T11:31:49.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.098+0000 7fd682000640 1 -- 172.21.15.44:0/2240087254 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840047a0 con 0x559cad7340f0 2024-04-20T11:31:49.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 -- 172.21.15.44:0/2240087254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad7340f0 msgr2=0x559cad7492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad7340f0 0x559cad7492a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x559cad740540 tx=0x7fd68401b000 comp rx=0 tx=0).stop 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 -- 172.21.15.44:0/2240087254 shutdown_connections 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad73d5f0 0x559cad72d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad73aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 --2- 172.21.15.44:0/2240087254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad7340f0 0x559cad7492a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 -- 172.21.15.44:0/2240087254 >> 172.21.15.44:0/2240087254 conn(0x559cad72e820 msgr2=0x559cad72d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 -- 172.21.15.44:0/2240087254 shutdown_connections 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 -- 172.21.15.44:0/2240087254 wait complete. 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.099+0000 7fd69b06f740 1 -- start start 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 Processor -- start 2024-04-20T11:31:49.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad741360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad7418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad73d5f0 0x559cad741de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901b11a0 con 0x559cad7340f0 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901b0ad0 con 0x559cad734a10 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901add20 con 0x559cad73d5f0 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.100+0000 7fd69b06f740 10 client.0 _ll_register_callbacks cb 0x7fd67c0f5f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad741360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad741360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50910/0 (socket says 172.21.15.44:50910) 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 -- 172.21.15.44:0/4145385787 learned_addr learned my addr 172.21.15.44:0/4145385787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695000640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad7418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd696400640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad73d5f0 0x559cad741de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 -- 172.21.15.44:0/4145385787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad73d5f0 msgr2=0x559cad741de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad73d5f0 0x559cad741de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 -- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 msgr2=0x559cad7418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad7418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 -- 172.21.15.44:0/4145385787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cad7b2e80 con 0x559cad7340f0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd696400640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad73d5f0 0x559cad741de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695000640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad7418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd695a00640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad741360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd684019040 tx=0x7fd684005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10fe2782f7d9116 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.101+0000 7fd677400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd677400640 1 -- 172.21.15.44:0/4145385787 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684039020 con 0x559cad7340f0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cad6a05b0 con 0x559cad7340f0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd677400640 1 -- 172.21.15.44:0/4145385787 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7fd68402b420 con 0x559cad7340f0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd677400640 1 -- 172.21.15.44:0/4145385787 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd684044050 con 0x559cad7340f0 2024-04-20T11:31:49.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd69b06f740 10 client.17868 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:49.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559cad7b4180 con 0x559cad7340f0 2024-04-20T11:31:49.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.102+0000 7fd677400640 1 -- 172.21.15.44:0/4145385787 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x559cad6a05b0 con 0x559cad7340f0 2024-04-20T11:31:49.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd677400640 1 -- 172.21.15.44:0/4145385787 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fd684035b40 con 0x559cad7340f0 2024-04-20T11:31:49.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd69b06f740 10 client.17868 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd69b06f740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd69b06f740 20 client.17868 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd69b06f740 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad7b4890 0x559cad7b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd69b06f740 4 client.17868 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559cad61cb10 con 0x559cad7b4890 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.103+0000 7fd695000640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad7b4890 0x559cad7b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.104+0000 7fd695000640 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad7b4890 0x559cad7b6c80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7161f85a6869d635 server_cookie=215342923d2299af in_seq=0 out_seq=0 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.104+0000 7fd677400640 10 client.17868 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd677400640 1 -- 172.21.15.44:0/4145385787 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559cad61cb10 con 0x559cad7b4890 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd677400640 10 client.17868 handle_command_reply: tid=0 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 client.17868 shutdown 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 msgr2=0x559cad741360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad741360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd684019040 tx=0x7fd684005540 comp rx=0 tx=0).stop 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 shutdown_connections 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559cad7b4890 0x559cad7b6c80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559cad73d5f0 0x559cad741de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559cad734a10 0x559cad7418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 --2- 172.21.15.44:0/4145385787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559cad7340f0 0x559cad741360 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd677400640 0 client.17868 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 >> 172.21.15.44:0/4145385787 conn(0x559cad72e820 msgr2=0x559cad72d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.105+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 shutdown_connections 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.106+0000 7fd69b06f740 1 -- 172.21.15.44:0/4145385787 wait complete. 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.106+0000 7fd69b06f740 20 client.17868 trim_cache size 0 max 16384 2024-04-20T11:31:49.108 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:31:49.109 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:31:49.109 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "3ae1f73e-99b4-47d7-8467-1dbe2c129080", 2024-04-20T11:31:49.109 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:31:49.109 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.106+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6780a3d20 msgr2=0x7fd6780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.106+0000 7fd696e00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6780a3d20 0x7fd6780a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x559cad733bc0 tx=0x7fd684005f90 comp rx=0 tx=0).stop 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.106+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 msgr2=0x7fd6901a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.106+0000 7fd696e00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd6901a0b70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009230 tx=0x7fd67c004c30 comp rx=0 tx=0).stop 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 shutdown_connections 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd69019a7a0 0x7fd69019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd6780a3d20 0x7fd6780a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690109340 0x7fd6901a0b70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 --2- 172.21.15.44:0/4081117972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd690104270 0x7fd6901a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 >> 172.21.15.44:0/4081117972 conn(0x7fd6900ffa70 msgr2=0x7fd690101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 shutdown_connections 2024-04-20T11:31:49.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.107+0000 7fd696e00640 1 -- 172.21.15.44:0/4081117972 wait complete. 2024-04-20T11:31:49.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:31:49.124 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:31:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 -- 172.21.15.44:0/790463126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac010d3d0 msgr2=0x7f2ac010d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 --2- 172.21.15.44:0/790463126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac010d3d0 0x7f2ac010d7b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009900 tx=0x7f2ab002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 -- 172.21.15.44:0/790463126 shutdown_connections 2024-04-20T11:31:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 --2- 172.21.15.44:0/790463126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ac0108ea0 0x7f2ac01152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 --2- 172.21.15.44:0/790463126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108500 0x7f2ac0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 --2- 172.21.15.44:0/790463126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac010d3d0 0x7f2ac010d7b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 -- 172.21.15.44:0/790463126 >> 172.21.15.44:0/790463126 conn(0x7f2ac0077d80 msgr2=0x7f2ac0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 -- 172.21.15.44:0/790463126 shutdown_connections 2024-04-20T11:31:49.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.339+0000 7f2abfe00640 1 -- 172.21.15.44:0/790463126 wait complete. 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.340+0000 7f2abfe00640 1 Processor -- start 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.340+0000 7f2abfe00640 1 -- start start 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.340+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 0x7f2ac019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108ea0 0x7f2ac019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ac010d3d0 0x7f2ac01a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01189a0 con 0x7f2ac0108500 2024-04-20T11:31:49.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abfe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0118820 con 0x7f2ac0108ea0 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0118b20 con 0x7f2ac010d3d0 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 0x7f2ac019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 0x7f2ac019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50936/0 (socket says 172.21.15.44:50936) 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 -- 172.21.15.44:0/1011231182 learned_addr learned my addr 172.21.15.44:0/1011231182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abe000640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108ea0 0x7f2ac019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abf400640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ac010d3d0 0x7f2ac01a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 -- 172.21.15.44:0/1011231182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ac010d3d0 msgr2=0x7f2ac01a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ac010d3d0 0x7f2ac01a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 -- 172.21.15.44:0/1011231182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108ea0 msgr2=0x7f2ac019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108ea0 0x7f2ac019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abea00640 1 -- 172.21.15.44:0/1011231182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac01a3e10 con 0x7f2ac0108500 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.341+0000 7f2abe000640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108ea0 0x7f2ac019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.342+0000 7f2abea00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 0x7f2ac019ee20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000c00 tx=0x7f2ab0004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.342+0000 7f2ab7400640 1 -- 172.21.15.44:0/1011231182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0043020 con 0x7f2ac0108500 2024-04-20T11:31:49.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.342+0000 7f2ab7400640 1 -- 172.21.15.44:0/1011231182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2ab0004250 con 0x7f2ac0108500 2024-04-20T11:31:49.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.342+0000 7f2ab7400640 1 -- 172.21.15.44:0/1011231182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0003a80 con 0x7f2ac0108500 2024-04-20T11:31:49.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.342+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01a40a0 con 0x7f2ac0108500 2024-04-20T11:31:49.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.342+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01ab990 con 0x7f2ac0108500 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ab7400640 1 -- 172.21.15.44:0/1011231182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2ab0046020 con 0x7f2ac0108500 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 Processor -- start 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 -- start start 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb41823760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823ca0 0x55eb4182c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4182c880 0x55eb4182ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eb41789150 con 0x55eb41823380 2024-04-20T11:31:49.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb4172f370 con 0x55eb41823ca0 2024-04-20T11:31:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2ac8797740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eb41785110 con 0x55eb4182c880 2024-04-20T11:31:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2abf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb41823760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2abf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb41823760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50944/0 (socket says 172.21.15.44:50944) 2024-04-20T11:31:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.346+0000 7f2abf400640 1 -- 172.21.15.44:0/211877305 learned_addr learned my addr 172.21.15.44:0/211877305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abea00640 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823ca0 0x55eb4182c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abe000640 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4182c880 0x55eb4182ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abf400640 1 -- 172.21.15.44:0/211877305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4182c880 msgr2=0x55eb4182ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abf400640 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4182c880 0x55eb4182ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abf400640 1 -- 172.21.15.44:0/211877305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823ca0 msgr2=0x55eb4182c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abf400640 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823ca0 0x55eb4182c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abf400640 1 -- 172.21.15.44:0/211877305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb415657d0 con 0x55eb41823380 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2ab7400640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2aa40a3d20 0x7f2aa40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2abf400640 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb41823760 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009f40 tx=0x7f2ab80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a1a5a7708210d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.347+0000 7f2ab7400640 1 -- 172.21.15.44:0/1011231182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f2ab00f2640 con 0x7f2ac0108500 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2aabe00640 1 -- 172.21.15.44:0/211877305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8004d20 con 0x55eb41823380 2024-04-20T11:31:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2abea00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2aa40a3d20 0x7f2aa40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2aabe00640 1 -- 172.21.15.44:0/211877305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2ab8004050 con 0x55eb41823380 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2aabe00640 1 -- 172.21.15.44:0/211877305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8005260 con 0x55eb41823380 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2abea00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2aa40a3d20 0x7f2aa40a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55eb41822e50 tx=0x7f2ab00f8530 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2ac8797740 1 -- 172.21.15.44:0/211877305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 msgr2=0x55eb41823760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.348+0000 7f2ac8797740 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb41823760 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009f40 tx=0x7f2ab80118d0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 -- 172.21.15.44:0/211877305 shutdown_connections 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4182c880 0x55eb4182ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823ca0 0x55eb4182c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 --2- 172.21.15.44:0/211877305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb41823760 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 -- 172.21.15.44:0/211877305 >> 172.21.15.44:0/211877305 conn(0x55eb4181dab0 msgr2=0x55eb4181de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 -- 172.21.15.44:0/211877305 shutdown_connections 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 -- 172.21.15.44:0/211877305 wait complete. 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 -- start start 2024-04-20T11:31:49.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.349+0000 7f2ac8797740 1 Processor -- start 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb418298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41829e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb4182a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55eb4182f670 con 0x55eb41823380 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb41742740 con 0x55eb4182c880 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eb4182f7f0 con 0x55eb41823ca0 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2ac8797740 10 client.0 _ll_register_callbacks cb 0x55eb4173db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2abf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb418298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2abe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41829e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2abf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb418298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50960/0 (socket says 172.21.15.44:50960) 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.350+0000 7f2abf400640 1 -- 172.21.15.44:0/1619819221 learned_addr learned my addr 172.21.15.44:0/1619819221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abf400640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb4182a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abe000640 1 -- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 msgr2=0x55eb4182a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abe000640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb4182a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abe000640 1 -- 172.21.15.44:0/1619819221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 msgr2=0x55eb418298d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abe000640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb418298d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abe000640 1 -- 172.21.15.44:0/1619819221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4149b3f0 con 0x55eb41823ca0 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abf400640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb4182a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2abe000640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41829e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2aac012cf0 tx=0x7f2aac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40ce5ee0566fcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2aa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2aa8c00640 1 -- 172.21.15.44:0/1619819221 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac012110 con 0x55eb41823ca0 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2aa8c00640 1 -- 172.21.15.44:0/1619819221 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f2aac004050 con 0x55eb41823ca0 2024-04-20T11:31:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.351+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ab8004050 con 0x55eb41823ca0 2024-04-20T11:31:49.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.352+0000 7f2aa8c00640 1 -- 172.21.15.44:0/1619819221 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2aac0122b0 con 0x55eb41823ca0 2024-04-20T11:31:49.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.352+0000 7f2ac8797740 10 client.28000 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.352+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55eb4145b0f0 con 0x55eb41823ca0 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.352+0000 7f2aa8c00640 1 -- 172.21.15.44:0/1619819221 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2ab8004050 con 0x55eb41823ca0 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.352+0000 7f2aa8c00640 1 -- 172.21.15.44:0/1619819221 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f2aac0254d0 con 0x55eb41823ca0 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.352+0000 7f2ac8797740 10 client.28000 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2ac8797740 10 client.28000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2ac8797740 20 client.28000 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2ac8797740 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb4183cc00 0x55eb4183f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2ac8797740 4 client.28000 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb41796b70 con 0x55eb4183cc00 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2abf400640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb4183cc00 0x55eb4183f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2abf400640 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb4183cc00 0x55eb4183f030 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed0c538bd479bed server_cookie=4091769099b98019 in_seq=0 out_seq=0 2024-04-20T11:31:49.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.353+0000 7f2aa8c00640 10 client.28000 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.354+0000 7f2aa8c00640 1 -- 172.21.15.44:0/1619819221 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb41796b70 con 0x55eb4183cc00 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.354+0000 7f2aa8c00640 10 client.28000 handle_command_reply: tid=0 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.354+0000 7f2ac8797740 1 client.28000 shutdown 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 msgr2=0x55eb41829e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41829e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2aac012cf0 tx=0x7f2aac00f790 comp rx=0 tx=0).stop 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 shutdown_connections 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb4183cc00 0x55eb4183f030 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb4182a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41829e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 --2- 172.21.15.44:0/1619819221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823380 0x55eb418298d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2aa8c00640 0 client.28000 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 >> 172.21.15.44:0/1619819221 conn(0x55eb4181dab0 msgr2=0x55eb4181f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 shutdown_connections 2024-04-20T11:31:49.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 1 -- 172.21.15.44:0/1619819221 wait complete. 2024-04-20T11:31:49.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.355+0000 7f2ac8797740 20 client.28000 trim_cache size 0 max 16384 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 Processor -- start 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 -- start start 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823380 0x55eb4182c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41838560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb4182c880 0x55eb41828810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b40a0 con 0x55eb4182c880 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b5be0 con 0x55eb41823380 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2ac8797740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b4770 con 0x55eb41823ca0 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2abf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823380 0x55eb4182c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2abf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb4182c880 0x55eb41828810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2abe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41838560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2abe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41838560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38720/0 (socket says 172.21.15.44:38720) 2024-04-20T11:31:49.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.375+0000 7f2abe000640 1 -- 172.21.15.44:0/2393578334 learned_addr learned my addr 172.21.15.44:0/2393578334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.376+0000 7f2abe000640 1 -- 172.21.15.44:0/2393578334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823380 msgr2=0x55eb4182c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.376+0000 7f2abe000640 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823380 0x55eb4182c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.376+0000 7f2abe000640 1 -- 172.21.15.44:0/2393578334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb4182c880 msgr2=0x55eb41828810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.376+0000 7f2abe000640 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb4182c880 0x55eb41828810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.376+0000 7f2abe000640 1 -- 172.21.15.44:0/2393578334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4181d300 con 0x55eb41823ca0 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.376+0000 7f2abe000640 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41838560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2aac001eb0 tx=0x7f2aac00fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd8909e7a9b208cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.377+0000 7f2aab400640 1 -- 172.21.15.44:0/2393578334 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac025b10 con 0x55eb41823ca0 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.377+0000 7f2aab400640 1 -- 172.21.15.44:0/2393578334 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2aac025cb0 con 0x55eb41823ca0 2024-04-20T11:31:49.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.377+0000 7f2aab400640 1 -- 172.21.15.44:0/2393578334 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac003880 con 0x55eb41823ca0 2024-04-20T11:31:49.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.377+0000 7f2ac8797740 1 -- 172.21.15.44:0/2393578334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 msgr2=0x55eb41838560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.377+0000 7f2ac8797740 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41838560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2aac001eb0 tx=0x7f2aac00fd10 comp rx=0 tx=0).stop 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 -- 172.21.15.44:0/2393578334 shutdown_connections 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb4182c880 0x55eb41828810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823ca0 0x55eb41838560 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 --2- 172.21.15.44:0/2393578334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb41823380 0x55eb4182c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 -- 172.21.15.44:0/2393578334 >> 172.21.15.44:0/2393578334 conn(0x55eb4181dab0 msgr2=0x55eb4181cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 -- 172.21.15.44:0/2393578334 shutdown_connections 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 -- 172.21.15.44:0/2393578334 wait complete. 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 -- start start 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.378+0000 7f2ac8797740 1 Processor -- start 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823380 0x55eb41826920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823ca0 0x55eb41826e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb418a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b5510 con 0x55eb41823ca0 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b4e40 con 0x55eb4182c880 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b2090 con 0x55eb41823380 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2ac8797740 10 client.0 _ll_register_callbacks cb 0x7f2aac009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2abf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb418a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2abe000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823ca0 0x55eb41826e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2abf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb418a21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:37316/0 (socket says 172.21.15.44:37316) 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2abf400640 1 -- 172.21.15.44:0/4234587792 learned_addr learned my addr 172.21.15.44:0/4234587792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.379+0000 7f2abf400640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823380 0x55eb41826920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abe000640 1 -- 172.21.15.44:0/4234587792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823380 msgr2=0x55eb41826920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abe000640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823380 0x55eb41826920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abe000640 1 -- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 msgr2=0x55eb418a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abe000640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb418a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abe000640 1 -- 172.21.15.44:0/4234587792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb418a26f0 con 0x55eb41823ca0 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abf400640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb418a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:31:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2abe000640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823ca0 0x55eb41826e60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55eb41823c70 tx=0x7f2aac005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d936a677c101868 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:31:49.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2aa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:31:49.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2aa8c00640 1 -- 172.21.15.44:0/4234587792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac032020 con 0x55eb41823ca0 2024-04-20T11:31:49.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2aa8c00640 1 -- 172.21.15.44:0/4234587792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8455+0+0 (secure 0 0 0) 0x7f2aac0053a0 con 0x55eb41823ca0 2024-04-20T11:31:49.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aac025cb0 con 0x55eb41823ca0 2024-04-20T11:31:49.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2aa8c00640 1 -- 172.21.15.44:0/4234587792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2aac03d050 con 0x55eb41823ca0 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2ac8797740 10 client.17886 fetch_fsmap learned FSMap version 55 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.380+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55eb418a3840 con 0x55eb41823ca0 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2aa8c00640 1 -- 172.21.15.44:0/4234587792 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2aac025cb0 con 0x55eb41823ca0 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2aa8c00640 1 -- 172.21.15.44:0/4234587792 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f2aac010700 con 0x55eb41823ca0 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2ac8797740 10 client.17886 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2ac8797740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2ac8797740 20 client.17886 populate_metadata read hostname 'smithi044' 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2ac8797740 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb418a3ef0 0x55eb418a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2ac8797740 4 client.17886 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb417909e0 con 0x55eb418a3ef0 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.381+0000 7f2abf400640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb418a3ef0 0x55eb418a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:31:49.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2abf400640 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb418a3ef0 0x55eb418a62e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f5256b150df3 server_cookie=1a7b3044314f5e1e in_seq=0 out_seq=0 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2aa8c00640 10 client.17886 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2aa8c00640 1 -- 172.21.15.44:0/4234587792 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eb417909e0 con 0x55eb418a3ef0 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2aa8c00640 10 client.17886 handle_command_reply: tid=0 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 client.17886 shutdown 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823ca0 msgr2=0x55eb41826e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823ca0 0x55eb41826e60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55eb41823c70 tx=0x7f2aac005970 comp rx=0 tx=0).stop 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 shutdown_connections 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55eb418a3ef0 0x55eb418a62e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb4182c880 0x55eb418a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55eb41823ca0 0x55eb41826e60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 --2- 172.21.15.44:0/4234587792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb41823380 0x55eb41826920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2aa8c00640 0 client.17886 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.382+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 >> 172.21.15.44:0/4234587792 conn(0x55eb4181dab0 msgr2=0x55eb41829890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 shutdown_connections 2024-04-20T11:31:49.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2ac8797740 1 -- 172.21.15.44:0/4234587792 wait complete. 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2ac8797740 20 client.17886 trim_cache size 0 max 16384 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: "3ae1f73e-99b4-47d7-8467-1dbe2c129080": { 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "3ae1f73e-99b4-47d7-8467-1dbe2c129080", 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:31:49.386 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:31:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2aa40a3d20 msgr2=0x7f2aa40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2aa40a3d20 0x7f2aa40a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55eb41822e50 tx=0x7f2ab00f8530 comp rx=0 tx=0).stop 2024-04-20T11:31:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 msgr2=0x7f2ac019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:31:49.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 0x7f2ac019ee20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000c00 tx=0x7f2ab0004950 comp rx=0 tx=0).stop 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 shutdown_connections 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ac010d3d0 0x7f2ac01a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0108ea0 0x7f2ac019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2aa40a3d20 0x7f2aa40a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 --2- 172.21.15.44:0/1011231182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ac0108500 0x7f2ac019ee20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.383+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 >> 172.21.15.44:0/1011231182 conn(0x7f2ac0077d80 msgr2=0x7f2ac0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.384+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 shutdown_connections 2024-04-20T11:31:49.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:31:49.384+0000 7f2abfe00640 1 -- 172.21.15.44:0/1011231182 wait complete. 2024-04-20T11:31:49.402 INFO:tasks.cephfs.filesystem:scrub status for tag:3ae1f73e-99b4-47d7-8467-1dbe2c129080 - {'path': '/', 'tag': '3ae1f73e-99b4-47d7-8467-1dbe2c129080', 'options': 'recursive,force'} 2024-04-20T11:31:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:49 smithi078 ceph-mon[21464]: pgmap v285: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:49 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:31:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:49 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:49 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:31:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:49 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:49 smithi044 ceph-mon[16855]: pgmap v285: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:49 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:31:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:49 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:49 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:31:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:49 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:49 smithi165 ceph-mon[21975]: pgmap v285: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:31:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:49 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:31:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:49 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:49 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:31:49.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:49 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:31:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:51 smithi078 ceph-mon[21464]: pgmap v286: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:51.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:51 smithi044 ceph-mon[16855]: pgmap v286: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:51 smithi165 ceph-mon[21975]: pgmap v286: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:31:52.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:52 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:31:52.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:52 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:31:52.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:52 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:31:53.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:53 smithi078 ceph-mon[21464]: pgmap v287: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:31:53.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:53 smithi044 ceph-mon[16855]: pgmap v287: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:31:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:53 smithi165 ceph-mon[21975]: pgmap v287: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:31:55.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:55 smithi078 ceph-mon[21464]: pgmap v288: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:31:55.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:55 smithi044 ceph-mon[16855]: pgmap v288: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:31:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:55 smithi165 ceph-mon[21975]: pgmap v288: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:31:57.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:57 smithi078 ceph-mon[21464]: pgmap v289: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:31:57.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:57 smithi044 ceph-mon[16855]: pgmap v289: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:31:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:57 smithi165 ceph-mon[21975]: pgmap v289: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:31:58.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:58 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:31:58.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:58 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:31:58.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:58 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:31:59.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:31:59 smithi078 ceph-mon[21464]: pgmap v290: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:31:59.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:31:59 smithi044 ceph-mon[16855]: pgmap v290: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:31:59.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:31:59 smithi165 ceph-mon[21975]: pgmap v290: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:00.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:00 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:32:00.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:00 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:32:00.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:00 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:32:00.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:00 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:32:00.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:00 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:32:00.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:00 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:32:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:00 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:32:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:00 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:32:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:00 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:32:01.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:01 smithi078 ceph-mon[21464]: pgmap v291: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:01 smithi044 ceph-mon[16855]: pgmap v291: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:01.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:01 smithi165 ceph-mon[21975]: pgmap v291: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:03.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:03 smithi078 ceph-mon[21464]: pgmap v292: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:32:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:03 smithi044 ceph-mon[16855]: pgmap v292: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:32:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:03 smithi165 ceph-mon[21975]: pgmap v292: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-20T11:32:05.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:05 smithi078 ceph-mon[21464]: pgmap v293: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:32:05.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:05 smithi044 ceph-mon[16855]: pgmap v293: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:32:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:05 smithi165 ceph-mon[21975]: pgmap v293: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:32:07.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:07 smithi078 ceph-mon[21464]: pgmap v294: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:32:07.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:07 smithi044 ceph-mon[16855]: pgmap v294: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:32:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:07 smithi165 ceph-mon[21975]: pgmap v294: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:32:08.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:08 smithi078 ceph-mon[21464]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T11:32:08.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:08 smithi044 ceph-mon[16855]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T11:32:08.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:08 smithi165 ceph-mon[21975]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T11:32:09.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:09 smithi078 ceph-mon[21464]: pgmap v296: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:32:09.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:09 smithi044 ceph-mon[16855]: pgmap v296: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:32:09.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:09 smithi165 ceph-mon[21975]: pgmap v296: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:32:11.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:11 smithi078 ceph-mon[21464]: pgmap v297: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:32:11.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:11 smithi044 ceph-mon[16855]: pgmap v297: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:32:11.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:11 smithi165 ceph-mon[21975]: pgmap v297: 97 pgs: 97 active+clean; 147 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:32:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:13 smithi078 ceph-mon[21464]: pgmap v298: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:13 smithi078 ceph-mon[21464]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T11:32:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:13 smithi044 ceph-mon[16855]: pgmap v298: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:13.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:13 smithi044 ceph-mon[16855]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T11:32:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:13 smithi165 ceph-mon[21975]: pgmap v298: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:13.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:13 smithi165 ceph-mon[21975]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T11:32:15.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:15 smithi044 ceph-mon[16855]: pgmap v300: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:32:15.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:15 smithi165 ceph-mon[21975]: pgmap v300: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:32:16.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:15 smithi078 ceph-mon[21464]: pgmap v300: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:32:17.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:17 smithi165 ceph-mon[21975]: pgmap v301: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 117 B/s wr, 5 op/s 2024-04-20T11:32:18.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:17 smithi078 ceph-mon[21464]: pgmap v301: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 117 B/s wr, 5 op/s 2024-04-20T11:32:18.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:17 smithi044 ceph-mon[16855]: pgmap v301: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 117 B/s wr, 5 op/s 2024-04-20T11:32:19.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:32:19.403 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:32:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 -- 172.21.15.44:0/3715923236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 msgr2=0x7f66d41033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 --2- 172.21.15.44:0/3715923236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 0x7f66d41033a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009860 tx=0x7f66c402bc10 comp rx=0 tx=0).stop 2024-04-20T11:32:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 -- 172.21.15.44:0/3715923236 shutdown_connections 2024-04-20T11:32:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 --2- 172.21.15.44:0/3715923236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d410c3e0 0x7f66d410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 --2- 172.21.15.44:0/3715923236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d41038e0 0x7f66d410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 --2- 172.21.15.44:0/3715923236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 0x7f66d41033a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 -- 172.21.15.44:0/3715923236 >> 172.21.15.44:0/3715923236 conn(0x7f66d40fc7e0 msgr2=0x7f66d40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 -- 172.21.15.44:0/3715923236 shutdown_connections 2024-04-20T11:32:19.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.631+0000 7f66d3e00640 1 -- 172.21.15.44:0/3715923236 wait complete. 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.632+0000 7f66d3e00640 1 Processor -- start 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.632+0000 7f66d3e00640 1 -- start start 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 0x7f66d419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 0x7f66d419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d410c3e0 0x7f66d4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4112300 con 0x7f66d41038e0 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4112180 con 0x7f66d4102fc0 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4112480 con 0x7f66d410c3e0 2024-04-20T11:32:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 0x7f66d419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 0x7f66d419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33834/0 (socket says 172.21.15.44:33834) 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d2000640 1 -- 172.21.15.44:0/2188218203 learned_addr learned my addr 172.21.15.44:0/2188218203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 0x7f66d419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.633+0000 7f66d3400640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d410c3e0 0x7f66d4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.634+0000 7f66d2000640 1 -- 172.21.15.44:0/2188218203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d410c3e0 msgr2=0x7f66d4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.634+0000 7f66d2000640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d410c3e0 0x7f66d4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.634+0000 7f66d2000640 1 -- 172.21.15.44:0/2188218203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 msgr2=0x7f66d419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.634+0000 7f66d2000640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 0x7f66d419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.634+0000 7f66d2000640 1 -- 172.21.15.44:0/2188218203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d4198da0 con 0x7f66d41038e0 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.634+0000 7f66d2000640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 0x7f66d419e960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00eb10 tx=0x7f66bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.635+0000 7f66cb400640 1 -- 172.21.15.44:0/2188218203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc010430 con 0x7f66d41038e0 2024-04-20T11:32:19.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.635+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d4199090 con 0x7f66d41038e0 2024-04-20T11:32:19.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.635+0000 7f66cb400640 1 -- 172.21.15.44:0/2188218203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc010ad0 con 0x7f66d41038e0 2024-04-20T11:32:19.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.635+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d41a52e0 con 0x7f66d41038e0 2024-04-20T11:32:19.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.635+0000 7f66cb400640 1 -- 172.21.15.44:0/2188218203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc026060 con 0x7f66d41038e0 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.638+0000 7f66dc44f740 1 Processor -- start 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.638+0000 7f66dc44f740 1 -- start start 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.638+0000 7f66cb400640 1 -- 172.21.15.44:0/2188218203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f66bc010c80 con 0x7f66d41038e0 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af1db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af26750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66dc44f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f26ae7f150 con 0x55f26af26c90 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66dc44f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f26ae25370 con 0x55f26af1e0b0 2024-04-20T11:32:19.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66dc44f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f26ae7b110 con 0x55f26af1d790 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af29080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33844/0 (socket says 172.21.15.44:33844) 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 -- 172.21.15.44:0/225528573 learned_addr learned my addr 172.21.15.44:0/225528573 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d3400640 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af1db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2a00640 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af26750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 -- 172.21.15.44:0/225528573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 msgr2=0x55f26af1db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af1db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 -- 172.21.15.44:0/225528573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 msgr2=0x55f26af26750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.639+0000 7f66d2000640 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af26750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66d2000640 1 -- 172.21.15.44:0/225528573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26ac5b7d0 con 0x55f26af26c90 2024-04-20T11:32:19.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66d2000640 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af29080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0ce3d0 tx=0x7f66bc0d4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542a34b1ed2bae62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66cb400640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f66b40a3d20 0x7f66b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66c3e00640 1 -- 172.21.15.44:0/225528573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc099030 con 0x55f26af26c90 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66cb400640 1 -- 172.21.15.44:0/2188218203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f66bc0d0d60 con 0x7f66d41038e0 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66c3e00640 1 -- 172.21.15.44:0/225528573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc098c80 con 0x55f26af26c90 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66d3400640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f66b40a3d20 0x7f66b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.640+0000 7f66c3e00640 1 -- 172.21.15.44:0/225528573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc09c1e0 con 0x55f26af26c90 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.641+0000 7f66dc44f740 1 -- 172.21.15.44:0/225528573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 msgr2=0x55f26af29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.641+0000 7f66dc44f740 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af29080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0ce3d0 tx=0x7f66bc0d4f90 comp rx=0 tx=0).stop 2024-04-20T11:32:19.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.641+0000 7f66d3400640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f66b40a3d20 0x7f66b40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009530 tx=0x7f66cc009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 -- 172.21.15.44:0/225528573 shutdown_connections 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af29080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af26750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 --2- 172.21.15.44:0/225528573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 -- 172.21.15.44:0/225528573 >> 172.21.15.44:0/225528573 conn(0x55f26af17ec0 msgr2=0x55f26af182a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 -- 172.21.15.44:0/225528573 shutdown_connections 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 -- 172.21.15.44:0/225528573 wait complete. 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 -- start start 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.642+0000 7f66dc44f740 1 Processor -- start 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af23ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1e0b0 0x55f26af241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af24720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f26af29900 con 0x55f26af1e0b0 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f26ae38740 con 0x55f26af26c90 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f26af29a80 con 0x55f26af1d790 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66dc44f740 10 client.0 _ll_register_callbacks cb 0x55f26ae33b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1e0b0 0x55f26af241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66d3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af24720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66d3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af24720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55468/0 (socket says 172.21.15.44:55468) 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66d3400640 1 -- 172.21.15.44:0/1094782555 learned_addr learned my addr 172.21.15.44:0/1094782555 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.643+0000 7f66d2a00640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af23ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d2a00640 1 -- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 msgr2=0x55f26af24720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d2a00640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af24720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d2a00640 1 -- 172.21.15.44:0/1094782555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1e0b0 msgr2=0x55f26af241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d2a00640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1e0b0 0x55f26af241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d2a00640 1 -- 172.21.15.44:0/1094782555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26ab913f0 con 0x55f26af1d790 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d3400640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af24720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66d2a00640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af23ca0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f66c402c150 tx=0x7f66c4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d2626e3d2ef3fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.644+0000 7f66c0c00640 1 -- 172.21.15.44:0/1094782555 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4043070 con 0x55f26af1d790 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.645+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66bc098c80 con 0x55f26af1d790 2024-04-20T11:32:19.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.645+0000 7f66c0c00640 1 -- 172.21.15.44:0/1094782555 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f66c4034640 con 0x55f26af1d790 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.645+0000 7f66c0c00640 1 -- 172.21.15.44:0/1094782555 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f66c404f050 con 0x55f26af1d790 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.645+0000 7f66dc44f740 10 client.28018 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.645+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f26ab510f0 con 0x55f26af1d790 2024-04-20T11:32:19.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.645+0000 7f66c0c00640 1 -- 172.21.15.44:0/1094782555 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc098c80 con 0x55f26af1d790 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66c0c00640 1 -- 172.21.15.44:0/1094782555 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f66c40037b0 con 0x55f26af1d790 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66dc44f740 10 client.28018 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66dc44f740 10 client.28018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66dc44f740 20 client.28018 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66dc44f740 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af36e10 0x55f26af39200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66dc44f740 4 client.28018 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f26ae8cb70 con 0x55f26af36e10 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.646+0000 7f66d2000640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af36e10 0x55f26af39200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.647+0000 7f66d2000640 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af36e10 0x55f26af39200 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e1940f2dfbe329 server_cookie=2c2faa912a103189 in_seq=0 out_seq=0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.647+0000 7f66c0c00640 10 client.28018 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.647+0000 7f66c0c00640 1 -- 172.21.15.44:0/1094782555 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f26ae8cb70 con 0x55f26af36e10 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66c0c00640 10 client.28018 handle_command_reply: tid=0 2024-04-20T11:32:19.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 client.28018 shutdown 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 msgr2=0x55f26af23ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af23ca0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f66c402c150 tx=0x7f66c4004040 comp rx=0 tx=0).stop 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 shutdown_connections 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af36e10 0x55f26af39200 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af24720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1e0b0 0x55f26af241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 --2- 172.21.15.44:0/1094782555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af23ca0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66c0c00640 0 client.28018 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 >> 172.21.15.44:0/1094782555 conn(0x55f26af17ec0 msgr2=0x55f26af19c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 shutdown_connections 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 1 -- 172.21.15.44:0/1094782555 wait complete. 2024-04-20T11:32:19.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.648+0000 7f66dc44f740 20 client.28018 trim_cache size 0 max 16384 2024-04-20T11:32:19.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.667+0000 7f66dc44f740 1 Processor -- start 2024-04-20T11:32:19.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 -- start start 2024-04-20T11:32:19.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af12990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66d41ab970 con 0x55f26af26c90 2024-04-20T11:32:19.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66d41ad4b0 con 0x55f26af1e0b0 2024-04-20T11:32:19.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66dc44f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66d41ac040 con 0x55f26af1d790 2024-04-20T11:32:19.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66d3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af12990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66d3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af12990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33856/0 (socket says 172.21.15.44:33856) 2024-04-20T11:32:19.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66d3400640 1 -- 172.21.15.44:0/3539532939 learned_addr learned my addr 172.21.15.44:0/3539532939 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66d2000640 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.668+0000 7f66d2a00640 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d3400640 1 -- 172.21.15.44:0/3539532939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 msgr2=0x55f26af263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d3400640 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af263d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d3400640 1 -- 172.21.15.44:0/3539532939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 msgr2=0x55f26af327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d3400640 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d3400640 1 -- 172.21.15.44:0/3539532939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26af17cf0 con 0x55f26af26c90 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d2000640 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66d3400640 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af12990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0036f0 tx=0x7f66cc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140671af1609bfc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.669+0000 7f66c3400640 1 -- 172.21.15.44:0/3539532939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc02e070 con 0x55f26af26c90 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66c3400640 1 -- 172.21.15.44:0/3539532939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66cc01eb60 con 0x55f26af26c90 2024-04-20T11:32:19.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66c3400640 1 -- 172.21.15.44:0/3539532939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc029d30 con 0x55f26af26c90 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 -- 172.21.15.44:0/3539532939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 msgr2=0x55f26af12990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af12990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0036f0 tx=0x7f66cc01b000 comp rx=0 tx=0).stop 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 -- 172.21.15.44:0/3539532939 shutdown_connections 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af26c90 0x55f26af12990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af1e0b0 0x55f26af327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 --2- 172.21.15.44:0/3539532939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1d790 0x55f26af263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 -- 172.21.15.44:0/3539532939 >> 172.21.15.44:0/3539532939 conn(0x55f26af17ec0 msgr2=0x55f26af12280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 -- 172.21.15.44:0/3539532939 shutdown_connections 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.670+0000 7f66dc44f740 1 -- 172.21.15.44:0/3539532939 wait complete. 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.671+0000 7f66dc44f740 1 -- start start 2024-04-20T11:32:19.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.671+0000 7f66dc44f740 1 Processor -- start 2024-04-20T11:32:19.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.671+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 0x55f26af20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.671+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1e0b0 0x55f26af21010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.671+0000 7f66dc44f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af21550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66dc44f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66d41acde0 con 0x55f26af1d790 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66dc44f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66d41ac710 con 0x55f26af26c90 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66dc44f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66d41a9960 con 0x55f26af1e0b0 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66dc44f740 10 client.0 _ll_register_callbacks cb 0x7f66bc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 0x55f26af20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 0x55f26af20ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33864/0 (socket says 172.21.15.44:33864) 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 -- 172.21.15.44:0/2704532224 learned_addr learned my addr 172.21.15.44:0/2704532224 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2000640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1e0b0 0x55f26af21010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d3400640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af21550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 -- 172.21.15.44:0/2704532224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1e0b0 msgr2=0x55f26af21010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1e0b0 0x55f26af21010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 -- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 msgr2=0x55f26af21550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af21550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 -- 172.21.15.44:0/2704532224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26af9c4f0 con 0x55f26af1d790 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2000640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1e0b0 0x55f26af21010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d3400640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af21550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66d2a00640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 0x55f26af20ad0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f66c4003c00 tx=0x7f66c40096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f119e1f4701f379f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.672+0000 7f66c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66c0c00640 1 -- 172.21.15.44:0/2704532224 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4057070 con 0x55f26af1d790 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66c0c00640 1 -- 172.21.15.44:0/2704532224 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f66c4034440 con 0x55f26af1d790 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66cc01eb60 con 0x55f26af1d790 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66c0c00640 1 -- 172.21.15.44:0/2704532224 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f66c4063050 con 0x55f26af1d790 2024-04-20T11:32:19.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66dc44f740 10 client.17904 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f26af9d7c0 con 0x55f26af1d790 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66c0c00640 1 -- 172.21.15.44:0/2704532224 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66cc01eb60 con 0x55f26af1d790 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.673+0000 7f66c0c00640 1 -- 172.21.15.44:0/2704532224 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f66c402c7b0 con 0x55f26af1d790 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66dc44f740 10 client.17904 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66dc44f740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66dc44f740 20 client.17904 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66dc44f740 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af9de50 0x55f26afa0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66dc44f740 4 client.17904 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f26ae869e0 con 0x55f26af9de50 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66d2000640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af9de50 0x55f26afa0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66d2000640 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af9de50 0x55f26afa0270 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c05170d345cca2 server_cookie=8ac2d57c4f9283b8 in_seq=0 out_seq=0 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.674+0000 7f66c0c00640 10 client.17904 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66c0c00640 1 -- 172.21.15.44:0/2704532224 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f26ae869e0 con 0x55f26af9de50 2024-04-20T11:32:19.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66c0c00640 10 client.17904 handle_command_reply: tid=0 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 client.17904 shutdown 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 msgr2=0x55f26af20ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 0x55f26af20ad0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f66c4003c00 tx=0x7f66c40096b0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 shutdown_connections 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55f26af9de50 0x55f26afa0270 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f26af26c90 0x55f26af21550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f26af1e0b0 0x55f26af21010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 --2- 172.21.15.44:0/2704532224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f26af1d790 0x55f26af20ad0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66c0c00640 0 client.17904 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 >> 172.21.15.44:0/2704532224 conn(0x55f26af17ec0 msgr2=0x55f26af12b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 shutdown_connections 2024-04-20T11:32:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 1 -- 172.21.15.44:0/2704532224 wait complete. 2024-04-20T11:32:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.675+0000 7f66dc44f740 20 client.17904 trim_cache size 0 max 16384 2024-04-20T11:32:19.681 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:32:19.681 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:32:19.681 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:32:19.681 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:32:19.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f66b40a3d20 msgr2=0x7f66b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f66b40a3d20 0x7f66b40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009530 tx=0x7f66cc009260 comp rx=0 tx=0).stop 2024-04-20T11:32:19.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 msgr2=0x7f66d419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 0x7f66d419e960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00eb10 tx=0x7f66bc00c2d0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 shutdown_connections 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d410c3e0 0x7f66d4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f66b40a3d20 0x7f66b40a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d41038e0 0x7f66d419e960 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 --2- 172.21.15.44:0/2188218203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102fc0 0x7f66d419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 >> 172.21.15.44:0/2188218203 conn(0x7f66d40fc7e0 msgr2=0x7f66d40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 shutdown_connections 2024-04-20T11:32:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.676+0000 7f66d3e00640 1 -- 172.21.15.44:0/2188218203 wait complete. 2024-04-20T11:32:19.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:32:19.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:19.762 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:19 smithi044 ceph-mon[16855]: pgmap v302: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.901+0000 7fe343200640 1 -- 172.21.15.44:0/1775507223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074300 msgr2=0x7fe3440746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.901+0000 7fe343200640 1 --2- 172.21.15.44:0/1775507223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074300 0x7fe3440746e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe334009ca0 tx=0x7fe33402c180 comp rx=0 tx=0).stop 2024-04-20T11:32:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 -- 172.21.15.44:0/1775507223 shutdown_connections 2024-04-20T11:32:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 --2- 172.21.15.44:0/1775507223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344112d90 0x7fe344115180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 --2- 172.21.15.44:0/1775507223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe344074c20 0x7fe344112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 --2- 172.21.15.44:0/1775507223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074300 0x7fe3440746e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 -- 172.21.15.44:0/1775507223 >> 172.21.15.44:0/1775507223 conn(0x7fe3440fd890 msgr2=0x7fe3440ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 -- 172.21.15.44:0/1775507223 shutdown_connections 2024-04-20T11:32:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.902+0000 7fe343200640 1 -- 172.21.15.44:0/1775507223 wait complete. 2024-04-20T11:32:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 Processor -- start 2024-04-20T11:32:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 -- start start 2024-04-20T11:32:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344074300 0x7fe34419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 0x7fe34419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe344112d90 0x7fe3441a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441188b0 con 0x7fe344074c20 2024-04-20T11:32:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe344118730 con 0x7fe344112d90 2024-04-20T11:32:19.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe343200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe344118a30 con 0x7fe344074300 2024-04-20T11:32:19.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe341400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 0x7fe34419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe341400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 0x7fe34419f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33884/0 (socket says 172.21.15.44:33884) 2024-04-20T11:32:19.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe341400640 1 -- 172.21.15.44:0/4027369169 learned_addr learned my addr 172.21.15.44:0/4027369169 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:19.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.903+0000 7fe341e00640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344074300 0x7fe34419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe342800640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe344112d90 0x7fe3441a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341e00640 1 -- 172.21.15.44:0/4027369169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe344112d90 msgr2=0x7fe3441a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341e00640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe344112d90 0x7fe3441a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341e00640 1 -- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 msgr2=0x7fe34419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:19.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341e00640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 0x7fe34419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:19.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341e00640 1 -- 172.21.15.44:0/4027369169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441a3d20 con 0x7fe344074300 2024-04-20T11:32:19.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341400640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 0x7fe34419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.904+0000 7fe341e00640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344074300 0x7fe34419ed30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe334006010 tx=0x7fe334034760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.905+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334039040 con 0x7fe344074300 2024-04-20T11:32:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.905+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3441a3fb0 con 0x7fe344074300 2024-04-20T11:32:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.905+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe334032720 con 0x7fe344074300 2024-04-20T11:32:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.905+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340043c0 con 0x7fe344074300 2024-04-20T11:32:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.905+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441ab8a0 con 0x7fe344074300 2024-04-20T11:32:19.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.906+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe344068ca0 con 0x7fe344074300 2024-04-20T11:32:19.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.908+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe334046030 con 0x7fe344074300 2024-04-20T11:32:19.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.910+0000 7fe33aa00640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3280a3da0 0x7fe3280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:19.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.910+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fe3340f21b0 con 0x7fe344074300 2024-04-20T11:32:19.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.910+0000 7fe341400640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3280a3da0 0x7fe3280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:19.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.911+0000 7fe341400640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3280a3da0 0x7fe3280a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe344105b60 tx=0x7fe32c00b4a0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:19.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:19.912+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe33404d060 con 0x7fe344074300 2024-04-20T11:32:19.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:19 smithi165 ceph-mon[21975]: pgmap v302: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:20.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.164+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe344105200 con 0x7fe344074300 2024-04-20T11:32:20.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.166+0000 7fe33aa00640 1 -- 172.21.15.44:0/4027369169 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fe3340be9d0 con 0x7fe344074300 2024-04-20T11:32:20.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:32:20.166 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:32:20.167 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:32:20.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3280a3da0 msgr2=0x7fe3280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3280a3da0 0x7fe3280a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe344105b60 tx=0x7fe32c00b4a0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344074300 msgr2=0x7fe34419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344074300 0x7fe34419ed30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe334006010 tx=0x7fe334034760 comp rx=0 tx=0).stop 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 shutdown_connections 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe344112d90 0x7fe3441a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3280a3da0 0x7fe3280a6260 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe344074c20 0x7fe34419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 --2- 172.21.15.44:0/4027369169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344074300 0x7fe34419ed30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.168+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 >> 172.21.15.44:0/4027369169 conn(0x7fe3440fd890 msgr2=0x7fe3440fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.169+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 shutdown_connections 2024-04-20T11:32:20.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.169+0000 7fe343200640 1 -- 172.21.15.44:0/4027369169 wait complete. 2024-04-20T11:32:20.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:19 smithi078 ceph-mon[21464]: pgmap v302: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:20.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 -- 172.21.15.44:0/3185352531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d810c480 msgr2=0x7f68d810e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 --2- 172.21.15.44:0/3185352531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d810c480 0x7f68d810e8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009ca0 tx=0x7f68d002c180 comp rx=0 tx=0).stop 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 -- 172.21.15.44:0/3185352531 shutdown_connections 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 --2- 172.21.15.44:0/3185352531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d810c480 0x7f68d810e8f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 --2- 172.21.15.44:0/3185352531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d81039d0 0x7f68d810beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 --2- 172.21.15.44:0/3185352531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81030b0 0x7f68d8103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 -- 172.21.15.44:0/3185352531 >> 172.21.15.44:0/3185352531 conn(0x7f68d80fc850 msgr2=0x7f68d80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 -- 172.21.15.44:0/3185352531 shutdown_connections 2024-04-20T11:32:20.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.406+0000 7f68dda00640 1 -- 172.21.15.44:0/3185352531 wait complete. 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.407+0000 7f68dda00640 1 Processor -- start 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.407+0000 7f68dda00640 1 -- start start 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.407+0000 7f68dda00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d81030b0 0x7f68d8100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 0x7f68d8101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dda00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d810c480 0x7f68d8101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dda00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8112370 con 0x7f68d81039d0 2024-04-20T11:32:20.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dda00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81121f0 con 0x7f68d81030b0 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dda00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81124f0 con 0x7f68d810c480 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 0x7f68d8101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 0x7f68d8101240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33902/0 (socket says 172.21.15.44:33902) 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68d7400640 1 -- 172.21.15.44:0/481473576 learned_addr learned my addr 172.21.15.44:0/481473576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dd000640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d810c480 0x7f68d8101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68d7e00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d81030b0 0x7f68d8100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dd000640 1 -- 172.21.15.44:0/481473576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d81030b0 msgr2=0x7f68d8100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dd000640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d81030b0 0x7f68d8100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dd000640 1 -- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 msgr2=0x7f68d8101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dd000640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 0x7f68d8101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68dd000640 1 -- 172.21.15.44:0/481473576 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d81a6c90 con 0x7f68d810c480 2024-04-20T11:32:20.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.408+0000 7f68d7400640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 0x7f68d8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.409+0000 7f68dd000640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d810c480 0x7f68d8101780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f68d002c790 tx=0x7f68d00346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.409+0000 7f68d4c00640 1 -- 172.21.15.44:0/481473576 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0039040 con 0x7f68d810c480 2024-04-20T11:32:20.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.409+0000 7f68d4c00640 1 -- 172.21.15.44:0/481473576 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f68d0044070 con 0x7f68d810c480 2024-04-20T11:32:20.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.409+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d81a6f20 con 0x7f68d810c480 2024-04-20T11:32:20.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.409+0000 7f68d4c00640 1 -- 172.21.15.44:0/481473576 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004330 con 0x7f68d810c480 2024-04-20T11:32:20.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.409+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81a73e0 con 0x7f68d810c480 2024-04-20T11:32:20.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.412+0000 7f68e1dce740 1 Processor -- start 2024-04-20T11:32:20.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68d4c00640 1 -- 172.21.15.44:0/481473576 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f68d00044d0 con 0x7f68d810c480 2024-04-20T11:32:20.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 -- start start 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558eefffe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 0x558ef0007130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ef0007670 0x558ef0009a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558eeff63050 con 0x558ef0007670 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558eeff09490 con 0x558eefffea90 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.413+0000 7f68e1dce740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558eeff5f010 con 0x558eefffe170 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 0x558ef0007130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ef0007670 0x558ef0009a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 0x558ef0007130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55536/0 (socket says 172.21.15.44:55536) 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 -- 172.21.15.44:0/946534731 learned_addr learned my addr 172.21.15.44:0/946534731 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7400640 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558eefffe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 -- 172.21.15.44:0/946534731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 msgr2=0x558eefffe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558eefffe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 -- 172.21.15.44:0/946534731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ef0007670 msgr2=0x558ef0009a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ef0007670 0x558ef0009a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d7e00640 1 -- 172.21.15.44:0/946534731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eeff11070 con 0x558eefffea90 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.414+0000 7f68d4c00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f68bc0a3d50 0x7f68bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.415+0000 7f68d4c00640 1 -- 172.21.15.44:0/481473576 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f68d00f2ae0 con 0x7f68d810c480 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.415+0000 7f68d7e00640 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 0x558ef0007130 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009cf0 tx=0x7f68c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645ed17d089d3455 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.415+0000 7f68d7400640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f68bc0a3d50 0x7f68bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.415+0000 7f68cd600640 1 -- 172.21.15.44:0/946534731 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8004230 con 0x558eefffea90 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.415+0000 7f68cd600640 1 -- 172.21.15.44:0/946534731 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f68c80043d0 con 0x558eefffea90 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.415+0000 7f68cd600640 1 -- 172.21.15.44:0/946534731 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8019900 con 0x558eefffea90 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 -- 172.21.15.44:0/946534731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 msgr2=0x558ef0007130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 0x558ef0007130 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009cf0 tx=0x7f68c8011810 comp rx=0 tx=0).stop 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68d7400640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f68bc0a3d50 0x7f68bc0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x558eefff9a30 tx=0x7f68c4009530 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 -- 172.21.15.44:0/946534731 shutdown_connections 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558ef0007670 0x558ef0009a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffea90 0x558ef0007130 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 --2- 172.21.15.44:0/946534731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558eefffe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 -- 172.21.15.44:0/946534731 >> 172.21.15.44:0/946534731 conn(0x558eefff88a0 msgr2=0x558eefff8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 -- 172.21.15.44:0/946534731 shutdown_connections 2024-04-20T11:32:20.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 -- 172.21.15.44:0/946534731 wait complete. 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.416+0000 7f68e1dce740 1 -- start start 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 Processor -- start 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558ef0004670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0004bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef00050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558ef000a460 con 0x558eefffea90 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558eeff1c860 con 0x558ef0007670 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ef000a5e0 con 0x558eefffe170 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68e1dce740 10 client.0 _ll_register_callbacks cb 0x558eefeeafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.417+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0004bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558ef0004670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef00050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0004bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33916/0 (socket says 172.21.15.44:33916) 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68d7400640 1 -- 172.21.15.44:0/4026174200 learned_addr learned my addr 172.21.15.44:0/4026174200 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 -- 172.21.15.44:0/4026174200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 msgr2=0x558ef0004670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558ef0004670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 -- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 msgr2=0x558ef0004bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0004bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 -- 172.21.15.44:0/4026174200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eeff20950 con 0x558ef0007670 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68d7400640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0004bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68d7e00640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558ef0004670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68dd000640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef00050f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f68d00f0320 tx=0x7f68d00be940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fe67c1aca56b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68c2000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.418+0000 7f68c2000640 1 -- 172.21.15.44:0/4026174200 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00bf270 con 0x558ef0007670 2024-04-20T11:32:20.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68c2000640 1 -- 172.21.15.44:0/4026174200 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f68d00df070 con 0x558ef0007670 2024-04-20T11:32:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c80043d0 con 0x558ef0007670 2024-04-20T11:32:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68c2000640 1 -- 172.21.15.44:0/4026174200 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f68d00beb70 con 0x558ef0007670 2024-04-20T11:32:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 10 client.28037 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558eefeee820 con 0x558ef0007670 2024-04-20T11:32:20.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68c2000640 1 -- 172.21.15.44:0/4026174200 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f68c80043d0 con 0x558ef0007670 2024-04-20T11:32:20.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68c2000640 1 -- 172.21.15.44:0/4026174200 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f68d00d9080 con 0x558ef0007670 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 10 client.28037 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 10 client.28037 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 20 client.28037 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef0017610 0x558ef0019a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 4 client.28037 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.419+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eeff70a70 con 0x558ef0017610 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.420+0000 7f68d7e00640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef0017610 0x558ef0019a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.420+0000 7f68d7e00640 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef0017610 0x558ef0019a70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9715c695d122287 server_cookie=128a6e2ab263ec74 in_seq=0 out_seq=0 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.420+0000 7f68c2000640 10 client.28037 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68c2000640 1 -- 172.21.15.44:0/4026174200 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558eeff70a70 con 0x558ef0017610 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68c2000640 10 client.28037 handle_command_reply: tid=0 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 client.28037 shutdown 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 msgr2=0x558ef00050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef00050f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f68d00f0320 tx=0x7f68d00be940 comp rx=0 tx=0).stop 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 shutdown_connections 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef0017610 0x558ef0019a70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef00050f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0004bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68e1dce740 1 --2- 172.21.15.44:0/4026174200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffe170 0x558ef0004670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.421+0000 7f68c2000640 0 client.28037 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.422+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 >> 172.21.15.44:0/4026174200 conn(0x558eefff88a0 msgr2=0x558eefffa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.422+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 shutdown_connections 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.422+0000 7f68e1dce740 1 -- 172.21.15.44:0/4026174200 wait complete. 2024-04-20T11:32:20.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.422+0000 7f68e1dce740 20 client.28037 trim_cache size 0 max 16384 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 Processor -- start 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 -- start start 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 0x558ef0006da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0013320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ef0007670 0x558ef001f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81afb10 con 0x558eefffea90 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81b1650 con 0x558eefffe170 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68e1dce740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81b01e0 con 0x558ef0007670 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 0x558ef0006da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0013320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 0x558ef0006da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55550/0 (socket says 172.21.15.44:55550) 2024-04-20T11:32:20.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.437+0000 7f68d7e00640 1 -- 172.21.15.44:0/2347456261 learned_addr learned my addr 172.21.15.44:0/2347456261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68dd000640 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ef0007670 0x558ef001f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68d7e00640 1 -- 172.21.15.44:0/2347456261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ef0007670 msgr2=0x558ef001f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68d7e00640 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ef0007670 0x558ef001f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68d7e00640 1 -- 172.21.15.44:0/2347456261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 msgr2=0x558ef0013320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68d7e00640 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0013320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68d7e00640 1 -- 172.21.15.44:0/2347456261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eefffe8c0 con 0x558eefffe170 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68d7e00640 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 0x558ef0006da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x558eefffdc40 tx=0x7f68c80194c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2f58065fbb23f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68ccc00640 1 -- 172.21.15.44:0/2347456261 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c80097a0 con 0x558eefffe170 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68ccc00640 1 -- 172.21.15.44:0/2347456261 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f68c8031070 con 0x558eefffe170 2024-04-20T11:32:20.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.438+0000 7f68ccc00640 1 -- 172.21.15.44:0/2347456261 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c802b630 con 0x558eefffe170 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 -- 172.21.15.44:0/2347456261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 msgr2=0x558ef0006da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 0x558ef0006da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x558eefffdc40 tx=0x7f68c80194c0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 -- 172.21.15.44:0/2347456261 shutdown_connections 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ef0007670 0x558ef001f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffea90 0x558ef0013320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 --2- 172.21.15.44:0/2347456261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558eefffe170 0x558ef0006da0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 -- 172.21.15.44:0/2347456261 >> 172.21.15.44:0/2347456261 conn(0x558eefff88a0 msgr2=0x558eefffee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 -- 172.21.15.44:0/2347456261 shutdown_connections 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.439+0000 7f68e1dce740 1 -- 172.21.15.44:0/2347456261 wait complete. 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 -- start start 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 Processor -- start 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 0x558ef000f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffea90 0x558ef000fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef000aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81b0f80 con 0x558eefffe170 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81b08b0 con 0x558ef0007670 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81adb00 con 0x558eefffea90 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.440+0000 7f68e1dce740 10 client.0 _ll_register_callbacks cb 0x7f68d00bae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 0x558ef000f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffea90 0x558ef000fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 0x558ef000f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33922/0 (socket says 172.21.15.44:33922) 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 -- 172.21.15.44:0/3026302085 learned_addr learned my addr 172.21.15.44:0/3026302085 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffea90 0x558ef000fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48104/0 (socket says 172.21.15.44:48104) 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68dd000640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef000aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 -- 172.21.15.44:0/3026302085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffea90 msgr2=0x558ef000fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffea90 0x558ef000fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 -- 172.21.15.44:0/3026302085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 msgr2=0x558ef000aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef000aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 -- 172.21.15.44:0/3026302085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eefc753f0 con 0x558eefffe170 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68dd000640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef000aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68d7e00640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 0x558ef000f880 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009cc0 tx=0x7f68c80053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04899171b56ce41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.441+0000 7f68c2000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68c2000640 1 -- 172.21.15.44:0/3026302085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c80035f0 con 0x558eefffe170 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68c2000640 1 -- 172.21.15.44:0/3026302085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f68c802a040 con 0x558eefffe170 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c8031070 con 0x558eefffe170 2024-04-20T11:32:20.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68c2000640 1 -- 172.21.15.44:0/3026302085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f68c8003790 con 0x558eefffe170 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68e1dce740 10 client.17928 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558eefd3f7d0 con 0x558eefffe170 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68c2000640 1 -- 172.21.15.44:0/3026302085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f68c8031070 con 0x558eefffe170 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.442+0000 7f68c2000640 1 -- 172.21.15.44:0/3026302085 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f68c8019cc0 con 0x558eefffe170 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68e1dce740 10 client.17928 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68e1dce740 10 client.17928 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68e1dce740 20 client.17928 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68e1dce740 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef007db40 0x558ef007ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68e1dce740 4 client.17928 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558eeff6a8e0 con 0x558ef007db40 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68d7400640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef007db40 0x558ef007ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68d7400640 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef007db40 0x558ef007ff60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8b25bb6a880f923 server_cookie=a81bb401452144ea in_seq=0 out_seq=0 2024-04-20T11:32:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.443+0000 7f68c2000640 10 client.17928 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68c2000640 1 -- 172.21.15.44:0/3026302085 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558eeff6a8e0 con 0x558ef007db40 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68c2000640 10 client.17928 handle_command_reply: tid=0 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 client.17928 shutdown 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 msgr2=0x558ef000f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 0x558ef000f880 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009cc0 tx=0x7f68c80053a0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 shutdown_connections 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x558ef007db40 0x558ef007ff60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ef0007670 0x558ef000aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558eefffea90 0x558ef000fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68c2000640 0 client.17928 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 --2- 172.21.15.44:0/3026302085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558eefffe170 0x558ef000f880 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 >> 172.21.15.44:0/3026302085 conn(0x558eefff88a0 msgr2=0x558ef0004610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 shutdown_connections 2024-04-20T11:32:20.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.444+0000 7f68e1dce740 1 -- 172.21.15.44:0/3026302085 wait complete. 2024-04-20T11:32:20.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68e1dce740 20 client.17928 trim_cache size 0 max 16384 2024-04-20T11:32:20.451 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f68bc0a3d50 msgr2=0x7f68bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f68bc0a3d50 0x7f68bc0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x558eefff9a30 tx=0x7f68c4009530 comp rx=0 tx=0).stop 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d810c480 msgr2=0x7f68d8101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d810c480 0x7f68d8101780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f68d002c790 tx=0x7f68d00346f0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 shutdown_connections 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d810c480 0x7f68d8101780 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f68bc0a3d50 0x7f68bc0a6210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d81039d0 0x7f68d8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 --2- 172.21.15.44:0/481473576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68d81030b0 0x7f68d8100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 >> 172.21.15.44:0/481473576 conn(0x7f68d80fc850 msgr2=0x7f68d80fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 shutdown_connections 2024-04-20T11:32:20.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.445+0000 7f68dda00640 1 -- 172.21.15.44:0/481473576 wait complete. 2024-04-20T11:32:20.463 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:32:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 -- 172.21.15.44:0/2982558495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420108500 msgr2=0x7f8420108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 --2- 172.21.15.44:0/2982558495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420108500 0x7f8420108960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8408009860 tx=0x7f840802bc00 comp rx=0 tx=0).stop 2024-04-20T11:32:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 -- 172.21.15.44:0/2982558495 shutdown_connections 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 --2- 172.21.15.44:0/2982558495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8420108ea0 0x7f84201152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 --2- 172.21.15.44:0/2982558495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420108500 0x7f8420108960 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 --2- 172.21.15.44:0/2982558495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f842010d3d0 0x7f842010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 -- 172.21.15.44:0/2982558495 >> 172.21.15.44:0/2982558495 conn(0x7f8420077d80 msgr2=0x7f8420106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 -- 172.21.15.44:0/2982558495 shutdown_connections 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.668+0000 7f841f200640 1 -- 172.21.15.44:0/2982558495 wait complete. 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.669+0000 7f841f200640 1 Processor -- start 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 -- start start 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8420108500 0x7f842019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 0x7f842019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842010d3d0 0x7f84201a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8420118950 con 0x7f8420108ea0 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84201187d0 con 0x7f842010d3d0 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8420118ad0 con 0x7f8420108500 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 0x7f842019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8420108500 0x7f842019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 0x7f842019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33936/0 (socket says 172.21.15.44:33936) 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841d400640 1 -- 172.21.15.44:0/1739115088 learned_addr learned my addr 172.21.15.44:0/1739115088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.670+0000 7f841e800640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842010d3d0 0x7f84201a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841d400640 1 -- 172.21.15.44:0/1739115088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8420108500 msgr2=0x7f842019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841d400640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8420108500 0x7f842019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841d400640 1 -- 172.21.15.44:0/1739115088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842010d3d0 msgr2=0x7f84201a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841d400640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842010d3d0 0x7f84201a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841d400640 1 -- 172.21.15.44:0/1739115088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84201a3e10 con 0x7f8420108ea0 2024-04-20T11:32:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841d400640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 0x7f842019f3b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8408037040 tx=0x7f840802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f8416a00640 1 -- 172.21.15.44:0/1739115088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408041020 con 0x7f8420108ea0 2024-04-20T11:32:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f8416a00640 1 -- 172.21.15.44:0/1739115088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f840802ce50 con 0x7f8420108ea0 2024-04-20T11:32:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a40a0 con 0x7f8420108ea0 2024-04-20T11:32:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.671+0000 7f8416a00640 1 -- 172.21.15.44:0/1739115088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408039040 con 0x7f8420108ea0 2024-04-20T11:32:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.672+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201ab990 con 0x7f8420108ea0 2024-04-20T11:32:20.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.675+0000 7f8416a00640 1 -- 172.21.15.44:0/1739115088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8408044020 con 0x7f8420108ea0 2024-04-20T11:32:20.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.675+0000 7f8427bd4740 1 Processor -- start 2024-04-20T11:32:20.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.675+0000 7f8427bd4740 1 -- start start 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c183392550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 0x55c18339b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c18339da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f8427bd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c1832f7050 con 0x55c183392a90 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f8427bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c18329d490 con 0x55c183392170 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f8427bd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c1832f3010 con 0x55c18339b670 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f841de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 0x55c18339b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f841e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c183392550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f841de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 0x55c18339b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33944/0 (socket says 172.21.15.44:33944) 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f841de00640 1 -- 172.21.15.44:0/346795488 learned_addr learned my addr 172.21.15.44:0/346795488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.676+0000 7f841d400640 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c18339da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841de00640 1 -- 172.21.15.44:0/346795488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 msgr2=0x55c18339da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841de00640 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c18339da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841de00640 1 -- 172.21.15.44:0/346795488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 msgr2=0x55c183392550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841de00640 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c183392550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841de00640 1 -- 172.21.15.44:0/346795488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1832a5070 con 0x55c183392a90 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841d400640 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c18339da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841e800640 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c183392550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841de00640 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 0x55c18339b130 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8410009c50 tx=0x7f8410011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc0debff7be6aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f8416a00640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84040a3d20 0x7f84040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f840f400640 1 -- 172.21.15.44:0/346795488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841001e070 con 0x55c183392a90 2024-04-20T11:32:20.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f840f400640 1 -- 172.21.15.44:0/346795488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8410019b30 con 0x55c183392a90 2024-04-20T11:32:20.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f8416a00640 1 -- 172.21.15.44:0/1739115088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f84080f1cb0 con 0x7f8420108ea0 2024-04-20T11:32:20.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f840f400640 1 -- 172.21.15.44:0/346795488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841000eb10 con 0x55c183392a90 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.677+0000 7f841e800640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84040a3d20 0x7f84040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.678+0000 7f8427bd4740 1 -- 172.21.15.44:0/346795488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 msgr2=0x55c18339b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.678+0000 7f841e800640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84040a3d20 0x7f84040a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8418005fd0 tx=0x7f8418009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.678+0000 7f8427bd4740 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 0x55c18339b130 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8410009c50 tx=0x7f8410011810 comp rx=0 tx=0).stop 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.678+0000 7f8427bd4740 1 -- 172.21.15.44:0/346795488 shutdown_connections 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c18339da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392a90 0x55c18339b130 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 --2- 172.21.15.44:0/346795488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c183392550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 -- 172.21.15.44:0/346795488 >> 172.21.15.44:0/346795488 conn(0x55c18338c8a0 msgr2=0x55c18338cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 -- 172.21.15.44:0/346795488 shutdown_connections 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 -- 172.21.15.44:0/346795488 wait complete. 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 -- start start 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.679+0000 7f8427bd4740 1 Processor -- start 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183398c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183399150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c18339e460 con 0x55c18339b670 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1832b0860 con 0x55c183392170 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c18339e5e0 con 0x55c183392a90 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f8427bd4740 10 client.0 _ll_register_callbacks cb 0x55c18327efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f841e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183399150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f841e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183399150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33948/0 (socket says 172.21.15.44:33948) 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f841e800640 1 -- 172.21.15.44:0/49887363 learned_addr learned my addr 172.21.15.44:0/49887363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f841d400640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183398c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.680+0000 7f841de00640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841e800640 1 -- 172.21.15.44:0/49887363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 msgr2=0x55c183398c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841e800640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183398c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841e800640 1 -- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 msgr2=0x55c1833986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841e800640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841e800640 1 -- 172.21.15.44:0/49887363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1832b4950 con 0x55c18339b670 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841de00640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f841e800640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183399150 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8418011590 tx=0x7f8418004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20388e6a413a0cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f8403e00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f8403e00640 1 -- 172.21.15.44:0/49887363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418011e70 con 0x55c18339b670 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f8403e00640 1 -- 172.21.15.44:0/49887363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f841801f9b0 con 0x55c18339b670 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.681+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8410019b30 con 0x55c18339b670 2024-04-20T11:32:20.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8403e00640 1 -- 172.21.15.44:0/49887363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f841801fbb0 con 0x55c18339b670 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 10 client.17940 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c183282820 con 0x55c18339b670 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8403e00640 1 -- 172.21.15.44:0/49887363 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8410019b30 con 0x55c18339b670 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8403e00640 1 -- 172.21.15.44:0/49887363 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f8418026070 con 0x55c18339b670 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 10 client.17940 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 10 client.17940 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 20 client.17940 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c1833ab770 0x55c1833adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 4 client.17940 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.682+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c183304a70 con 0x55c1833ab770 2024-04-20T11:32:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.683+0000 7f841de00640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c1833ab770 0x55c1833adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.683+0000 7f841de00640 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c1833ab770 0x55c1833adba0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6077a3e91b79967 server_cookie=8a83ffbb1606f6ae in_seq=0 out_seq=0 2024-04-20T11:32:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.683+0000 7f8403e00640 10 client.17940 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.684+0000 7f8403e00640 1 -- 172.21.15.44:0/49887363 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c183304a70 con 0x55c1833ab770 2024-04-20T11:32:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.684+0000 7f8403e00640 10 client.17940 handle_command_reply: tid=0 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.684+0000 7f8427bd4740 1 client.17940 shutdown 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.684+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 msgr2=0x55c183399150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.684+0000 7f8427bd4740 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183399150 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8418011590 tx=0x7f8418004ab0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 shutdown_connections 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c1833ab770 0x55c1833adba0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183399150 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183398c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 --2- 172.21.15.44:0/49887363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8403e00640 0 client.17940 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:20.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 >> 172.21.15.44:0/49887363 conn(0x55c18338c8a0 msgr2=0x55c18338e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 shutdown_connections 2024-04-20T11:32:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 1 -- 172.21.15.44:0/49887363 wait complete. 2024-04-20T11:32:20.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.685+0000 7f8427bd4740 20 client.17940 trim_cache size 0 max 16384 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.700+0000 7f8427bd4740 1 Processor -- start 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.700+0000 7f8427bd4740 1 -- start start 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392170 0x55c18339ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 0x55c1833a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c1833b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f8427bd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b4170 con 0x55c183392170 2024-04-20T11:32:20.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f8427bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b5cb0 con 0x55c183392a90 2024-04-20T11:32:20.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f8427bd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b4840 con 0x55c18339b670 2024-04-20T11:32:20.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 0x55c1833a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392170 0x55c18339ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 0x55c1833a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55608/0 (socket says 172.21.15.44:55608) 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 -- 172.21.15.44:0/3037832677 learned_addr learned my addr 172.21.15.44:0/3037832677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841e800640 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c1833b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 -- 172.21.15.44:0/3037832677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 msgr2=0x55c1833b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c1833b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 -- 172.21.15.44:0/3037832677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392170 msgr2=0x55c18339ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.701+0000 7f841d400640 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392170 0x55c18339ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.702+0000 7f841d400640 1 -- 172.21.15.44:0/3037832677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1830093f0 con 0x55c183392a90 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.702+0000 7f841de00640 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392170 0x55c18339ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.702+0000 7f841d400640 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 0x55c1833a7350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f84080bdcd0 tx=0x7f8408005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914dffc46a6fdce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.703+0000 7f840ea00640 1 -- 172.21.15.44:0/3037832677 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080bd880 con 0x55c183392a90 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.703+0000 7f840ea00640 1 -- 172.21.15.44:0/3037832677 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84080349d0 con 0x55c183392a90 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.703+0000 7f840ea00640 1 -- 172.21.15.44:0/3037832677 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080bd6a0 con 0x55c183392a90 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 -- 172.21.15.44:0/3037832677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 msgr2=0x55c1833a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 0x55c1833a7350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f84080bdcd0 tx=0x7f8408005f90 comp rx=0 tx=0).stop 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 -- 172.21.15.44:0/3037832677 shutdown_connections 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c18339b670 0x55c1833b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392a90 0x55c1833a7350 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 --2- 172.21.15.44:0/3037832677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c183392170 0x55c18339ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.704+0000 7f8427bd4740 1 -- 172.21.15.44:0/3037832677 >> 172.21.15.44:0/3037832677 conn(0x55c18338c8a0 msgr2=0x55c183392e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.705+0000 7f8427bd4740 1 -- 172.21.15.44:0/3037832677 shutdown_connections 2024-04-20T11:32:20.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.705+0000 7f8427bd4740 1 -- 172.21.15.44:0/3037832677 wait complete. 2024-04-20T11:32:20.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.705+0000 7f8427bd4740 1 -- start start 2024-04-20T11:32:20.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.706+0000 7f8427bd4740 1 Processor -- start 2024-04-20T11:32:20.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.706+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.706+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183394f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.706+0000 7f8427bd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183395440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f8427bd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b55e0 con 0x55c18339b670 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f8427bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b4f10 con 0x55c183392170 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f8427bd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84201b2160 con 0x55c183392a90 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55616/0 (socket says 172.21.15.44:55616) 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 -- 172.21.15.44:0/2888592007 learned_addr learned my addr 172.21.15.44:0/2888592007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841d400640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183394f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f8427bd4740 10 client.0 _ll_register_callbacks cb 0x7f84180114a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841e800640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183395440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 -- 172.21.15.44:0/2888592007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 msgr2=0x55c183394f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183394f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 -- 172.21.15.44:0/2888592007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 msgr2=0x55c183395440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183395440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 -- 172.21.15.44:0/2888592007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1830d37d0 con 0x55c183392170 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841e800640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183395440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.707+0000 7f841de00640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833a3a50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8410014040 tx=0x7f8410009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4c0a09ed1eb49a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.708+0000 7f8403e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.708+0000 7f8403e00640 1 -- 172.21.15.44:0/2888592007 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841001e040 con 0x55c183392170 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.708+0000 7f8403e00640 1 -- 172.21.15.44:0/2888592007 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f841002b3c0 con 0x55c183392170 2024-04-20T11:32:20.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.708+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c1832fe8e0 con 0x55c183392170 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.709+0000 7f8403e00640 1 -- 172.21.15.44:0/2888592007 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f841002bbc0 con 0x55c183392170 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.709+0000 7f8427bd4740 10 client.28061 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.709+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c183411780 con 0x55c183392170 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.709+0000 7f8403e00640 1 -- 172.21.15.44:0/2888592007 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55c1832fe8e0 con 0x55c183392170 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8403e00640 1 -- 172.21.15.44:0/2888592007 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f8410019740 con 0x55c183392170 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8427bd4740 10 client.28061 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8427bd4740 10 client.28061 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:32:20.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8427bd4740 20 client.28061 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8427bd4740 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c183411d00 0x55c1834140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8427bd4740 4 client.28061 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c18327a5a0 con 0x55c183411d00 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.710+0000 7f841d400640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c183411d00 0x55c1834140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f841d400640 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c183411d00 0x55c1834140f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef40fc3069d97f12 server_cookie=438f0a0bb97e15b8 in_seq=0 out_seq=0 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8403e00640 10 client.28061 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8403e00640 1 -- 172.21.15.44:0/2888592007 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c18327a5a0 con 0x55c183411d00 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8403e00640 10 client.28061 handle_command_reply: tid=0 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 client.28061 shutdown 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 msgr2=0x55c1833a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833a3a50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8410014040 tx=0x7f8410009620 comp rx=0 tx=0).stop 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 shutdown_connections 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55c183411d00 0x55c1834140f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c18339b670 0x55c183395440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c183392a90 0x55c183394f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 --2- 172.21.15.44:0/2888592007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c183392170 0x55c1833a3a50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 >> 172.21.15.44:0/2888592007 conn(0x55c18338c8a0 msgr2=0x55c183398690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8403e00640 0 client.28061 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 shutdown_connections 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.711+0000 7f8427bd4740 1 -- 172.21.15.44:0/2888592007 wait complete. 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f8427bd4740 20 client.28061 trim_cache size 0 max 16384 2024-04-20T11:32:20.714 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84040a3d20 msgr2=0x7f84040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84040a3d20 0x7f84040a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8418005fd0 tx=0x7f8418009500 comp rx=0 tx=0).stop 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 msgr2=0x7f842019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 0x7f842019f3b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8408037040 tx=0x7f840802c850 comp rx=0 tx=0).stop 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 shutdown_connections 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842010d3d0 0x7f84201a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f84040a3d20 0x7f84040a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420108ea0 0x7f842019f3b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 --2- 172.21.15.44:0/1739115088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8420108500 0x7f842019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.712+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 >> 172.21.15.44:0/1739115088 conn(0x7f8420077d80 msgr2=0x7f84200783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.713+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 shutdown_connections 2024-04-20T11:32:20.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.713+0000 7f841f200640 1 -- 172.21.15.44:0/1739115088 wait complete. 2024-04-20T11:32:20.728 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:20 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4027369169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:32:20.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:32:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 -- 172.21.15.44:0/2268691556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c941043b0 msgr2=0x7f5c94104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 --2- 172.21.15.44:0/2268691556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c941043b0 0x7f5c94104790 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009860 tx=0x7f5c8802bb70 comp rx=0 tx=0).stop 2024-04-20T11:32:20.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 -- 172.21.15.44:0/2268691556 shutdown_connections 2024-04-20T11:32:20.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 --2- 172.21.15.44:0/2268691556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c94109480 0x7f5c94111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 --2- 172.21.15.44:0/2268691556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94104d60 0x7f5c94108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 --2- 172.21.15.44:0/2268691556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c941043b0 0x7f5c94104790 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.941+0000 7f5c9c600640 1 -- 172.21.15.44:0/2268691556 >> 172.21.15.44:0/2268691556 conn(0x7f5c940ffbd0 msgr2=0x7f5c94101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.942+0000 7f5c9c600640 1 -- 172.21.15.44:0/2268691556 shutdown_connections 2024-04-20T11:32:20.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.942+0000 7f5c9c600640 1 -- 172.21.15.44:0/2268691556 wait complete. 2024-04-20T11:32:20.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 Processor -- start 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 -- start start 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c941043b0 0x7f5c941a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 0x7f5c941a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c9419a9d0 0x7f5c9419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94114670 con 0x7f5c94109480 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941144f0 con 0x7f5c9419a9d0 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c9c600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941147f0 con 0x7f5c941043b0 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.943+0000 7f5c99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 0x7f5c941a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 0x7f5c941a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34000/0 (socket says 172.21.15.44:34000) 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 -- 172.21.15.44:0/1859109458 learned_addr learned my addr 172.21.15.44:0/1859109458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c941043b0 0x7f5c941a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c9a600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c9419a9d0 0x7f5c9419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 -- 172.21.15.44:0/1859109458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c941043b0 msgr2=0x7f5c941a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c941043b0 0x7f5c941a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 -- 172.21.15.44:0/1859109458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c9419a9d0 msgr2=0x7f5c9419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c9419a9d0 0x7f5c9419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 -- 172.21.15.44:0/1859109458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9419b4f0 con 0x7f5c94109480 2024-04-20T11:32:20.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99c00640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c941043b0 0x7f5c941a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c9a600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c9419a9d0 0x7f5c9419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:20.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.944+0000 7f5c99200640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 0x7f5c941a0d10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e9f0 tx=0x7f5c8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.945+0000 7f5c92000640 1 -- 172.21.15.44:0/1859109458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000ccc0 con 0x7f5c94109480 2024-04-20T11:32:20.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.945+0000 7f5c92000640 1 -- 172.21.15.44:0/1859109458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c80010930 con 0x7f5c94109480 2024-04-20T11:32:20.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.945+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9419b7e0 con 0x7f5c94109480 2024-04-20T11:32:20.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.945+0000 7f5c92000640 1 -- 172.21.15.44:0/1859109458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800189f0 con 0x7f5c94109480 2024-04-20T11:32:20.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.945+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c941a79e0 con 0x7f5c94109480 2024-04-20T11:32:20.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.948+0000 7f5c9f3d6740 1 Processor -- start 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.948+0000 7f5c92000640 1 -- 172.21.15.44:0/1859109458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5c80010ae0 con 0x7f5c94109480 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.948+0000 7f5c9f3d6740 1 -- start start 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc865780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc8680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564bcc7b9050 con 0x564bcc854770 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564bcc75f490 con 0x564bcc853dc0 2024-04-20T11:32:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564bcc7b5010 con 0x564bcc865cc0 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c99c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc865780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55638/0 (socket says 172.21.15.44:55638) 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9a600640 1 -- 172.21.15.44:0/3642649945 learned_addr learned my addr 172.21.15.44:0/3642649945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c99200640 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc8680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.949+0000 7f5c9a600640 1 -- 172.21.15.44:0/3642649945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 msgr2=0x564bcc8680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c9a600640 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc8680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c9a600640 1 -- 172.21.15.44:0/3642649945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 msgr2=0x564bcc865780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c9a600640 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc865780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c9a600640 1 -- 172.21.15.44:0/3642649945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bcc767070 con 0x564bcc853dc0 2024-04-20T11:32:20.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c9a600640 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8541a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0117c0 tx=0x7f5c8c011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9601a3e0338bf62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c86a00640 1 -- 172.21.15.44:0/3642649945 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c003c30 con 0x564bcc853dc0 2024-04-20T11:32:20.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c86a00640 1 -- 172.21.15.44:0/3642649945 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c8c003dd0 con 0x564bcc853dc0 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.950+0000 7f5c86a00640 1 -- 172.21.15.44:0/3642649945 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c019640 con 0x564bcc853dc0 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c92000640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5c7c0a3d20 0x7f5c7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c92000640 1 -- 172.21.15.44:0/1859109458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f5c800d0a00 con 0x7f5c94109480 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c99c00640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5c7c0a3d20 0x7f5c7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3642649945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 msgr2=0x564bcc8541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8541a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0117c0 tx=0x7f5c8c011c80 comp rx=0 tx=0).stop 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3642649945 shutdown_connections 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc8680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc865780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3642649945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8541a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3642649945 >> 172.21.15.44:0/3642649945 conn(0x564bcc84e6e0 msgr2=0x564bcc84eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3642649945 shutdown_connections 2024-04-20T11:32:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.951+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3642649945 wait complete. 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.952+0000 7f5c9f3d6740 1 -- start start 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.952+0000 7f5c99c00640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5c7c0a3d20 0x7f5c7c0a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009830 tx=0x7f5c8802e040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.952+0000 7f5c9f3d6740 1 Processor -- start 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.952+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc853dc0 0x564bcc862d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc854770 0x564bcc8632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc863800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564bcc868a80 con 0x564bcc865cc0 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564bcc772860 con 0x564bcc854770 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564bcc868c00 con 0x564bcc853dc0 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9f3d6740 10 client.0 _ll_register_callbacks cb 0x564bcc740fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc863800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c99c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc853dc0 0x564bcc862d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc863800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34022/0 (socket says 172.21.15.44:34022) 2024-04-20T11:32:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 -- 172.21.15.44:0/1986555030 learned_addr learned my addr 172.21.15.44:0/1986555030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c99c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc853dc0 0x564bcc862d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48190/0 (socket says 172.21.15.44:48190) 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c99200640 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc854770 0x564bcc8632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 -- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc853dc0 msgr2=0x564bcc862d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc853dc0 0x564bcc862d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 -- 172.21.15.44:0/1986555030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc854770 msgr2=0x564bcc8632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc854770 0x564bcc8632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.953+0000 7f5c9a600640 1 -- 172.21.15.44:0/1986555030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bcc776950 con 0x564bcc865cc0 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c9a600640 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc863800 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c005f40 tx=0x7f5c8c011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c3c1f03ef97af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c7b400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c7b400640 1 -- 172.21.15.44:0/1986555030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004260 con 0x564bcc865cc0 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c7b400640 1 -- 172.21.15.44:0/1986555030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f5c8c004a40 con 0x564bcc865cc0 2024-04-20T11:32:20.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c8c003dd0 con 0x564bcc865cc0 2024-04-20T11:32:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c7b400640 1 -- 172.21.15.44:0/1986555030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5c8c031050 con 0x564bcc865cc0 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c9f3d6740 10 client.17958 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.954+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564bcc744820 con 0x564bcc865cc0 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c7b400640 1 -- 172.21.15.44:0/1986555030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c8c003dd0 con 0x564bcc865cc0 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c7b400640 1 -- 172.21.15.44:0/1986555030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f5c8c018070 con 0x564bcc865cc0 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c9f3d6740 10 client.17958 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c9f3d6740 10 client.17958 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c9f3d6740 20 client.17958 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc875c30 0x564bcc878020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c9f3d6740 4 client.17958 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bcc7c6a70 con 0x564bcc875c30 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.955+0000 7f5c99c00640 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc875c30 0x564bcc878020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.956+0000 7f5c99c00640 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc875c30 0x564bcc878020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=108338be6bf83528 server_cookie=1a0f2fc94dd84921 in_seq=0 out_seq=0 2024-04-20T11:32:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.956+0000 7f5c7b400640 10 client.17958 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c7b400640 1 -- 172.21.15.44:0/1986555030 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bcc7c6a70 con 0x564bcc875c30 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c7b400640 10 client.17958 handle_command_reply: tid=0 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 client.17958 shutdown 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 msgr2=0x564bcc863800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc863800 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c005f40 tx=0x7f5c8c011530 comp rx=0 tx=0).stop 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 shutdown_connections 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc875c30 0x564bcc878020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc863800 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc854770 0x564bcc8632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/1986555030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc853dc0 0x564bcc862d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c7b400640 0 client.17958 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.957+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 >> 172.21.15.44:0/1986555030 conn(0x564bcc84e6e0 msgr2=0x564bcc850430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.958+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 shutdown_connections 2024-04-20T11:32:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.958+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/1986555030 wait complete. 2024-04-20T11:32:20.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.958+0000 7f5c9f3d6740 20 client.17958 trim_cache size 0 max 16384 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 Processor -- start 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 -- start start 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc861ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc871980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941abb50 con 0x564bcc854770 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941ad690 con 0x564bcc853dc0 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941ac220 con 0x564bcc865cc0 2024-04-20T11:32:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc861ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c99200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc861ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34024/0 (socket says 172.21.15.44:34024) 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.973+0000 7f5c99200640 1 -- 172.21.15.44:0/2651743892 learned_addr learned my addr 172.21.15.44:0/2651743892 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99c00640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c9a600640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc871980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99200640 1 -- 172.21.15.44:0/2651743892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 msgr2=0x564bcc871980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99200640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc871980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99200640 1 -- 172.21.15.44:0/2651743892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 msgr2=0x564bcc8654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99200640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99200640 1 -- 172.21.15.44:0/2651743892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bcc854510 con 0x564bcc854770 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c9a600640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc871980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99c00640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c99200640 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc861ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000cf10 tx=0x7f5c800bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ed3397f5f90357 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c86000640 1 -- 172.21.15.44:0/2651743892 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8009ca30 con 0x564bcc854770 2024-04-20T11:32:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c86000640 1 -- 172.21.15.44:0/2651743892 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c800a0d30 con 0x564bcc854770 2024-04-20T11:32:20.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.974+0000 7f5c86000640 1 -- 172.21.15.44:0/2651743892 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8009be80 con 0x564bcc854770 2024-04-20T11:32:20.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/2651743892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 msgr2=0x564bcc861ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc861ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000cf10 tx=0x7f5c800bd040 comp rx=0 tx=0).stop 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/2651743892 shutdown_connections 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc865cc0 0x564bcc871980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc854770 0x564bcc861ff0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/2651743892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc8654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/2651743892 >> 172.21.15.44:0/2651743892 conn(0x564bcc84e6e0 msgr2=0x564bcc84d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/2651743892 shutdown_connections 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.975+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/2651743892 wait complete. 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 -- start start 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 Processor -- start 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc86e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc854770 0x564bcc85f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc85fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941acfc0 con 0x564bcc865cc0 2024-04-20T11:32:20.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941ac8f0 con 0x564bcc853dc0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941a9b40 con 0x564bcc854770 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.976+0000 7f5c9f3d6740 10 client.0 _ll_register_callbacks cb 0x7f5c8c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc85fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc85fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34026/0 (socket says 172.21.15.44:34026) 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 -- 172.21.15.44:0/3493001182 learned_addr learned my addr 172.21.15.44:0/3493001182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c99200640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc854770 0x564bcc85f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c99c00640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc86e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 -- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc854770 msgr2=0x564bcc85f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc854770 0x564bcc85f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 -- 172.21.15.44:0/3493001182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 msgr2=0x564bcc86e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc86e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 -- 172.21.15.44:0/3493001182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bcc4cb3f0 con 0x564bcc865cc0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c9a600640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc85fb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c006010 tx=0x7f5c8c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226cf558350b9a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.977+0000 7f5c7b400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c7b400640 1 -- 172.21.15.44:0/3493001182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c005920 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c7b400640 1 -- 172.21.15.44:0/3493001182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f5c8c004260 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c800a0d30 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c7b400640 1 -- 172.21.15.44:0/3493001182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5c8c004460 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c9f3d6740 10 client.17964 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564bcc5957d0 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.978+0000 7f5c7b400640 1 -- 172.21.15.44:0/3493001182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5c800a0d30 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c7b400640 1 -- 172.21.15.44:0/3493001182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f5c8c018040 con 0x564bcc865cc0 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c9f3d6740 10 client.17964 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c9f3d6740 10 client.17964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c9f3d6740 20 client.17964 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc8dc1b0 0x564bcc8de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c9f3d6740 4 client.17964 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bcc7c08e0 con 0x564bcc8dc1b0 2024-04-20T11:32:20.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.979+0000 7f5c99c00640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc8dc1b0 0x564bcc8de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:20.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:20 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4027369169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:32:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c99c00640 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc8dc1b0 0x564bcc8de5d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60ee91fe99ad1abd server_cookie=481202f60cba0da2 in_seq=0 out_seq=0 2024-04-20T11:32:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c7b400640 10 client.17964 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c7b400640 1 -- 172.21.15.44:0/3493001182 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bcc7c08e0 con 0x564bcc8dc1b0 2024-04-20T11:32:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c7b400640 10 client.17964 handle_command_reply: tid=0 2024-04-20T11:32:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 client.17964 shutdown 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 msgr2=0x564bcc85fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc85fb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c006010 tx=0x7f5c8c009ee0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 shutdown_connections 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x564bcc8dc1b0 0x564bcc8de5d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bcc865cc0 0x564bcc85fb90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c7b400640 0 client.17964 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bcc854770 0x564bcc85f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 --2- 172.21.15.44:0/3493001182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcc853dc0 0x564bcc86e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.980+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 >> 172.21.15.44:0/3493001182 conn(0x564bcc84e6e0 msgr2=0x564bcc862b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 shutdown_connections 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9f3d6740 1 -- 172.21.15.44:0/3493001182 wait complete. 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9f3d6740 20 client.17964 trim_cache size 0 max 16384 2024-04-20T11:32:20.985 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5c7c0a3d20 msgr2=0x7f5c7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9c600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5c7c0a3d20 0x7f5c7c0a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009830 tx=0x7f5c8802e040 comp rx=0 tx=0).stop 2024-04-20T11:32:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 msgr2=0x7f5c941a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.981+0000 7f5c9c600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 0x7f5c941a0d10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e9f0 tx=0x7f5c8000eeb0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 shutdown_connections 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c9419a9d0 0x7f5c9419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5c7c0a3d20 0x7f5c7c0a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c94109480 0x7f5c941a0d10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 --2- 172.21.15.44:0/1859109458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c941043b0 0x7f5c941a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 >> 172.21.15.44:0/1859109458 conn(0x7f5c940ffbd0 msgr2=0x7f5c94101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 shutdown_connections 2024-04-20T11:32:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:20.982+0000 7f5c9c600640 1 -- 172.21.15.44:0/1859109458 wait complete. 2024-04-20T11:32:21.000 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:32:21.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:20 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4027369169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.203+0000 7f09fa600640 1 -- 172.21.15.44:0/2469029361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 msgr2=0x7f09f4112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.203+0000 7f09fa600640 1 --2- 172.21.15.44:0/2469029361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 0x7f09f4112760 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009890 tx=0x7f09e002bb70 comp rx=0 tx=0).stop 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 -- 172.21.15.44:0/2469029361 shutdown_connections 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 --2- 172.21.15.44:0/2469029361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f4115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 --2- 172.21.15.44:0/2469029361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 0x7f09f4112760 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 --2- 172.21.15.44:0/2469029361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09f4076490 0x7f09f4076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 -- 172.21.15.44:0/2469029361 >> 172.21.15.44:0/2469029361 conn(0x7f09f40ffad0 msgr2=0x7f09f4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 -- 172.21.15.44:0/2469029361 shutdown_connections 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.204+0000 7f09fa600640 1 -- 172.21.15.44:0/2469029361 wait complete. 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.205+0000 7f09fa600640 1 Processor -- start 2024-04-20T11:32:21.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.205+0000 7f09fa600640 1 -- start start 2024-04-20T11:32:21.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.205+0000 7f09fa600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09f4076490 0x7f09f419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.205+0000 7f09fa600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 0x7f09f419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.205+0000 7f09fa600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f41a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.205+0000 7f09fa600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4118920 con 0x7f09f4112ca0 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09fa600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41187a0 con 0x7f09f4076db0 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09fa600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4118aa0 con 0x7f09f4076490 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f41a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f41a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34050/0 (socket says 172.21.15.44:34050) 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 -- 172.21.15.44:0/2778350085 learned_addr learned my addr 172.21.15.44:0/2778350085 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9200640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09f4076490 0x7f09f419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f3e00640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 0x7f09f419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 -- 172.21.15.44:0/2778350085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09f4076490 msgr2=0x7f09f419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09f4076490 0x7f09f419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 -- 172.21.15.44:0/2778350085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 msgr2=0x7f09f419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 0x7f09f419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.206+0000 7f09f9c00640 1 -- 172.21.15.44:0/2778350085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f41a3de0 con 0x7f09f4112ca0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.207+0000 7f09f9c00640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f41a36b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00eb40 tx=0x7f09ec00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.207+0000 7f09f1600640 1 -- 172.21.15.44:0/2778350085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00ce40 con 0x7f09f4112ca0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.207+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f41a40d0 con 0x7f09f4112ca0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.207+0000 7f09f1600640 1 -- 172.21.15.44:0/2778350085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09ec004b20 con 0x7f09f4112ca0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.207+0000 7f09f1600640 1 -- 172.21.15.44:0/2778350085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0107a0 con 0x7f09f4112ca0 2024-04-20T11:32:21.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.207+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f41ab960 con 0x7f09f4112ca0 2024-04-20T11:32:21.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.211+0000 7f09fea05740 1 Processor -- start 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.212+0000 7f09f1600640 1 -- 172.21.15.44:0/2778350085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f09ec004cd0 con 0x7f09f4112ca0 2024-04-20T11:32:21.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.212+0000 7f09fea05740 1 -- start start 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.212+0000 7f09fea05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7170 0x557b831a7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.212+0000 7f09fea05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09fea05740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09fea05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b8310c050 con 0x557b831a7a90 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09fea05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b830b2490 con 0x557b831a7170 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09fea05740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b83108010 con 0x557b831b0670 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34052/0 (socket says 172.21.15.44:34052) 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 -- 172.21.15.44:0/3773496970 learned_addr learned my addr 172.21.15.44:0/3773496970 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7170 0x557b831a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f1600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f09dc0a3da0 0x7f09dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 -- 172.21.15.44:0/3773496970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 msgr2=0x557b831b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 -- 172.21.15.44:0/3773496970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7170 msgr2=0x557b831a7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7170 0x557b831a7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f9200640 1 -- 172.21.15.44:0/3773496970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b830ba070 con 0x557b831a7a90 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.213+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7170 0x557b831a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.214+0000 7f09f1600640 1 -- 172.21.15.44:0/2778350085 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f09ec0d0e40 con 0x7f09f4112ca0 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.214+0000 7f09f9c00640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f09dc0a3da0 0x7f09dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.214+0000 7f09f9200640 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831b0130 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009cd0 tx=0x7f09e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d85a8fc4e1e6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.214+0000 7f09e6000640 1 -- 172.21.15.44:0/3773496970 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e801e070 con 0x557b831a7a90 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.214+0000 7f09e6000640 1 -- 172.21.15.44:0/3773496970 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09e8019b30 con 0x557b831a7a90 2024-04-20T11:32:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.214+0000 7f09e6000640 1 -- 172.21.15.44:0/3773496970 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e800eb30 con 0x557b831a7a90 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 -- 172.21.15.44:0/3773496970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 msgr2=0x557b831b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831b0130 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009cd0 tx=0x7f09e8011810 comp rx=0 tx=0).stop 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09f9c00640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f09dc0a3da0 0x7f09dc0a6260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f09ec09cd50 tx=0x7f09ec005820 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 -- 172.21.15.44:0/3773496970 shutdown_connections 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831b0130 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 --2- 172.21.15.44:0/3773496970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7170 0x557b831a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 -- 172.21.15.44:0/3773496970 >> 172.21.15.44:0/3773496970 conn(0x557b831a18a0 msgr2=0x557b831a1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 -- 172.21.15.44:0/3773496970 shutdown_connections 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.215+0000 7f09fea05740 1 -- 172.21.15.44:0/3773496970 wait complete. 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 -- start start 2024-04-20T11:32:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 Processor -- start 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 0x557b831ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 0x557b831adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557b831b3460 con 0x557b831a7170 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b830c5860 con 0x557b831a7a90 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.216+0000 7f09fea05740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b831b35e0 con 0x557b831b0670 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09fea05740 10 client.0 _ll_register_callbacks cb 0x557b83093fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 0x557b831ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 0x557b831adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 0x557b831adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55696/0 (socket says 172.21.15.44:55696) 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 -- 172.21.15.44:0/3188480223 learned_addr learned my addr 172.21.15.44:0/3188480223 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 0x557b831ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34056/0 (socket says 172.21.15.44:34056) 2024-04-20T11:32:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 -- 172.21.15.44:0/3188480223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 msgr2=0x557b831ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 -- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 msgr2=0x557b831ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 0x557b831ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f9200640 1 -- 172.21.15.44:0/3188480223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b830c9950 con 0x557b831a7a90 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.217+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 0x557b831ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09f9200640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 0x557b831adc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f09e801c040 tx=0x7f09e8019aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7752d919e30e9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09daa00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09daa00640 1 -- 172.21.15.44:0/3188480223 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e801e050 con 0x557b831a7a90 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e8019b30 con 0x557b831a7a90 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09daa00640 1 -- 172.21.15.44:0/3188480223 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f09e800eca0 con 0x557b831a7a90 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09daa00640 1 -- 172.21.15.44:0/3188480223 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f09e8033050 con 0x557b831a7a90 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09fea05740 10 client.28091 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557b83097820 con 0x557b831a7a90 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.218+0000 7f09daa00640 1 -- 172.21.15.44:0/3188480223 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09e8019b30 con 0x557b831a7a90 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09daa00640 1 -- 172.21.15.44:0/3188480223 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f09e8019ce0 con 0x557b831a7a90 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09fea05740 10 client.28091 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09fea05740 10 client.28091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09fea05740 20 client.28091 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09fea05740 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b831c0600 0x557b831c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09fea05740 4 client.28091 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b83119a70 con 0x557b831c0600 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.219+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b831c0600 0x557b831c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.220+0000 7f09f3e00640 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b831c0600 0x557b831c29f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf5d0d4b8eb6df84 server_cookie=1746a696e1abc62d in_seq=0 out_seq=0 2024-04-20T11:32:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.220+0000 7f09daa00640 10 client.28091 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.220+0000 7f09daa00640 1 -- 172.21.15.44:0/3188480223 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b83119a70 con 0x557b831c0600 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.220+0000 7f09daa00640 10 client.28091 handle_command_reply: tid=0 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 client.28091 shutdown 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 msgr2=0x557b831adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 0x557b831adc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f09e801c040 tx=0x7f09e8019aa0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 shutdown_connections 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b831c0600 0x557b831c29f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831b0670 0x557b831ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831a7a90 0x557b831adc40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 --2- 172.21.15.44:0/3188480223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7170 0x557b831ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09daa00640 0 client.28091 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 >> 172.21.15.44:0/3188480223 conn(0x557b831a18a0 msgr2=0x557b831a35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 shutdown_connections 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 1 -- 172.21.15.44:0/3188480223 wait complete. 2024-04-20T11:32:21.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.221+0000 7f09fea05740 20 client.28091 trim_cache size 0 max 16384 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.236+0000 7f09fea05740 1 Processor -- start 2024-04-20T11:32:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.236+0000 7f09fea05740 1 -- start start 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09fea05740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09fea05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09fea05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09fea05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b4130 con 0x557b831a7a90 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09fea05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b5c70 con 0x557b831b0670 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09fea05740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b4800 con 0x557b831a7170 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831bc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34064/0 (socket says 172.21.15.44:34064) 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 -- 172.21.15.44:0/4033675588 learned_addr learned my addr 172.21.15.44:0/4033675588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f3e00640 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 -- 172.21.15.44:0/4033675588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 msgr2=0x557b831afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 -- 172.21.15.44:0/4033675588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 msgr2=0x557b831a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.237+0000 7f09f9200640 1 -- 172.21.15.44:0/4033675588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b82e1e3f0 con 0x557b831a7a90 2024-04-20T11:32:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.238+0000 7f09f9200640 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831bc320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f09e8012fd0 tx=0x7f09e8019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6767f6421712a764 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.238+0000 7f09e5600640 1 -- 172.21.15.44:0/4033675588 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e803c070 con 0x557b831a7a90 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.238+0000 7f09e5600640 1 -- 172.21.15.44:0/4033675588 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09e802a070 con 0x557b831a7a90 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.238+0000 7f09e5600640 1 -- 172.21.15.44:0/4033675588 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e801e070 con 0x557b831a7a90 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 -- 172.21.15.44:0/4033675588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 msgr2=0x557b831bc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831bc320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f09e8012fd0 tx=0x7f09e8019780 comp rx=0 tx=0).stop 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 -- 172.21.15.44:0/4033675588 shutdown_connections 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831bc320 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 --2- 172.21.15.44:0/4033675588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 -- 172.21.15.44:0/4033675588 >> 172.21.15.44:0/4033675588 conn(0x557b831a18a0 msgr2=0x557b831a00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 -- 172.21.15.44:0/4033675588 shutdown_connections 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 -- 172.21.15.44:0/4033675588 wait complete. 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 -- start start 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.239+0000 7f09fea05740 1 Processor -- start 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b55a0 con 0x557b831a7a90 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b4ed0 con 0x557b831b0670 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b2120 con 0x557b831a7170 2024-04-20T11:32:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09fea05740 10 client.0 _ll_register_callbacks cb 0x7f09e8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09f9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09f9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831a9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34068/0 (socket says 172.21.15.44:34068) 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09f9200640 1 -- 172.21.15.44:0/2076402521 learned_addr learned my addr 172.21.15.44:0/2076402521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.240+0000 7f09f3e00640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f9200640 1 -- 172.21.15.44:0/2076402521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 msgr2=0x557b831b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f9200640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f9200640 1 -- 172.21.15.44:0/2076402521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 msgr2=0x557b831aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f9200640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f9200640 1 -- 172.21.15.44:0/2076402521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b82ee87d0 con 0x557b831a7a90 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f3e00640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09f9200640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831a9f90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f09e80048e0 tx=0x7f09e8009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2e01fc0653439f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09daa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09daa00640 1 -- 172.21.15.44:0/2076402521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e803c040 con 0x557b831a7a90 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09daa00640 1 -- 172.21.15.44:0/2076402521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f09e8009260 con 0x557b831a7a90 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e802a070 con 0x557b831a7a90 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.241+0000 7f09daa00640 1 -- 172.21.15.44:0/2076402521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f09e801e040 con 0x557b831a7a90 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 10 client.17982 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557b83226610 con 0x557b831a7a90 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09daa00640 1 -- 172.21.15.44:0/2076402521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f09e802a070 con 0x557b831a7a90 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09daa00640 1 -- 172.21.15.44:0/2076402521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f09e8040560 con 0x557b831a7a90 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 10 client.17982 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 10 client.17982 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 20 client.17982 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b83226c70 0x557b83229090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 4 client.17982 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b831138e0 con 0x557b83226c70 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.242+0000 7f09f3e00640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b83226c70 0x557b83229090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.243+0000 7f09f3e00640 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b83226c70 0x557b83229090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eff31eb144ff7c13 server_cookie=eab3e3c665922dc3 in_seq=0 out_seq=0 2024-04-20T11:32:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.243+0000 7f09daa00640 10 client.17982 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.243+0000 7f09daa00640 1 -- 172.21.15.44:0/2076402521 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b831138e0 con 0x557b83226c70 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.243+0000 7f09daa00640 10 client.17982 handle_command_reply: tid=0 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.243+0000 7f09fea05740 1 client.17982 shutdown 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 msgr2=0x557b831a9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831a9f90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f09e80048e0 tx=0x7f09e8009e80 comp rx=0 tx=0).stop 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 shutdown_connections 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x557b83226c70 0x557b83229090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b831b0670 0x557b831aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b831a7a90 0x557b831a9f90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 --2- 172.21.15.44:0/2076402521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b831a7170 0x557b831b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09daa00640 0 client.17982 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 >> 172.21.15.44:0/2076402521 conn(0x557b831a18a0 msgr2=0x557b831adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 shutdown_connections 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 1 -- 172.21.15.44:0/2076402521 wait complete. 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.244+0000 7f09fea05740 20 client.17982 trim_cache size 0 max 16384 2024-04-20T11:32:21.249 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f09dc0a3da0 msgr2=0x7f09dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f09dc0a3da0 0x7f09dc0a6260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f09ec09cd50 tx=0x7f09ec005820 comp rx=0 tx=0).stop 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 msgr2=0x7f09f41a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f41a36b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00eb40 tx=0x7f09ec00ef40 comp rx=0 tx=0).stop 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 shutdown_connections 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f09dc0a3da0 0x7f09dc0a6260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09f4112ca0 0x7f09f41a36b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f4076db0 0x7f09f419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 --2- 172.21.15.44:0/2778350085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09f4076490 0x7f09f419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 >> 172.21.15.44:0/2778350085 conn(0x7f09f40ffad0 msgr2=0x7f09f4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 shutdown_connections 2024-04-20T11:32:21.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.245+0000 7f09fa600640 1 -- 172.21.15.44:0/2778350085 wait complete. 2024-04-20T11:32:21.264 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:32:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 -- 172.21.15.44:0/768678737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e68109480 msgr2=0x7f4e68111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 --2- 172.21.15.44:0/768678737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e68109480 0x7f4e68111050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009ca0 tx=0x7f4e6002c100 comp rx=0 tx=0).stop 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 -- 172.21.15.44:0/768678737 shutdown_connections 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 --2- 172.21.15.44:0/768678737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e68109480 0x7f4e68111050 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 --2- 172.21.15.44:0/768678737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e68104d60 0x7f4e68108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 --2- 172.21.15.44:0/768678737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e681043b0 0x7f4e68104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.482+0000 7f4e6f200640 1 -- 172.21.15.44:0/768678737 >> 172.21.15.44:0/768678737 conn(0x7f4e680ffbd0 msgr2=0x7f4e68101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.483+0000 7f4e6f200640 1 -- 172.21.15.44:0/768678737 shutdown_connections 2024-04-20T11:32:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.483+0000 7f4e6f200640 1 -- 172.21.15.44:0/768678737 wait complete. 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 Processor -- start 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 -- start start 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 0x7f4e681a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e68109480 0x7f4e681a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6819a8f0 0x7f4e6819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68114650 con 0x7f4e6819a8f0 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681144d0 con 0x7f4e681043b0 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681147d0 con 0x7f4e68109480 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e67e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 0x7f4e681a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6819a8f0 0x7f4e6819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e67e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 0x7f4e681a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55736/0 (socket says 172.21.15.44:55736) 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.484+0000 7f4e67e00640 1 -- 172.21.15.44:0/2580683244 learned_addr learned my addr 172.21.15.44:0/2580683244 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67400640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e68109480 0x7f4e681a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67e00640 1 -- 172.21.15.44:0/2580683244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e68109480 msgr2=0x7f4e681a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67e00640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e68109480 0x7f4e681a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67e00640 1 -- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6819a8f0 msgr2=0x7f4e6819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67e00640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6819a8f0 0x7f4e6819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67e00640 1 -- 172.21.15.44:0/2580683244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6819b410 con 0x7f4e681043b0 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e6d200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6819a8f0 0x7f4e6819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e67e00640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 0x7f4e681a0740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800ea60 tx=0x7f4e5800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e64c00640 1 -- 172.21.15.44:0/2580683244 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800cde0 con 0x7f4e681043b0 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e64c00640 1 -- 172.21.15.44:0/2580683244 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e580035f0 con 0x7f4e681043b0 2024-04-20T11:32:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6819b700 con 0x7f4e681043b0 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.485+0000 7f4e64c00640 1 -- 172.21.15.44:0/2580683244 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58010720 con 0x7f4e681043b0 2024-04-20T11:32:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.486+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681a7950 con 0x7f4e681043b0 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 Processor -- start 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 -- start start 2024-04-20T11:32:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e64c00640 1 -- 172.21.15.44:0/2580683244 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4e580037a0 con 0x7f4e681043b0 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 0x5576cbf2b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf33f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5576cbe90050 con 0x5576cbf2afb0 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5576cbe36490 con 0x5576cbf344b0 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.489+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5576cbe8c010 con 0x5576cbf2b8d0 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 0x5576cbf2b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf33f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 0x5576cbf2b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34102/0 (socket says 172.21.15.44:34102) 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 -- 172.21.15.44:0/702734030 learned_addr learned my addr 172.21.15.44:0/702734030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e67400640 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 -- 172.21.15.44:0/702734030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 msgr2=0x5576cbf33f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf33f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 -- 172.21.15.44:0/702734030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 msgr2=0x5576cbf368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.490+0000 7f4e6d200640 1 -- 172.21.15.44:0/702734030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576cbe3e070 con 0x5576cbf2afb0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e6d200640 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 0x5576cbf2b390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009ca0 tx=0x7f4e60005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52a559de80f0ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e64c00640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4e4c0a3d20 0x7f4e4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e5d600640 1 -- 172.21.15.44:0/702734030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60043070 con 0x5576cbf2afb0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e5d600640 1 -- 172.21.15.44:0/702734030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e60032d40 con 0x5576cbf2afb0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e64c00640 1 -- 172.21.15.44:0/2580683244 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f4e580d0b30 con 0x7f4e681043b0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e5d600640 1 -- 172.21.15.44:0/702734030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60039040 con 0x5576cbf2afb0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.491+0000 7f4e67e00640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4e4c0a3d20 0x7f4e4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 -- 172.21.15.44:0/702734030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 msgr2=0x5576cbf2b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 0x5576cbf2b390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009ca0 tx=0x7f4e60005ef0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e67e00640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4e4c0a3d20 0x7f4e4c0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5576cbf2aa80 tx=0x7f4e580d45e0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 -- 172.21.15.44:0/702734030 shutdown_connections 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf33f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/702734030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2afb0 0x5576cbf2b390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 -- 172.21.15.44:0/702734030 >> 172.21.15.44:0/702734030 conn(0x5576cbf256e0 msgr2=0x5576cbf25ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 -- 172.21.15.44:0/702734030 shutdown_connections 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 -- 172.21.15.44:0/702734030 wait complete. 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.492+0000 7f4e71fc4740 1 -- start start 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 Processor -- start 2024-04-20T11:32:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2afb0 0x5576cbf315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 0x5576cbf31af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5576cbf37240 con 0x5576cbf2b8d0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5576cbe49860 con 0x5576cbf344b0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5576cbf373c0 con 0x5576cbf2afb0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e71fc4740 10 client.0 _ll_register_callbacks cb 0x5576cbe17fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e67400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 0x5576cbf31af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e67400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 0x5576cbf31af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34106/0 (socket says 172.21.15.44:34106) 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.493+0000 7f4e67400640 1 -- 172.21.15.44:0/1383738613 learned_addr learned my addr 172.21.15.44:0/1383738613 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e6d200640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2afb0 0x5576cbf315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e6d200640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e67400640 1 -- 172.21.15.44:0/1383738613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2afb0 msgr2=0x5576cbf315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e67400640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2afb0 0x5576cbf315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e67400640 1 -- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 msgr2=0x5576cbf32030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e67400640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf32030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e67400640 1 -- 172.21.15.44:0/1383738613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576cbe4d950 con 0x5576cbf2b8d0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e6d200640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf32030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e67400640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 0x5576cbf31af0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4e54012700 tx=0x7f4e54012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ec2e0493d358ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e52000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.494+0000 7f4e52000640 1 -- 172.21.15.44:0/1383738613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400f920 con 0x5576cbf2b8d0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e52000640 1 -- 172.21.15.44:0/1383738613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f4e5400fac0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e60032d40 con 0x5576cbf2b8d0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e52000640 1 -- 172.21.15.44:0/1383738613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4e5400fcc0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e71fc4740 10 client.17994 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5576cbe1b820 con 0x5576cbf2b8d0 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e52000640 1 -- 172.21.15.44:0/1383738613 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e60032d40 con 0x5576cbf2b8d0 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e52000640 1 -- 172.21.15.44:0/1383738613 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f4e54010550 con 0x5576cbf2b8d0 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e71fc4740 10 client.17994 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.495+0000 7f4e71fc4740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.496+0000 7f4e71fc4740 20 client.17994 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.496+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbf445d0 0x5576cbf46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.496+0000 7f4e71fc4740 4 client.17994 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.496+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576cbe9da70 con 0x5576cbf445d0 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.496+0000 7f4e6d200640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbf445d0 0x5576cbf46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.497+0000 7f4e6d200640 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbf445d0 0x5576cbf46a10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75aa59ab2f226f04 server_cookie=1d966121812c982f in_seq=0 out_seq=0 2024-04-20T11:32:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.497+0000 7f4e52000640 10 client.17994 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.497+0000 7f4e52000640 1 -- 172.21.15.44:0/1383738613 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576cbe9da70 con 0x5576cbf445d0 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.497+0000 7f4e52000640 10 client.17994 handle_command_reply: tid=0 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 client.17994 shutdown 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 msgr2=0x5576cbf31af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 0x5576cbf31af0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4e54012700 tx=0x7f4e54012bc0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 shutdown_connections 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbf445d0 0x5576cbf46a10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf344b0 0x5576cbf32030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf2b8d0 0x5576cbf31af0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/1383738613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2afb0 0x5576cbf315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e52000640 0 client.17994 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 >> 172.21.15.44:0/1383738613 conn(0x5576cbf256e0 msgr2=0x5576cbf27430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 shutdown_connections 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 1 -- 172.21.15.44:0/1383738613 wait complete. 2024-04-20T11:32:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.498+0000 7f4e71fc4740 20 client.17994 trim_cache size 0 max 16384 2024-04-20T11:32:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 Processor -- start 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 -- start start 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf40100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681aff10 con 0x5576cbf344b0 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681b1a50 con 0x5576cbf2afb0 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.513+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681b05e0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf40100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf33ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55760/0 (socket says 172.21.15.44:55760) 2024-04-20T11:32:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e6d200640 1 -- 172.21.15.44:0/4243301977 learned_addr learned my addr 172.21.15.44:0/4243301977 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e6d200640 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf27240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e67400640 1 -- 172.21.15.44:0/4243301977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 msgr2=0x5576cbf33ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e67400640 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf33ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e67400640 1 -- 172.21.15.44:0/4243301977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 msgr2=0x5576cbf27240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e67400640 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf27240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.514+0000 7f4e67400640 1 -- 172.21.15.44:0/4243301977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576cbba23f0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.515+0000 7f4e67400640 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf40100 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4e54010940 tx=0x7f4e54005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5909bd06b88c9af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.515+0000 7f4e5cc00640 1 -- 172.21.15.44:0/4243301977 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54029070 con 0x5576cbf2b8d0 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.515+0000 7f4e5cc00640 1 -- 172.21.15.44:0/4243301977 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e5400f9a0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.515+0000 7f4e5cc00640 1 -- 172.21.15.44:0/4243301977 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54010430 con 0x5576cbf2b8d0 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 -- 172.21.15.44:0/4243301977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 msgr2=0x5576cbf40100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf40100 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4e54010940 tx=0x7f4e54005d30 comp rx=0 tx=0).stop 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 -- 172.21.15.44:0/4243301977 shutdown_connections 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf40100 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/4243301977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 -- 172.21.15.44:0/4243301977 >> 172.21.15.44:0/4243301977 conn(0x5576cbf256e0 msgr2=0x5576cbf26b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 -- 172.21.15.44:0/4243301977 shutdown_connections 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 -- 172.21.15.44:0/4243301977 wait complete. 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 -- start start 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.516+0000 7f4e71fc4740 1 Processor -- start 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf37bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681b1380 con 0x5576cbf344b0 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681b0cb0 con 0x5576cbf2afb0 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681adf00 con 0x5576cbf2b8d0 2024-04-20T11:32:21.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e71fc4740 10 client.0 _ll_register_callbacks cb 0x7f4e54009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e6d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf38130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34122/0 (socket says 172.21.15.44:34122) 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e6d200640 1 -- 172.21.15.44:0/3177170245 learned_addr learned my addr 172.21.15.44:0/3177170245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e67400640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf37bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.517+0000 7f4e6d200640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e67400640 1 -- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 msgr2=0x5576cbf3c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e67400640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf3c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e67400640 1 -- 172.21.15.44:0/3177170245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 msgr2=0x5576cbf38130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e67400640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf38130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e67400640 1 -- 172.21.15.44:0/3177170245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576cbc6c7d0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e6d200640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:21.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e67400640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf37bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4e54000c00 tx=0x7f4e54011f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84a3f0003b6a459 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e52000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e52000640 1 -- 172.21.15.44:0/3177170245 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54029030 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e52000640 1 -- 172.21.15.44:0/3177170245 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f4e54005320 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.518+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5400f9a0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e52000640 1 -- 172.21.15.44:0/3177170245 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4e54005520 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 10 client.28117 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5576cbfaa6b0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e52000640 1 -- 172.21.15.44:0/3177170245 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e5400f9a0 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e52000640 1 -- 172.21.15.44:0/3177170245 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f4e5401c470 con 0x5576cbf2b8d0 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 10 client.28117 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:21.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 10 client.28117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 20 client.28117 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbfaace0 0x5576cbfad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 4 client.28117 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.519+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576cbe978e0 con 0x5576cbfaace0 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.520+0000 7f4e6d200640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbfaace0 0x5576cbfad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.520+0000 7f4e6d200640 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbfaace0 0x5576cbfad100 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aee656ac6d075cd server_cookie=23646b9d9e195ba1 in_seq=0 out_seq=0 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.520+0000 7f4e52000640 10 client.28117 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.520+0000 7f4e52000640 1 -- 172.21.15.44:0/3177170245 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576cbe978e0 con 0x5576cbfaace0 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.520+0000 7f4e52000640 10 client.28117 handle_command_reply: tid=0 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.520+0000 7f4e71fc4740 1 client.28117 shutdown 2024-04-20T11:32:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 msgr2=0x5576cbf37bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf37bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4e54000c00 tx=0x7f4e54011f00 comp rx=0 tx=0).stop 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 shutdown_connections 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5576cbfaace0 0x5576cbfad100 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5576cbf344b0 0x5576cbf38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5576cbf2b8d0 0x5576cbf37bf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 --2- 172.21.15.44:0/3177170245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576cbf2afb0 0x5576cbf3c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 >> 172.21.15.44:0/3177170245 conn(0x5576cbf256e0 msgr2=0x5576cbf31e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e52000640 0 client.28117 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 shutdown_connections 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 1 -- 172.21.15.44:0/3177170245 wait complete. 2024-04-20T11:32:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.521+0000 7f4e71fc4740 20 client.28117 trim_cache size 0 max 16384 2024-04-20T11:32:21.529 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4e4c0a3d20 msgr2=0x7f4e4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4e4c0a3d20 0x7f4e4c0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5576cbf2aa80 tx=0x7f4e580d45e0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 msgr2=0x7f4e681a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 0x7f4e681a0740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800ea60 tx=0x7f4e5800ef20 comp rx=0 tx=0).stop 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 shutdown_connections 2024-04-20T11:32:21.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f4e4c0a3d20 0x7f4e4c0a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6819a8f0 0x7f4e6819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e68109480 0x7f4e681a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 --2- 172.21.15.44:0/2580683244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e681043b0 0x7f4e681a0740 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 >> 172.21.15.44:0/2580683244 conn(0x7f4e680ffbd0 msgr2=0x7f4e681006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 shutdown_connections 2024-04-20T11:32:21.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:21.522+0000 7f4e6f200640 1 -- 172.21.15.44:0/2580683244 wait complete. 2024-04-20T11:32:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:21 smithi044 ceph-mon[16855]: pgmap v303: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:21.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:21 smithi165 ceph-mon[21975]: pgmap v303: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:22.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:21 smithi078 ceph-mon[21464]: pgmap v303: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:32:22.540 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T11:32:22.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:32:22.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 -- 172.21.15.44:0/2516032977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe658102a00 msgr2=0x7fe658106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 --2- 172.21.15.44:0/2516032977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe658102a00 0x7fe658106a40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe640009900 tx=0x7fe64002bc30 comp rx=0 tx=0).stop 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 -- 172.21.15.44:0/2516032977 shutdown_connections 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 --2- 172.21.15.44:0/2516032977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe658107120 0x7fe65810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 --2- 172.21.15.44:0/2516032977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe658102a00 0x7fe658106a40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 --2- 172.21.15.44:0/2516032977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 0x7fe658102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 -- 172.21.15.44:0/2516032977 >> 172.21.15.44:0/2516032977 conn(0x7fe6580fd930 msgr2=0x7fe6580ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 -- 172.21.15.44:0/2516032977 shutdown_connections 2024-04-20T11:32:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.770+0000 7fe65e000640 1 -- 172.21.15.44:0/2516032977 wait complete. 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.771+0000 7fe65e000640 1 Processor -- start 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.771+0000 7fe65e000640 1 -- start start 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 0x7fe65819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe658107120 0x7fe65819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 0x7fe658198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe658112280 con 0x7fe6581986a0 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65e000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe658112100 con 0x7fe658102050 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe658112400 con 0x7fe658107120 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 0x7fe658198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 0x7fe658198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34140/0 (socket says 172.21.15.44:34140) 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 0x7fe65819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 -- 172.21.15.44:0/4156378534 learned_addr learned my addr 172.21.15.44:0/4156378534 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe657e00640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe658107120 0x7fe65819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 -- 172.21.15.44:0/4156378534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe658107120 msgr2=0x7fe65819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe658107120 0x7fe65819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 -- 172.21.15.44:0/4156378534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 msgr2=0x7fe65819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.772+0000 7fe65d600640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 0x7fe65819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe65d600640 1 -- 172.21.15.44:0/4156378534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6581992a0 con 0x7fe6581986a0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe65cc00640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 0x7fe65819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe65d600640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 0x7fe658198b20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe65000e970 tx=0x7fe65000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe655600640 1 -- 172.21.15.44:0/4156378534 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65000ccc0 con 0x7fe6581986a0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe655600640 1 -- 172.21.15.44:0/4156378534 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe65000ce60 con 0x7fe6581986a0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6581a5330 con 0x7fe6581986a0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe655600640 1 -- 172.21.15.44:0/4156378534 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650010690 con 0x7fe6581986a0 2024-04-20T11:32:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.773+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6581a56e0 con 0x7fe6581986a0 2024-04-20T11:32:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 Processor -- start 2024-04-20T11:32:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe655600640 1 -- 172.21.15.44:0/4156378534 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe6500046c0 con 0x7fe6581986a0 2024-04-20T11:32:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 -- start start 2024-04-20T11:32:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a1a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a27c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e1797ad20 con 0x559e17a1a310 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e179213e0 con 0x559e17a281d0 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.777+0000 7fe6623a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559e17976ce0 con 0x559e17a1ac30 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a27c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe657e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a1a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a27c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48320/0 (socket says 172.21.15.44:48320) 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 -- 172.21.15.44:0/1654244818 learned_addr learned my addr 172.21.15.44:0/1654244818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 -- 172.21.15.44:0/1654244818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 msgr2=0x559e17a2a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 -- 172.21.15.44:0/1654244818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 msgr2=0x559e17a1a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a1a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 -- 172.21.15.44:0/1654244818 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e17940980 con 0x559e17a1ac30 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.778+0000 7fe65cc00640 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a27c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe648011860 tx=0x7fe648011d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5334db2d9970ff99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.779+0000 7fe655600640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.779+0000 7fe64dc00640 1 -- 172.21.15.44:0/1654244818 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480112f0 con 0x559e17a1ac30 2024-04-20T11:32:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.779+0000 7fe64dc00640 1 -- 172.21.15.44:0/1654244818 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe648004b20 con 0x559e17a1ac30 2024-04-20T11:32:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.779+0000 7fe64dc00640 1 -- 172.21.15.44:0/1654244818 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648005810 con 0x559e17a1ac30 2024-04-20T11:32:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.779+0000 7fe655600640 1 -- 172.21.15.44:0/4156378534 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fe6500d0b70 con 0x7fe6581986a0 2024-04-20T11:32:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.779+0000 7fe657e00640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 -- 172.21.15.44:0/1654244818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 msgr2=0x559e17a27c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a27c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe648011860 tx=0x7fe648011d20 comp rx=0 tx=0).stop 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 -- 172.21.15.44:0/1654244818 shutdown_connections 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a27c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1654244818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a1a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 -- 172.21.15.44:0/1654244818 >> 172.21.15.44:0/1654244818 conn(0x559e17a14c30 msgr2=0x559e17a15010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe657e00640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe640009230 tx=0x7fe640005e80 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 -- 172.21.15.44:0/1654244818 shutdown_connections 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 -- 172.21.15.44:0/1654244818 wait complete. 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.780+0000 7fe6623a9740 1 -- start start 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 Processor -- start 2024-04-20T11:32:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a11310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a11850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a11d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559e17a2ac20 con 0x559e17a1a310 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e179347a0 con 0x559e17a281d0 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559e17a2ada0 con 0x559e17a1ac30 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe6623a9740 10 client.0 _ll_register_callbacks cb 0x559e17936b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a11310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a11310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34156/0 (socket says 172.21.15.44:34156) 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe65cc00640 1 -- 172.21.15.44:0/4067132608 learned_addr learned my addr 172.21.15.44:0/4067132608 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.781+0000 7fe657e00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a11850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe65d600640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a11d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe657e00640 1 -- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 msgr2=0x559e17a11d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe657e00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a11d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe657e00640 1 -- 172.21.15.44:0/4067132608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 msgr2=0x559e17a11310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe657e00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a11310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe657e00640 1 -- 172.21.15.44:0/4067132608 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e177557e0 con 0x559e17a1ac30 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe65cc00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a11310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe65d600640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a11d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe657e00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a11850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe640036040 tx=0x7fe64002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc794ba78cc9636f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.782+0000 7fe646000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe646000640 1 -- 172.21.15.44:0/4067132608 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64004e020 con 0x559e17a1ac30 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe646000640 1 -- 172.21.15.44:0/4067132608 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fe640004380 con 0x559e17a1ac30 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe648004b20 con 0x559e17a1ac30 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe646000640 1 -- 172.21.15.44:0/4067132608 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe640059050 con 0x559e17a1ac30 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe6623a9740 10 client.28129 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559e177b5f70 con 0x559e17a1ac30 2024-04-20T11:32:22.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.783+0000 7fe646000640 1 -- 172.21.15.44:0/4067132608 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe648004b20 con 0x559e17a1ac30 2024-04-20T11:32:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe646000640 1 -- 172.21.15.44:0/4067132608 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe640003b10 con 0x559e17a1ac30 2024-04-20T11:32:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe6623a9740 10 client.28129 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe6623a9740 10 client.28129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe6623a9740 20 client.28129 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a38160 0x559e17a3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe6623a9740 4 client.28129 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e17988740 con 0x559e17a38160 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.784+0000 7fe65cc00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a38160 0x559e17a3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.785+0000 7fe65cc00640 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a38160 0x559e17a3a5f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b46b4bfb0eeed8b server_cookie=481e8c557aa5a1ca in_seq=0 out_seq=0 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.785+0000 7fe646000640 10 client.28129 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe646000640 1 -- 172.21.15.44:0/4067132608 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e17988740 con 0x559e17a38160 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe646000640 10 client.28129 handle_command_reply: tid=0 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 client.28129 shutdown 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 msgr2=0x559e17a11850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a11850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe640036040 tx=0x7fe64002bc30 comp rx=0 tx=0).stop 2024-04-20T11:32:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 shutdown_connections 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a38160 0x559e17a3a5f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a11d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a11850 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4067132608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a11310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 >> 172.21.15.44:0/4067132608 conn(0x559e17a14c30 msgr2=0x559e17a16980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe646000640 0 client.28129 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 shutdown_connections 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 1 -- 172.21.15.44:0/4067132608 wait complete. 2024-04-20T11:32:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.786+0000 7fe6623a9740 20 client.28129 trim_cache size 0 max 16384 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 Processor -- start 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 -- start start 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a1ac30 0x559e17a33ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a281d0 0x559e17a24380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6581ab870 con 0x559e17a1a310 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6581ad3b0 con 0x559e17a1ac30 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe6623a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6581abf40 con 0x559e17a281d0 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe65d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a281d0 0x559e17a24380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34172/0 (socket says 172.21.15.44:34172) 2024-04-20T11:32:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.806+0000 7fe65cc00640 1 -- 172.21.15.44:0/4064480813 learned_addr learned my addr 172.21.15.44:0/4064480813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe657e00640 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a1ac30 0x559e17a33ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65d600640 1 -- 172.21.15.44:0/4064480813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a1ac30 msgr2=0x559e17a33ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65d600640 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a1ac30 0x559e17a33ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65d600640 1 -- 172.21.15.44:0/4064480813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 msgr2=0x559e17a139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65d600640 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65d600640 1 -- 172.21.15.44:0/4064480813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e17a1aa60 con 0x559e17a281d0 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65cc00640 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe657e00640 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a1ac30 0x559e17a33ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe65d600640 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a281d0 0x559e17a24380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe6500ce270 tx=0x7fe650098e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64c334f104af99b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe64d200640 1 -- 172.21.15.44:0/4064480813 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6500b5020 con 0x559e17a281d0 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe64d200640 1 -- 172.21.15.44:0/4064480813 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6500a5760 con 0x559e17a281d0 2024-04-20T11:32:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.807+0000 7fe64d200640 1 -- 172.21.15.44:0/4064480813 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65009c1a0 con 0x559e17a281d0 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 -- 172.21.15.44:0/4064480813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a281d0 msgr2=0x559e17a24380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a281d0 0x559e17a24380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe6500ce270 tx=0x7fe650098e10 comp rx=0 tx=0).stop 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 -- 172.21.15.44:0/4064480813 shutdown_connections 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a281d0 0x559e17a24380 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a1ac30 0x559e17a33ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 --2- 172.21.15.44:0/4064480813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 -- 172.21.15.44:0/4064480813 >> 172.21.15.44:0/4064480813 conn(0x559e17a14c30 msgr2=0x559e17a23c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 -- 172.21.15.44:0/4064480813 shutdown_connections 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.808+0000 7fe6623a9740 1 -- 172.21.15.44:0/4064480813 wait complete. 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 -- start start 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 Processor -- start 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a9d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6581acce0 con 0x559e17a1a310 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6581ac610 con 0x559e17a281d0 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6581a9860 con 0x559e17a1ac30 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe6623a9740 10 client.0 _ll_register_callbacks cb 0x7fe640009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.809+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a272b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34178/0 (socket says 172.21.15.44:34178) 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 -- 172.21.15.44:0/1993542922 learned_addr learned my addr 172.21.15.44:0/1993542922 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe657e00640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65d600640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a9d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 -- 172.21.15.44:0/1993542922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 msgr2=0x559e17a277f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a277f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 -- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 msgr2=0x559e17a9d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a9d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 -- 172.21.15.44:0/1993542922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1764d170 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65d600640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a9d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe65cc00640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a272b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cfdf837579a1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.810+0000 7fe646000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe646000640 1 -- 172.21.15.44:0/1993542922 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648035060 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe646000640 1 -- 172.21.15.44:0/1993542922 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fe648003970 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6500a5760 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe646000640 1 -- 172.21.15.44:0/1993542922 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe64801e040 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 10 client.18018 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559e17a9ed80 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe646000640 1 -- 172.21.15.44:0/1993542922 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6500a5760 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe646000640 1 -- 172.21.15.44:0/1993542922 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7fe648024040 con 0x559e17a1a310 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 10 client.18018 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 20 client.18018 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.811+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a9f370 0x559e17aa1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.812+0000 7fe6623a9740 4 client.18018 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T11:32:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.812+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e179825b0 con 0x559e17a9f370 2024-04-20T11:32:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.812+0000 7fe657e00640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a9f370 0x559e17aa1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.812+0000 7fe657e00640 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a9f370 0x559e17aa1790 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36926ee3b513752f server_cookie=813aa6795e265a13 in_seq=0 out_seq=0 2024-04-20T11:32:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.812+0000 7fe646000640 10 client.18018 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe646000640 1 -- 172.21.15.44:0/1993542922 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e179825b0 con 0x559e17a9f370 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe646000640 10 client.18018 handle_command_reply: tid=0 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 client.18018 shutdown 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 msgr2=0x559e17a272b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a272b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648005370 comp rx=0 tx=0).stop 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 shutdown_connections 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x559e17a9f370 0x559e17aa1790 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e17a281d0 0x559e17a9d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559e17a1ac30 0x559e17a277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 --2- 172.21.15.44:0/1993542922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e17a1a310 0x559e17a272b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 >> 172.21.15.44:0/1993542922 conn(0x559e17a14c30 msgr2=0x559e17a112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe646000640 0 client.18018 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 shutdown_connections 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 1 -- 172.21.15.44:0/1993542922 wait complete. 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.814+0000 7fe6623a9740 20 client.18018 trim_cache size 0 max 16384 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-04-20T11:32:22.818 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "c60319c5-1a08-45c6-9e99-ff015d26abc8", 2024-04-20T11:32:22.819 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-04-20T11:32:22.819 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:32:22.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe63c0a3d20 msgr2=0x7fe63c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe640009230 tx=0x7fe640005e80 comp rx=0 tx=0).stop 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 msgr2=0x7fe658198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 0x7fe658198b20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe65000e970 tx=0x7fe65000ed70 comp rx=0 tx=0).stop 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 shutdown_connections 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6581986a0 0x7fe658198b20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe658107120 0x7fe65819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 --2- 172.21.15.44:0/4156378534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe658102050 0x7fe65819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 >> 172.21.15.44:0/4156378534 conn(0x7fe6580fd930 msgr2=0x7fe6580ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 shutdown_connections 2024-04-20T11:32:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:22.815+0000 7fe65e000640 1 -- 172.21.15.44:0/4156378534 wait complete. 2024-04-20T11:32:22.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:32:22.835 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 -- 172.21.15.44:0/2047577499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a410c540 msgr2=0x7f31a410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 --2- 172.21.15.44:0/2047577499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a410c540 0x7f31a410e9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f319c009860 tx=0x7f319c02bc20 comp rx=0 tx=0).stop 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 -- 172.21.15.44:0/2047577499 shutdown_connections 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 --2- 172.21.15.44:0/2047577499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a410c540 0x7f31a410e9c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 --2- 172.21.15.44:0/2047577499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103b20 0x7f31a410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 --2- 172.21.15.44:0/2047577499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a4103200 0x7f31a41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 -- 172.21.15.44:0/2047577499 >> 172.21.15.44:0/2047577499 conn(0x7f31a4077fc0 msgr2=0x7f31a40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 -- 172.21.15.44:0/2047577499 shutdown_connections 2024-04-20T11:32:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.054+0000 7f31a9200640 1 -- 172.21.15.44:0/2047577499 wait complete. 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.055+0000 7f31a9200640 1 Processor -- start 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.055+0000 7f31a9200640 1 -- start start 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 0x7f31a41a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4103b20 0x7f31a41a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a410c540 0x7f31a419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4112440 con 0x7f31a4103200 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a9200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41122c0 con 0x7f31a4103b20 2024-04-20T11:32:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a9200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41125c0 con 0x7f31a410c540 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 0x7f31a41a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 0x7f31a41a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34188/0 (socket says 172.21.15.44:34188) 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 -- 172.21.15.44:0/3306130064 learned_addr learned my addr 172.21.15.44:0/3306130064 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1400640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4103b20 0x7f31a41a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a2800640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a410c540 0x7f31a419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 -- 172.21.15.44:0/3306130064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a410c540 msgr2=0x7f31a419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a410c540 0x7f31a419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 -- 172.21.15.44:0/3306130064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4103b20 msgr2=0x7f31a41a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4103b20 0x7f31a41a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1e00640 1 -- 172.21.15.44:0/3306130064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a419b140 con 0x7f31a4103200 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.056+0000 7f31a1400640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4103b20 0x7f31a41a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.057+0000 7f31a1e00640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 0x7f31a41a0810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f319400c420 tx=0x7f319400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.057+0000 7f319aa00640 1 -- 172.21.15.44:0/3306130064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004b20 con 0x7f31a4103200 2024-04-20T11:32:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.057+0000 7f319aa00640 1 -- 172.21.15.44:0/3306130064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3194004cc0 con 0x7f31a4103200 2024-04-20T11:32:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.057+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a419b430 con 0x7f31a4103200 2024-04-20T11:32:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.057+0000 7f319aa00640 1 -- 172.21.15.44:0/3306130064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400b630 con 0x7f31a4103200 2024-04-20T11:32:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.057+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41a76d0 con 0x7f31a4103200 2024-04-20T11:32:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.060+0000 7f31abe06740 1 Processor -- start 2024-04-20T11:32:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.060+0000 7f319aa00640 1 -- 172.21.15.44:0/3306130064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f319401d020 con 0x7f31a4103200 2024-04-20T11:32:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.060+0000 7f31abe06740 1 -- start start 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31abe06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f85390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31abe06740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f85960 0x558c99f96970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31abe06740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c99f992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31abe06740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558c99eeb150 con 0x558c99f96eb0 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31abe06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c99e91370 con 0x558c99f84fb0 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31abe06740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558c99ee7110 con 0x558c99f85960 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31a2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f85390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31a2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f85390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:55854/0 (socket says 172.21.15.44:55854) 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31a2800640 1 -- 172.21.15.44:0/1877517655 learned_addr learned my addr 172.21.15.44:0/1877517655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31a1400640 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c99f992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.061+0000 7f31a1e00640 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f85960 0x558c99f96970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f31a2800640 1 -- 172.21.15.44:0/1877517655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f85960 msgr2=0x558c99f96970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f31a2800640 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f85960 0x558c99f96970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f31a2800640 1 -- 172.21.15.44:0/1877517655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 msgr2=0x558c99f992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f31a2800640 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c99f992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f31a2800640 1 -- 172.21.15.44:0/1877517655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c99cc77d0 con 0x558c99f84fb0 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f31a2800640 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f85390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f319c009860 tx=0x7f319c005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c286eb36455ff7a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f318f400640 1 -- 172.21.15.44:0/1877517655 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c044030 con 0x558c99f84fb0 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f318f400640 1 -- 172.21.15.44:0/1877517655 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f319c035d40 con 0x558c99f84fb0 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.062+0000 7f318f400640 1 -- 172.21.15.44:0/1877517655 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c039040 con 0x558c99f84fb0 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f319aa00640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31880a3d20 0x7f31880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f319aa00640 1 -- 172.21.15.44:0/3306130064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f31940d0760 con 0x7f31a4103200 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31a1e00640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31880a3d20 0x7f31880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 -- 172.21.15.44:0/1877517655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 msgr2=0x558c99f85390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f85390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f319c009860 tx=0x7f319c005710 comp rx=0 tx=0).stop 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 -- 172.21.15.44:0/1877517655 shutdown_connections 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c99f992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f85960 0x558c99f96970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 --2- 172.21.15.44:0/1877517655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f85390 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.063+0000 7f31abe06740 1 -- 172.21.15.44:0/1877517655 >> 172.21.15.44:0/1877517655 conn(0x558c99f7f8d0 msgr2=0x558c99f7fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.064+0000 7f31abe06740 1 -- 172.21.15.44:0/1877517655 shutdown_connections 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.064+0000 7f31abe06740 1 -- 172.21.15.44:0/1877517655 wait complete. 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.064+0000 7f31a1e00640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31880a3d20 0x7f31880a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x558c99f80b60 tx=0x7f31940d6530 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.064+0000 7f31abe06740 1 -- start start 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.064+0000 7f31abe06740 1 Processor -- start 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f93f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f94480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f96eb0 0x558c99f949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558c99f99c10 con 0x558c99f85960 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c99ea4740 con 0x558c99f84fb0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558c99f99d90 con 0x558c99f96eb0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31abe06740 10 client.0 _ll_register_callbacks cb 0x558c99e9fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31a1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f94480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31a2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f93f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31a1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f94480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34208/0 (socket says 172.21.15.44:34208) 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31a1400640 1 -- 172.21.15.44:0/2409321174 learned_addr learned my addr 172.21.15.44:0/2409321174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.065+0000 7f31a2800640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f96eb0 0x558c99f949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a2800640 1 -- 172.21.15.44:0/2409321174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f96eb0 msgr2=0x558c99f949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a2800640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f96eb0 0x558c99f949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a2800640 1 -- 172.21.15.44:0/2409321174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 msgr2=0x558c99f94480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a2800640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f94480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a2800640 1 -- 172.21.15.44:0/2409321174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c99bfd3f0 con 0x558c99f84fb0 2024-04-20T11:32:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a1400640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f94480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31a2800640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f93f40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f319c005da0 tx=0x7f319c0055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fb1f25a787628e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f3183e00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f3183e00640 1 -- 172.21.15.44:0/2409321174 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c050020 con 0x558c99f84fb0 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f319c035d40 con 0x558c99f84fb0 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f3183e00640 1 -- 172.21.15.44:0/2409321174 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f319c0346b0 con 0x558c99f84fb0 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f3183e00640 1 -- 172.21.15.44:0/2409321174 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f319c044070 con 0x558c99f84fb0 2024-04-20T11:32:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31abe06740 10 client.28139 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.066+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558c99bbd0f0 con 0x558c99f84fb0 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f3183e00640 1 -- 172.21.15.44:0/2409321174 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f319c035d40 con 0x558c99f84fb0 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f3183e00640 1 -- 172.21.15.44:0/2409321174 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f319c02c5c0 con 0x558c99f84fb0 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31abe06740 10 client.28139 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31abe06740 10 client.28139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31abe06740 20 client.28139 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31abe06740 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c99fa7100 0x558c99fa94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31abe06740 4 client.28139 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c99ef8b70 con 0x558c99fa7100 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.067+0000 7f31a1400640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c99fa7100 0x558c99fa94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.068+0000 7f31a1400640 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c99fa7100 0x558c99fa94f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a80999a77c7d997 server_cookie=817964e00bb8191a in_seq=0 out_seq=0 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.068+0000 7f3183e00640 10 client.28139 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f3183e00640 1 -- 172.21.15.44:0/2409321174 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c99ef8b70 con 0x558c99fa7100 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f3183e00640 10 client.28139 handle_command_reply: tid=0 2024-04-20T11:32:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 client.28139 shutdown 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 msgr2=0x558c99f93f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f93f40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f319c005da0 tx=0x7f319c0055f0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 shutdown_connections 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c99fa7100 0x558c99fa94f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f96eb0 0x558c99f949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f94480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 --2- 172.21.15.44:0/2409321174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f84fb0 0x558c99f93f40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 >> 172.21.15.44:0/2409321174 conn(0x558c99f7f8d0 msgr2=0x558c99f81620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f3183e00640 0 client.28139 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 shutdown_connections 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.069+0000 7f31abe06740 1 -- 172.21.15.44:0/2409321174 wait complete. 2024-04-20T11:32:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.070+0000 7f31abe06740 20 client.28139 trim_cache size 0 max 16384 2024-04-20T11:32:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.089+0000 7f31abe06740 1 Processor -- start 2024-04-20T11:32:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.089+0000 7f31abe06740 1 -- start start 2024-04-20T11:32:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31abe06740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99f96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31abe06740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31abe06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f96eb0 0x558c99f911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31abe06740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41aba40 con 0x558c99f85960 2024-04-20T11:32:23.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31abe06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41ad580 con 0x558c99f96eb0 2024-04-20T11:32:23.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31abe06740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41ac110 con 0x558c99f84fb0 2024-04-20T11:32:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f947e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34216/0 (socket says 172.21.15.44:34216) 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a1400640 1 -- 172.21.15.44:0/1326083787 learned_addr learned my addr 172.21.15.44:0/1326083787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a2800640 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f96eb0 0x558c99f911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a2800640 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99f96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a1400640 1 -- 172.21.15.44:0/1326083787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 msgr2=0x558c99f96670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.090+0000 7f31a1400640 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99f96670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f31a1400640 1 -- 172.21.15.44:0/1326083787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f96eb0 msgr2=0x558c99f911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f31a1400640 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f96eb0 0x558c99f911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f31a1400640 1 -- 172.21.15.44:0/1326083787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c99f7f400 con 0x558c99f85960 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f31a1400640 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f947e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3190012940 tx=0x7f3190005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c34195b700ba33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f318ea00640 1 -- 172.21.15.44:0/1326083787 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190004030 con 0x558c99f85960 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f318ea00640 1 -- 172.21.15.44:0/1326083787 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3190026030 con 0x558c99f85960 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.091+0000 7f318ea00640 1 -- 172.21.15.44:0/1326083787 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31900224e0 con 0x558c99f85960 2024-04-20T11:32:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 -- 172.21.15.44:0/1326083787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 msgr2=0x558c99f947e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f947e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3190012940 tx=0x7f3190005f90 comp rx=0 tx=0).stop 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 -- 172.21.15.44:0/1326083787 shutdown_connections 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f96eb0 0x558c99f911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f85960 0x558c99f947e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 --2- 172.21.15.44:0/1326083787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99f96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 -- 172.21.15.44:0/1326083787 >> 172.21.15.44:0/1326083787 conn(0x558c99f7f8d0 msgr2=0x558c99f92f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 -- 172.21.15.44:0/1326083787 shutdown_connections 2024-04-20T11:32:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.092+0000 7f31abe06740 1 -- 172.21.15.44:0/1326083787 wait complete. 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 -- start start 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 Processor -- start 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99fa9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f85960 0x558c99fa9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c9a0052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41aceb0 con 0x558c99f96eb0 2024-04-20T11:32:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41ac7e0 con 0x558c99f85960 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.093+0000 7f31abe06740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31a41a9a70 con 0x558c99f84fb0 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31abe06740 10 client.0 _ll_register_callbacks cb 0x7f3190009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99fa9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c9a0052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99fa9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:48386/0 (socket says 172.21.15.44:48386) 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 -- 172.21.15.44:0/767126705 learned_addr learned my addr 172.21.15.44:0/767126705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a1400640 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f85960 0x558c99fa9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 -- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f85960 msgr2=0x558c99fa9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f85960 0x558c99fa9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 -- 172.21.15.44:0/767126705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 msgr2=0x558c9a0052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c9a0052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.094+0000 7f31a2800640 1 -- 172.21.15.44:0/767126705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c9a005890 con 0x558c99f84fb0 2024-04-20T11:32:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f31a2800640 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99fa9400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f319c02cbd0 tx=0x7f319c005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c42f66aaf74bd0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f3183e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f3183e00640 1 -- 172.21.15.44:0/767126705 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c050040 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f3183e00640 1 -- 172.21.15.44:0/767126705 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f319c0048c0 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3190026030 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f3183e00640 1 -- 172.21.15.44:0/767126705 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f319c039070 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f31abe06740 10 client.28153 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.095+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558c9a00cb90 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f3183e00640 1 -- 172.21.15.44:0/767126705 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3190026030 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f3183e00640 1 -- 172.21.15.44:0/767126705 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f319c003b20 con 0x558c99f84fb0 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31abe06740 10 client.28153 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31abe06740 10 client.28153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31abe06740 20 client.28153 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31abe06740 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c9a00d200 0x558c9a00f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31abe06740 4 client.28153 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:32:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c99ef29e0 con 0x558c9a00d200 2024-04-20T11:32:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.096+0000 7f31a1400640 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c9a00d200 0x558c9a00f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31a1400640 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c9a00d200 0x558c9a00f5f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a083d563dc504b19 server_cookie=2024b4e6932ee266 in_seq=0 out_seq=0 2024-04-20T11:32:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f3183e00640 10 client.28153 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f3183e00640 1 -- 172.21.15.44:0/767126705 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c99ef29e0 con 0x558c9a00d200 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f3183e00640 10 client.28153 handle_command_reply: tid=0 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 client.28153 shutdown 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 msgr2=0x558c99fa9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99fa9400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f319c02cbd0 tx=0x7f319c005710 comp rx=0 tx=0).stop 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 shutdown_connections 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x558c9a00d200 0x558c9a00f5f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c99f96eb0 0x558c9a0052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c99f85960 0x558c99fa9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 --2- 172.21.15.44:0/767126705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558c99f84fb0 0x558c99fa9400 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 >> 172.21.15.44:0/767126705 conn(0x558c99f7f8d0 msgr2=0x558c99f7e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.097+0000 7f3183e00640 0 client.28153 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 shutdown_connections 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31abe06740 1 -- 172.21.15.44:0/767126705 wait complete. 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31abe06740 20 client.28153 trim_cache size 0 max 16384 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stdout: "c60319c5-1a08-45c6-9e99-ff015d26abc8": { 2024-04-20T11:32:23.102 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-04-20T11:32:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "c60319c5-1a08-45c6-9e99-ff015d26abc8", 2024-04-20T11:32:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-04-20T11:32:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:23.103 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31880a3d20 msgr2=0x7f31880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31880a3d20 0x7f31880a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x558c99f80b60 tx=0x7f31940d6530 comp rx=0 tx=0).stop 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 msgr2=0x7f31a41a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 0x7f31a41a0810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f319400c420 tx=0x7f319400c8e0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 shutdown_connections 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a410c540 0x7f31a419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4103b20 0x7f31a41a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f31880a3d20 0x7f31880a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 --2- 172.21.15.44:0/3306130064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31a4103200 0x7f31a41a0810 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.098+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 >> 172.21.15.44:0/3306130064 conn(0x7f31a4077fc0 msgr2=0x7f31a4102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.099+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 shutdown_connections 2024-04-20T11:32:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:23.099+0000 7f31a9200640 1 -- 172.21.15.44:0/3306130064 wait complete. 2024-04-20T11:32:23.116 INFO:tasks.cephfs.filesystem:scrub status for tag:c60319c5-1a08-45c6-9e99-ff015d26abc8 - {'path': '/', 'tag': 'c60319c5-1a08-45c6-9e99-ff015d26abc8', 'options': 'recursive,force'} 2024-04-20T11:32:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:23 smithi165 ceph-mon[21975]: pgmap v304: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:32:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:23 smithi165 ceph-mon[21975]: scrub queued for path: / 2024-04-20T11:32:23.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:23 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:32:23.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:23 smithi165 ceph-mon[21975]: scrub summary: active paths [/] 2024-04-20T11:32:23.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:23 smithi165 ceph-mon[21975]: scrub summary: idle+waiting paths [/] 2024-04-20T11:32:24.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:23 smithi078 ceph-mon[21464]: pgmap v304: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:32:24.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:23 smithi078 ceph-mon[21464]: scrub queued for path: / 2024-04-20T11:32:24.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:23 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:32:24.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:23 smithi078 ceph-mon[21464]: scrub summary: active paths [/] 2024-04-20T11:32:24.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:23 smithi078 ceph-mon[21464]: scrub summary: idle+waiting paths [/] 2024-04-20T11:32:24.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:23 smithi044 ceph-mon[16855]: pgmap v304: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:32:24.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:23 smithi044 ceph-mon[16855]: scrub queued for path: / 2024-04-20T11:32:24.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:23 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:32:24.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:23 smithi044 ceph-mon[16855]: scrub summary: active paths [/] 2024-04-20T11:32:24.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:23 smithi044 ceph-mon[16855]: scrub summary: idle+waiting paths [/] 2024-04-20T11:32:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:25 smithi165 ceph-mon[21975]: pgmap v305: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-20T11:32:26.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:25 smithi078 ceph-mon[21464]: pgmap v305: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-20T11:32:26.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:25 smithi044 ceph-mon[16855]: pgmap v305: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-20T11:32:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:27 smithi165 ceph-mon[21975]: pgmap v306: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:32:27.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:27 smithi165 ceph-mon[21975]: scrub summary: idle 2024-04-20T11:32:28.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:27 smithi078 ceph-mon[21464]: pgmap v306: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:32:28.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:27 smithi078 ceph-mon[21464]: scrub summary: idle 2024-04-20T11:32:28.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:27 smithi044 ceph-mon[16855]: pgmap v306: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:32:28.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:27 smithi044 ceph-mon[16855]: scrub summary: idle 2024-04-20T11:32:30.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:29 smithi078 ceph-mon[21464]: pgmap v307: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:32:30.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:29 smithi044 ceph-mon[16855]: pgmap v307: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:32:30.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:29 smithi165 ceph-mon[21975]: pgmap v307: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:32:32.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:31 smithi078 ceph-mon[21464]: pgmap v308: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:32.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:31 smithi044 ceph-mon[16855]: pgmap v308: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:32.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:31 smithi165 ceph-mon[21975]: pgmap v308: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:33.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:32 smithi078 ceph-mon[21464]: pgmap v309: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:33.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:32 smithi044 ceph-mon[16855]: pgmap v309: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:33.229 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:32 smithi165 ceph-mon[21975]: pgmap v309: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:35.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:35 smithi078 ceph-mon[21464]: pgmap v310: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:35.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:35 smithi044 ceph-mon[16855]: pgmap v310: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:35.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:35 smithi165 ceph-mon[21975]: pgmap v310: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:37 smithi044 ceph-mon[16855]: pgmap v311: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:37.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:37 smithi165 ceph-mon[21975]: pgmap v311: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:37 smithi078 ceph-mon[21464]: pgmap v311: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-20T11:32:39.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:39 smithi165 ceph-mon[21975]: pgmap v312: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:39.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:39 smithi078 ceph-mon[21464]: pgmap v312: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:39 smithi044 ceph-mon[16855]: pgmap v312: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-20T11:32:41.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:41 smithi165 ceph-mon[21975]: pgmap v313: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:41 smithi078 ceph-mon[21464]: pgmap v313: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:41.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:41 smithi044 ceph-mon[16855]: pgmap v313: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:43.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:43 smithi165 ceph-mon[21975]: pgmap v314: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:32:43.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:43 smithi078 ceph-mon[21464]: pgmap v314: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:32:43.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:43 smithi044 ceph-mon[16855]: pgmap v314: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:32:45.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:45 smithi165 ceph-mon[21975]: pgmap v315: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:45.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:45 smithi078 ceph-mon[21464]: pgmap v315: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:45.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:45 smithi044 ceph-mon[16855]: pgmap v315: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:47.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:47 smithi165 ceph-mon[21975]: pgmap v316: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:47.865 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:47 smithi044 ceph-mon[16855]: pgmap v316: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:47.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T11:32:47.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:47 smithi078 ceph-mon[21464]: pgmap v316: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:48.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 -- 172.21.15.44:0/3770167088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d41084a0 msgr2=0x7f60d41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 --2- 172.21.15.44:0/3770167088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d41084a0 0x7f60d41088e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009930 tx=0x7f60c002bc30 comp rx=0 tx=0).stop 2024-04-20T11:32:48.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 -- 172.21.15.44:0/3770167088 shutdown_connections 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 --2- 172.21.15.44:0/3770167088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60d4108e20 0x7f60d4115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 --2- 172.21.15.44:0/3770167088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d41084a0 0x7f60d41088e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 --2- 172.21.15.44:0/3770167088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d410e430 0x7f60d410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.084+0000 7f60daa00640 1 -- 172.21.15.44:0/3770167088 >> 172.21.15.44:0/3770167088 conn(0x7f60d4077d40 msgr2=0x7f60d4106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.085+0000 7f60daa00640 1 -- 172.21.15.44:0/3770167088 shutdown_connections 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.085+0000 7f60daa00640 1 -- 172.21.15.44:0/3770167088 wait complete. 2024-04-20T11:32:48.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.086+0000 7f60daa00640 1 Processor -- start 2024-04-20T11:32:48.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.086+0000 7f60daa00640 1 -- start start 2024-04-20T11:32:48.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60daa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 0x7f60d419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60daa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60d4108e20 0x7f60d419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d410e430 0x7f60d41a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60daa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4118870 con 0x7f60d4108e20 2024-04-20T11:32:48.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60daa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41186f0 con 0x7f60d41084a0 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60daa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41189f0 con 0x7f60d410e430 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60d9600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 0x7f60d419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60d8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60d4108e20 0x7f60d419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60d9600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 0x7f60d419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:53250/0 (socket says 172.21.15.44:53250) 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60d9600640 1 -- 172.21.15.44:0/360057934 learned_addr learned my addr 172.21.15.44:0/360057934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.087+0000 7f60da000640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d410e430 0x7f60d41a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.088+0000 7f60d9600640 1 -- 172.21.15.44:0/360057934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d410e430 msgr2=0x7f60d41a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.088+0000 7f60d9600640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d410e430 0x7f60d41a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.088+0000 7f60d9600640 1 -- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60d4108e20 msgr2=0x7f60d419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.088+0000 7f60d9600640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60d4108e20 0x7f60d419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.088+0000 7f60d9600640 1 -- 172.21.15.44:0/360057934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d41a3d40 con 0x7f60d41084a0 2024-04-20T11:32:48.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.088+0000 7f60d9600640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 0x7f60d419ed50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60c800ebc0 tx=0x7f60c800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.089+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8010630 con 0x7f60d41084a0 2024-04-20T11:32:48.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.089+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f60c8010cd0 con 0x7f60d41084a0 2024-04-20T11:32:48.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.089+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d41a4030 con 0x7f60d41084a0 2024-04-20T11:32:48.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.089+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8018cf0 con 0x7f60d41084a0 2024-04-20T11:32:48.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.089+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d41ab8c0 con 0x7f60d41084a0 2024-04-20T11:32:48.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.090+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d4050540 con 0x7f60d41084a0 2024-04-20T11:32:48.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.092+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f60c80107d0 con 0x7f60d41084a0 2024-04-20T11:32:48.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.094+0000 7f60d2000640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f60bc0a3d10 0x7f60bc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.094+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f60c80d0e60 con 0x7f60d41084a0 2024-04-20T11:32:48.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.094+0000 7f60d8c00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f60bc0a3d10 0x7f60bc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.095+0000 7f60d8c00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f60bc0a3d10 0x7f60bc0a61d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f60c0000c00 tx=0x7f60c0005fd0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.098+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f60c809ce90 con 0x7f60d41084a0 2024-04-20T11:32:48.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.332+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f60d40623a0 con 0x7f60bc0a3d10 2024-04-20T11:32:48.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.348+0000 7f60d2000640 1 -- 172.21.15.44:0/360057934 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f60d40623a0 con 0x7f60bc0a3d10 2024-04-20T11:32:48.348 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T11:22:46", "first": "2024-04-20T11:23:00", "last": "2024-04-20T11:32:00", "last_pruned": "2024-04-20T11:32:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-20T11:32:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.351+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f60bc0a3d10 msgr2=0x7f60bc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.351+0000 7f60daa00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f60bc0a3d10 0x7f60bc0a61d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f60c0000c00 tx=0x7f60c0005fd0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.351+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 msgr2=0x7f60d419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.351+0000 7f60daa00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 0x7f60d419ed50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60c800ebc0 tx=0x7f60c800c6a0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 shutdown_connections 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60d410e430 0x7f60d41a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f60bc0a3d10 0x7f60bc0a61d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60d4108e20 0x7f60d419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 --2- 172.21.15.44:0/360057934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d41084a0 0x7f60d419ed50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 >> 172.21.15.44:0/360057934 conn(0x7f60d4077d40 msgr2=0x7f60d4078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 shutdown_connections 2024-04-20T11:32:48.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.352+0000 7f60daa00640 1 -- 172.21.15.44:0/360057934 wait complete. 2024-04-20T11:32:48.371 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T11:32:48.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 -- 172.21.15.44:0/378321811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41101084a0 msgr2=0x7f41101088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 --2- 172.21.15.44:0/378321811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41101084a0 0x7f41101088e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009cd0 tx=0x7f40fc02c0f0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 -- 172.21.15.44:0/378321811 shutdown_connections 2024-04-20T11:32:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 --2- 172.21.15.44:0/378321811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 0x7f4110115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 --2- 172.21.15.44:0/378321811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41101084a0 0x7f41101088e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 --2- 172.21.15.44:0/378321811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f411010e430 0x7f411010e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 -- 172.21.15.44:0/378321811 >> 172.21.15.44:0/378321811 conn(0x7f4110077d40 msgr2=0x7f4110106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 -- 172.21.15.44:0/378321811 shutdown_connections 2024-04-20T11:32:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.583+0000 7f410f200640 1 -- 172.21.15.44:0/378321811 wait complete. 2024-04-20T11:32:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.584+0000 7f410f200640 1 Processor -- start 2024-04-20T11:32:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.584+0000 7f410f200640 1 -- start start 2024-04-20T11:32:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 0x7f411019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 0x7f411019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f411010e430 0x7f41101a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41101188c0 con 0x7f411010e430 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4110118740 con 0x7f41101084a0 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4110118a40 con 0x7f4110108e20 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 0x7f411019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 0x7f411019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54318/0 (socket says 172.21.15.44:54318) 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410de00640 1 -- 172.21.15.44:0/249187261 learned_addr learned my addr 172.21.15.44:0/249187261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f411010e430 0x7f41101a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.585+0000 7f410d400640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 0x7f411019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410de00640 1 -- 172.21.15.44:0/249187261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 msgr2=0x7f411019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410de00640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 0x7f411019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410de00640 1 -- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f411010e430 msgr2=0x7f41101a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410de00640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f411010e430 0x7f41101a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410de00640 1 -- 172.21.15.44:0/249187261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41101a3d10 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410d400640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 0x7f411019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410de00640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 0x7f411019ed20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f410000ea60 tx=0x7f410000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410000cde0 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41101a4000 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41000035f0 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100010720 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.586+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101ab890 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.588+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4110050540 con 0x7f41101084a0 2024-04-20T11:32:48.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.589+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f41000037a0 con 0x7f41101084a0 2024-04-20T11:32:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.591+0000 7f4106a00640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f40f40a3d10 0x7f40f40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:48.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.591+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f41000d0bf0 con 0x7f41101084a0 2024-04-20T11:32:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.591+0000 7f410d400640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f40f40a3d10 0x7f40f40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:48.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.592+0000 7f410d400640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f40f40a3d10 0x7f40f40a61d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006010 tx=0x7f40fc005c50 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:48.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.595+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f410009d270 con 0x7f41101084a0 2024-04-20T11:32:48.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.836+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 --> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f41100623a0 con 0x7f40f40a3d10 2024-04-20T11:32:48.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.849+0000 7f4106a00640 1 -- 172.21.15.44:0/249187261 <== mgr.15174 v2:172.21.15.44:6800/4029191467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f41100623a0 con 0x7f40f40a3d10 2024-04-20T11:32:48.848 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1m 6m3h 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f40f40a3d10 msgr2=0x7f40f40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f40f40a3d10 0x7f40f40a61d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006010 tx=0x7f40fc005c50 comp rx=0 tx=0).stop 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 msgr2=0x7f411019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 0x7f411019ed20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f410000ea60 tx=0x7f410000ef20 comp rx=0 tx=0).stop 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 shutdown_connections 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f40f40a3d10 0x7f40f40a61d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f411010e430 0x7f41101a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4110108e20 0x7f411019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 --2- 172.21.15.44:0/249187261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101084a0 0x7f411019ed20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 >> 172.21.15.44:0/249187261 conn(0x7f4110077d40 msgr2=0x7f4110078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 shutdown_connections 2024-04-20T11:32:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:48.852+0000 7f410f200640 1 -- 172.21.15.44:0/249187261 wait complete. 2024-04-20T11:32:48.872 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-20T11:32:48.883 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-20T11:32:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:49 smithi165 ceph-mon[21975]: from='client.28151 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:49.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:49 smithi165 ceph-mon[21975]: pgmap v317: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:32:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:49 smithi078 ceph-mon[21464]: from='client.28151 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:49 smithi078 ceph-mon[21464]: pgmap v317: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:32:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:49 smithi044 ceph-mon[16855]: from='client.28151 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:49.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:49 smithi044 ceph-mon[16855]: pgmap v317: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:32:50.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:50 smithi078 ceph-mon[21464]: from='client.28157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:50.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:50 smithi044 ceph-mon[16855]: from='client.28157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:50.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:50 smithi165 ceph-mon[21975]: from='client.28157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T11:32:51.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:51 smithi078 ceph-mon[21464]: pgmap v318: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:51.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:51 smithi044 ceph-mon[16855]: pgmap v318: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:51.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:51 smithi165 ceph-mon[21975]: pgmap v318: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:32:53.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T11:32:53.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T11:32:53.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 -- 172.21.15.44:0/545126220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0104280 msgr2=0x7f12e0104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 --2- 172.21.15.44:0/545126220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0104280 0x7f12e0104660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009860 tx=0x7f12d002bb70 comp rx=0 tx=0).stop 2024-04-20T11:32:53.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 -- 172.21.15.44:0/545126220 shutdown_connections 2024-04-20T11:32:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 --2- 172.21.15.44:0/545126220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e0109350 0x7f12e0110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 --2- 172.21.15.44:0/545126220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12e0104c30 0x7f12e0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 --2- 172.21.15.44:0/545126220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0104280 0x7f12e0104660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 -- 172.21.15.44:0/545126220 >> 172.21.15.44:0/545126220 conn(0x7f12e00ffaa0 msgr2=0x7f12e0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 -- 172.21.15.44:0/545126220 shutdown_connections 2024-04-20T11:32:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.349+0000 7f12e4a00640 1 -- 172.21.15.44:0/545126220 wait complete. 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.350+0000 7f12e4a00640 1 Processor -- start 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.350+0000 7f12e4a00640 1 -- start start 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12e4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12e0104280 0x7f12e01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12e4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 0x7f12e01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12e4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e019a840 0x7f12e019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12e4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0114520 con 0x7f12e019a840 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12e4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01143a0 con 0x7f12e0109350 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12e4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01146a0 con 0x7f12e0104280 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 0x7f12e01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 0x7f12e01a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54334/0 (socket says 172.21.15.44:54334) 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12d7e00640 1 -- 172.21.15.44:0/1168246881 learned_addr learned my addr 172.21.15.44:0/1168246881 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e019a840 0x7f12e019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.351+0000 7f12df400640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12e0104280 0x7f12e01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12d7e00640 1 -- 172.21.15.44:0/1168246881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12e0104280 msgr2=0x7f12e01a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12d7e00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12e0104280 0x7f12e01a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12d7e00640 1 -- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e019a840 msgr2=0x7f12e019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12d7e00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e019a840 0x7f12e019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12d7e00640 1 -- 172.21.15.44:0/1168246881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e019b3d0 con 0x7f12e0109350 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12d7e00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 0x7f12e01a0c10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9c0 tx=0x7f12c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12dcc00640 1 -- 172.21.15.44:0/1168246881 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c800cd80 con 0x7f12e0109350 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e019b6c0 con 0x7f12e0109350 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12dcc00640 1 -- 172.21.15.44:0/1168246881 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12c8005520 con 0x7f12e0109350 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12dcc00640 1 -- 172.21.15.44:0/1168246881 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8010690 con 0x7f12e0109350 2024-04-20T11:32:53.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.352+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e01a78a0 con 0x7f12e0109350 2024-04-20T11:32:53.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.355+0000 7f12e8dd4740 1 Processor -- start 2024-04-20T11:32:53.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 -- start start 2024-04-20T11:32:53.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188125e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x561881267340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x561881269c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5618811c4150 con 0x561881267880 2024-04-20T11:32:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56188116a370 con 0x56188125eca0 2024-04-20T11:32:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5618811c0110 con 0x56188125e380 2024-04-20T11:32:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12dcc00640 1 -- 172.21.15.44:0/1168246881 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f12c80046c0 con 0x7f12e0109350 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x561881267340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188125e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x561881267340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54340/0 (socket says 172.21.15.44:54340) 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188125e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44248/0 (socket says 172.21.15.44:44248) 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.356+0000 7f12df400640 1 -- 172.21.15.44:0/1989040648 learned_addr learned my addr 172.21.15.44:0/1989040648 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12d7e00640 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x561881269c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12dfe00640 1 -- 172.21.15.44:0/1989040648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 msgr2=0x561881267340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x561881267340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12dfe00640 1 -- 172.21.15.44:0/1989040648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 msgr2=0x561881269c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x561881269c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12dfe00640 1 -- 172.21.15.44:0/1989040648 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561880fa07d0 con 0x56188125e380 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188125e760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005bd0 tx=0x7f12d80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9a1c3fe102b28e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.357+0000 7f12cfe00640 1 -- 172.21.15.44:0/1989040648 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8004b20 con 0x56188125e380 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12cfe00640 1 -- 172.21.15.44:0/1989040648 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d8004cc0 con 0x56188125e380 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12cfe00640 1 -- 172.21.15.44:0/1989040648 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8004020 con 0x56188125e380 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12dcc00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3d20 0x7f12c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12dcc00640 1 -- 172.21.15.44:0/1168246881 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f12c80d0b40 con 0x7f12e0109350 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12df400640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3d20 0x7f12c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1989040648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 msgr2=0x56188125e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188125e760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005bd0 tx=0x7f12d80095c0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1989040648 shutdown_connections 2024-04-20T11:32:53.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x561881269c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x561881267340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1989040648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188125e760 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.358+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1989040648 >> 172.21.15.44:0/1989040648 conn(0x561881258ab0 msgr2=0x561881258e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.359+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1989040648 shutdown_connections 2024-04-20T11:32:53.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.359+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1989040648 wait complete. 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.359+0000 7f12df400640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3d20 0x7f12c00a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000c00 tx=0x7f12d00056d0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.359+0000 7f12e8dd4740 1 -- start start 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.359+0000 7f12e8dd4740 1 Processor -- start 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x5618812648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125eca0 0x561881264e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56188126a670 con 0x56188125eca0 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56188117d740 con 0x561881267880 2024-04-20T11:32:53.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56188126a7f0 con 0x56188125e380 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12e8dd4740 10 client.0 _ll_register_callbacks cb 0x561881178b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125eca0 0x561881264e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x5618812648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54356/0 (socket says 172.21.15.44:54356) 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.360+0000 7f12dfe00640 1 -- 172.21.15.44:0/338069203 learned_addr learned my addr 172.21.15.44:0/338069203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12dfe00640 1 -- 172.21.15.44:0/338069203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 msgr2=0x5618812648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12dfe00640 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x5618812648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12dfe00640 1 -- 172.21.15.44:0/338069203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125eca0 msgr2=0x561881264e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12dfe00640 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125eca0 0x561881264e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12dfe00640 1 -- 172.21.15.44:0/338069203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561880ed63f0 con 0x561881267880 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12d7e00640 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125eca0 0x561881264e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:53.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12dfe00640 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009de0 tx=0x7f12d800ddc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4e175482835db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12ccc00640 1 -- 172.21.15.44:0/338069203 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8027020 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12ccc00640 1 -- 172.21.15.44:0/338069203 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f12d8023460 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.361+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d8004cc0 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12ccc00640 1 -- 172.21.15.44:0/338069203 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12d8032050 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 10 client.28169 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561880e960f0 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12ccc00640 1 -- 172.21.15.44:0/338069203 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d8004cc0 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12ccc00640 1 -- 172.21.15.44:0/338069203 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f12d800b040 con 0x561881267880 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 10 client.28169 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 10 client.28169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 20 client.28169 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561881277bd0 0x561881279fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 4 client.28169 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:53.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.362+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618811d1b70 con 0x561881277bd0 2024-04-20T11:32:53.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.363+0000 7f12df400640 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561881277bd0 0x561881279fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.363+0000 7f12df400640 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561881277bd0 0x561881279fc0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d3b92d86f92b10 server_cookie=226f09cac61c1aa7 in_seq=0 out_seq=0 2024-04-20T11:32:53.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.363+0000 7f12ccc00640 10 client.28169 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12ccc00640 1 -- 172.21.15.44:0/338069203 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618811d1b70 con 0x561881277bd0 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12ccc00640 10 client.28169 handle_command_reply: tid=0 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 client.28169 shutdown 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 msgr2=0x561881265360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009de0 tx=0x7f12d800ddc0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 shutdown_connections 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x561881277bd0 0x561881279fc0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265360 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125eca0 0x561881264e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/338069203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x5618812648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12ccc00640 0 client.28169 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:53.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.364+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 >> 172.21.15.44:0/338069203 conn(0x561881258ab0 msgr2=0x56188125a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.365+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 shutdown_connections 2024-04-20T11:32:53.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.365+0000 7f12e8dd4740 1 -- 172.21.15.44:0/338069203 wait complete. 2024-04-20T11:32:53.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.365+0000 7f12e8dd4740 20 client.28169 trim_cache size 0 max 16384 2024-04-20T11:32:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 Processor -- start 2024-04-20T11:32:53.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 -- start start 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125e380 0x561881273530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125eca0 0x561881257990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01afe10 con 0x56188125e380 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b1950 con 0x561881267880 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.384+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b04e0 con 0x56188125eca0 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125e380 0x561881273530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54368/0 (socket says 172.21.15.44:54368) 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 -- 172.21.15.44:0/1172144797 learned_addr learned my addr 172.21.15.44:0/1172144797 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:53.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12d7e00640 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125eca0 0x561881257990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 -- 172.21.15.44:0/1172144797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125eca0 msgr2=0x561881257990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125eca0 0x561881257990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 -- 172.21.15.44:0/1172144797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125e380 msgr2=0x561881273530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125e380 0x561881273530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12dfe00640 1 -- 172.21.15.44:0/1172144797 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56188125ead0 con 0x561881267880 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.385+0000 7f12df400640 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125e380 0x561881273530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:53.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.386+0000 7f12dfe00640 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f12d8017b10 tx=0x7f12d8009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0969f23defcd2db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.386+0000 7f12cf400640 1 -- 172.21.15.44:0/1172144797 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8027030 con 0x561881267880 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.386+0000 7f12cf400640 1 -- 172.21.15.44:0/1172144797 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d800d980 con 0x561881267880 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.386+0000 7f12cf400640 1 -- 172.21.15.44:0/1172144797 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8004030 con 0x561881267880 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1172144797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 msgr2=0x561881265180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f12d8017b10 tx=0x7f12d8009ab0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1172144797 shutdown_connections 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561881267880 0x561881265180 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125eca0 0x561881257990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/1172144797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56188125e380 0x561881273530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1172144797 >> 172.21.15.44:0/1172144797 conn(0x561881258ab0 msgr2=0x561881257280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1172144797 shutdown_connections 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 -- 172.21.15.44:0/1172144797 wait complete. 2024-04-20T11:32:53.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 -- start start 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.387+0000 7f12e8dd4740 1 Processor -- start 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188126fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x56188126b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x56188126b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b1280 con 0x561881267880 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b0bb0 con 0x56188125eca0 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01ade00 con 0x56188125e380 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12e8dd4740 10 client.0 _ll_register_callbacks cb 0x7f12d8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x56188126b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x56188126b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54358/0 (socket says 172.21.15.44:54358) 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 -- 172.21.15.44:0/2520174351 learned_addr learned my addr 172.21.15.44:0/2520174351 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12d7e00640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x56188126b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12df400640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188126fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 -- 172.21.15.44:0/2520174351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 msgr2=0x56188126fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188126fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 -- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 msgr2=0x56188126b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x56188126b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.388+0000 7f12dfe00640 1 -- 172.21.15.44:0/2520174351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188126ba80 con 0x561881267880 2024-04-20T11:32:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12dfe00640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x56188126b540 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005d50 tx=0x7f12d800dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a005d347bccf5a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:53.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12ccc00640 1 -- 172.21.15.44:0/2520174351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8027070 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12ccc00640 1 -- 172.21.15.44:0/2520174351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f12d8017b90 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d800d980 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12ccc00640 1 -- 172.21.15.44:0/2520174351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12d8046050 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12e8dd4740 10 client.18066 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.389+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5618812ddb90 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12ccc00640 1 -- 172.21.15.44:0/2520174351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12d800d980 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12ccc00640 1 -- 172.21.15.44:0/2520174351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f12d800b070 con 0x561881267880 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12e8dd4740 10 client.18066 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12e8dd4740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12e8dd4740 20 client.18066 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:53.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5618812de1f0 0x5618812e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12e8dd4740 4 client.18066 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561881254170 con 0x5618812de1f0 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.390+0000 7f12df400640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5618812de1f0 0x5618812e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12df400640 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5618812de1f0 0x5618812e0610 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbb571e15be4811 server_cookie=dcaf4ee4000eecc3 in_seq=0 out_seq=0 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12ccc00640 10 client.18066 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12ccc00640 1 -- 172.21.15.44:0/2520174351 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561881254170 con 0x5618812de1f0 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12ccc00640 10 client.18066 handle_command_reply: tid=0 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12e8dd4740 1 client.18066 shutdown 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 msgr2=0x56188126b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x56188126b540 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005d50 tx=0x7f12d800dba0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.391+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 shutdown_connections 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x5618812de1f0 0x5618812e0610 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561881267880 0x56188126b540 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56188125eca0 0x56188126b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 --2- 172.21.15.44:0/2520174351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56188125e380 0x56188126fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12ccc00640 0 client.18066 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 >> 172.21.15.44:0/2520174351 conn(0x561881258ab0 msgr2=0x561881259ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 shutdown_connections 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 1 -- 172.21.15.44:0/2520174351 wait complete. 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e8dd4740 20 client.18066 trim_cache size 0 max 16384 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-04-20T11:32:53.396 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:32:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3d20 msgr2=0x7f12c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e4a00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3d20 0x7f12c00a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000c00 tx=0x7f12d00056d0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.392+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 msgr2=0x7f12e01a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 0x7f12e01a0c10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9c0 tx=0x7f12c800ee80 comp rx=0 tx=0).stop 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 shutdown_connections 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f12c00a3d20 0x7f12c00a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12e019a840 0x7f12e019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0109350 0x7f12e01a0c10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 --2- 172.21.15.44:0/1168246881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12e0104280 0x7f12e01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 >> 172.21.15.44:0/1168246881 conn(0x7f12e00ffaa0 msgr2=0x7f12e0101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 shutdown_connections 2024-04-20T11:32:53.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.393+0000 7f12e4a00640 1 -- 172.21.15.44:0/1168246881 wait complete. 2024-04-20T11:32:53.411 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T11:32:53.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:32:53.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.624+0000 7fa157200640 1 -- 172.21.15.44:0/3220668027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108400 msgr2=0x7fa158108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.624+0000 7fa157200640 1 --2- 172.21.15.44:0/3220668027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108400 0x7fa158108860 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa140009900 tx=0x7fa14002bcd0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 -- 172.21.15.44:0/3220668027 shutdown_connections 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 --2- 172.21.15.44:0/3220668027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa158108da0 0x7fa158115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 --2- 172.21.15.44:0/3220668027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108400 0x7fa158108860 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 --2- 172.21.15.44:0/3220668027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15810e470 0x7fa15810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 -- 172.21.15.44:0/3220668027 >> 172.21.15.44:0/3220668027 conn(0x7fa158077ae0 msgr2=0x7fa158079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 -- 172.21.15.44:0/3220668027 shutdown_connections 2024-04-20T11:32:53.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.625+0000 7fa157200640 1 -- 172.21.15.44:0/3220668027 wait complete. 2024-04-20T11:32:53.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 Processor -- start 2024-04-20T11:32:53.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 -- start start 2024-04-20T11:32:53.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 0x7fa1581a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108da0 0x7fa1581a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15810e470 0x7fa15819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581188b0 con 0x7fa158108400 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa158118730 con 0x7fa158108da0 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.626+0000 7fa157200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa158118a30 con 0x7fa15810e470 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 0x7fa1581a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 0x7fa1581a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54374/0 (socket says 172.21.15.44:54374) 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 -- 172.21.15.44:0/4204259757 learned_addr learned my addr 172.21.15.44:0/4204259757 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa14d400640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108da0 0x7fa1581a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa156800640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15810e470 0x7fa15819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 -- 172.21.15.44:0/4204259757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15810e470 msgr2=0x7fa15819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15810e470 0x7fa15819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 -- 172.21.15.44:0/4204259757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108da0 msgr2=0x7fa1581a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108da0 0x7fa1581a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 -- 172.21.15.44:0/4204259757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15819f230 con 0x7fa158108400 2024-04-20T11:32:53.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.627+0000 7fa155e00640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 0x7fa1581a4900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e9f0 tx=0x7fa14800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.628+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14800cd80 con 0x7fa158108400 2024-04-20T11:32:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.628+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15819f520 con 0x7fa158108400 2024-04-20T11:32:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.628+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa148010930 con 0x7fa158108400 2024-04-20T11:32:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.628+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148018b00 con 0x7fa158108400 2024-04-20T11:32:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.628+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1581ab7c0 con 0x7fa158108400 2024-04-20T11:32:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.629+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1580690f0 con 0x7fa158108400 2024-04-20T11:32:53.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.631+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa148018ca0 con 0x7fa158108400 2024-04-20T11:32:53.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.632+0000 7fa14f400640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa13c0a3d20 0x7fa13c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:53.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.633+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fa1480d0ba0 con 0x7fa158108400 2024-04-20T11:32:53.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.633+0000 7fa14d400640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa13c0a3d20 0x7fa13c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.634+0000 7fa14d400640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa13c0a3d20 0x7fa13c0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa140006010 tx=0x7fa140005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.638+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa14809cbd0 con 0x7fa158108400 2024-04-20T11:32:53.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.886+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa15802d5d0 con 0x7fa158108400 2024-04-20T11:32:53.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.888+0000 7fa14f400640 1 -- 172.21.15.44:0/4204259757 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fa1480a0ed0 con 0x7fa158108400 2024-04-20T11:32:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:32:53.889 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:32:53.889 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:32:53.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa13c0a3d20 msgr2=0x7fa13c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa13c0a3d20 0x7fa13c0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa140006010 tx=0x7fa140005e40 comp rx=0 tx=0).stop 2024-04-20T11:32:53.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 msgr2=0x7fa1581a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:53.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 0x7fa1581a4900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e9f0 tx=0x7fa14800eeb0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 shutdown_connections 2024-04-20T11:32:53.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15810e470 0x7fa15819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa158108da0 0x7fa1581a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fa13c0a3d20 0x7fa13c0a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 --2- 172.21.15.44:0/4204259757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa158108400 0x7fa1581a4900 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:53.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.891+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 >> 172.21.15.44:0/4204259757 conn(0x7fa158077ae0 msgr2=0x7fa1580791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:53.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.892+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 shutdown_connections 2024-04-20T11:32:53.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:53.892+0000 7fa157200640 1 -- 172.21.15.44:0/4204259757 wait complete. 2024-04-20T11:32:53.907 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:53 smithi044 ceph-mon[16855]: pgmap v319: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T11:32:53.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-20T11:32:53.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:53 smithi078 ceph-mon[21464]: pgmap v319: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T11:32:53.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:53 smithi165 ceph-mon[21975]: pgmap v319: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T11:32:54.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.130+0000 7f18d5600640 1 -- 172.21.15.44:0/2264043348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 msgr2=0x7f18d00746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.130+0000 7f18d5600640 1 --2- 172.21.15.44:0/2264043348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d00746e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009ca0 tx=0x7f18c002c190 comp rx=0 tx=0).stop 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 -- 172.21.15.44:0/2264043348 shutdown_connections 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 --2- 172.21.15.44:0/2264043348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 0x7f18d0115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 --2- 172.21.15.44:0/2264043348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0074c20 0x7f18d01128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 --2- 172.21.15.44:0/2264043348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d00746e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 -- 172.21.15.44:0/2264043348 >> 172.21.15.44:0/2264043348 conn(0x7f18d00fd930 msgr2=0x7f18d00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 -- 172.21.15.44:0/2264043348 shutdown_connections 2024-04-20T11:32:54.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.131+0000 7f18d5600640 1 -- 172.21.15.44:0/2264043348 wait complete. 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.132+0000 7f18d5600640 1 Processor -- start 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.132+0000 7f18d5600640 1 -- start start 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.132+0000 7f18d5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0074c20 0x7f18d019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 0x7f18d01a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d5600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0118930 con 0x7f18d0112e10 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d5600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01187b0 con 0x7f18d0074c20 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0118ab0 con 0x7f18d0074300 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 0x7f18d01a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44306/0 (socket says 172.21.15.44:44306) 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 -- 172.21.15.44:0/1995386183 learned_addr learned my addr 172.21.15.44:0/1995386183 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 0x7f18d01a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54390/0 (socket says 172.21.15.44:54390) 2024-04-20T11:32:54.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cf400640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0074c20 0x7f18d019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 -- 172.21.15.44:0/1995386183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0074c20 msgr2=0x7f18d019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0074c20 0x7f18d019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 -- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 msgr2=0x7f18d01a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 0x7f18d01a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.133+0000 7f18cfe00640 1 -- 172.21.15.44:0/1995386183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d01a3e00 con 0x7f18d0074300 2024-04-20T11:32:54.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.134+0000 7f18cfe00640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d019ee10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005e30 tx=0x7f18c00346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.134+0000 7f18ccc00640 1 -- 172.21.15.44:0/1995386183 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0039040 con 0x7f18d0074300 2024-04-20T11:32:54.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.134+0000 7f18ccc00640 1 -- 172.21.15.44:0/1995386183 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18c0044070 con 0x7f18d0074300 2024-04-20T11:32:54.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.134+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a4090 con 0x7f18d0074300 2024-04-20T11:32:54.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.134+0000 7f18ccc00640 1 -- 172.21.15.44:0/1995386183 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0004330 con 0x7f18d0074300 2024-04-20T11:32:54.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.134+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01ab980 con 0x7f18d0074300 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.137+0000 7f18d9a2c740 1 Processor -- start 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 -- start start 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b5c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18ccc00640 1 -- 172.21.15.44:0/1995386183 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f18c00044d0 con 0x7f18d0074300 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b67a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565307ac1050 con 0x565307b5c170 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565307a67490 con 0x565307b65670 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565307abd010 con 0x565307b5ca90 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b5c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b5c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54394/0 (socket says 172.21.15.44:54394) 2024-04-20T11:32:54.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18d4c00640 1 -- 172.21.15.44:0/516002562 learned_addr learned my addr 172.21.15.44:0/516002562 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18cf400640 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b67a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.138+0000 7f18cfe00640 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18cf400640 1 -- 172.21.15.44:0/516002562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 msgr2=0x565307b65130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18cf400640 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b65130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18cf400640 1 -- 172.21.15.44:0/516002562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 msgr2=0x565307b5c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18cf400640 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b5c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18cf400640 1 -- 172.21.15.44:0/516002562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565307a6f070 con 0x565307b65670 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18d4c00640 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18cf400640 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b67a60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009bd0 tx=0x7f18b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0afbebd016a9c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18c5200640 1 -- 172.21.15.44:0/516002562 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8004230 con 0x565307b65670 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.139+0000 7f18c5200640 1 -- 172.21.15.44:0/516002562 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18b8009e40 con 0x565307b65670 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18c5200640 1 -- 172.21.15.44:0/516002562 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8004dd0 con 0x565307b65670 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18ccc00640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f18b40a3d20 0x7f18b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18ccc00640 1 -- 172.21.15.44:0/1995386183 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f18c00f2a60 con 0x7f18d0074300 2024-04-20T11:32:54.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 -- 172.21.15.44:0/516002562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 msgr2=0x565307b67a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d4c00640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f18b40a3d20 0x7f18b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b67a60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009bd0 tx=0x7f18b8011810 comp rx=0 tx=0).stop 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 -- 172.21.15.44:0/516002562 shutdown_connections 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b67a60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b65130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/516002562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 -- 172.21.15.44:0/516002562 >> 172.21.15.44:0/516002562 conn(0x565307b568a0 msgr2=0x565307b56c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.140+0000 7f18d9a2c740 1 -- 172.21.15.44:0/516002562 shutdown_connections 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.141+0000 7f18d9a2c740 1 -- 172.21.15.44:0/516002562 wait complete. 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.141+0000 7f18d9a2c740 1 -- start start 2024-04-20T11:32:54.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.141+0000 7f18d4c00640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f18b40a3d20 0x7f18b40a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f18c80099b0 tx=0x7f18c8005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.141+0000 7f18d9a2c740 1 Processor -- start 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b63110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x565307b68460 con 0x565307b5c170 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565307a7a860 con 0x565307b65670 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565307b685e0 con 0x565307b5ca90 2024-04-20T11:32:54.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d9a2c740 10 client.0 _ll_register_callbacks cb 0x565307a48fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b63110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b62690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54408/0 (socket says 172.21.15.44:54408) 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 -- 172.21.15.44:0/624066094 learned_addr learned my addr 172.21.15.44:0/624066094 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b63110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54426/0 (socket says 172.21.15.44:54426) 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 -- 172.21.15.44:0/624066094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 msgr2=0x565307b62bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b62bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 -- 172.21.15.44:0/624066094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 msgr2=0x565307b63110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b63110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.142+0000 7f18cfe00640 1 -- 172.21.15.44:0/624066094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565307a7e950 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18cfe00640 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b62690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004af0 tx=0x7f18c00099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1629347afdc474f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18bd600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18bd600640 1 -- 172.21.15.44:0/624066094 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00be630 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18bd600640 1 -- 172.21.15.44:0/624066094 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f18c00df070 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565307acea70 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18bd600640 1 -- 172.21.15.44:0/624066094 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f18c00be450 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18d9a2c740 10 client.18084 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x565307a4c820 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.143+0000 7f18bd600640 1 -- 172.21.15.44:0/624066094 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x565307acea70 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18bd600640 1 -- 172.21.15.44:0/624066094 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f18c003c030 con 0x565307b5c170 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18d9a2c740 10 client.18084 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18d9a2c740 10 client.18084 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18d9a2c740 20 client.18084 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307b75630 0x565307b77a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18d9a2c740 4 client.18084 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565307ac88e0 con 0x565307b75630 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.144+0000 7f18cf400640 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307b75630 0x565307b77a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.145+0000 7f18cf400640 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307b75630 0x565307b77a60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6cab48db8fcd19a9 server_cookie=eb07f5d8031e5ba8 in_seq=0 out_seq=0 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.145+0000 7f18bd600640 10 client.18084 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18bd600640 1 -- 172.21.15.44:0/624066094 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565307ac88e0 con 0x565307b75630 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18bd600640 10 client.18084 handle_command_reply: tid=0 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 client.18084 shutdown 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 msgr2=0x565307b62690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b62690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004af0 tx=0x7f18c00099c0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 shutdown_connections 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307b75630 0x565307b77a60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b63110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5ca90 0x565307b62bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18bd600640 0 client.18084 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:54.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/624066094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5c170 0x565307b62690 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 >> 172.21.15.44:0/624066094 conn(0x565307b568a0 msgr2=0x565307b585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 shutdown_connections 2024-04-20T11:32:54.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 1 -- 172.21.15.44:0/624066094 wait complete. 2024-04-20T11:32:54.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.146+0000 7f18d9a2c740 20 client.18084 trim_cache size 0 max 16384 2024-04-20T11:32:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.161+0000 7f18d9a2c740 1 Processor -- start 2024-04-20T11:32:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.161+0000 7f18d9a2c740 1 -- start start 2024-04-20T11:32:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b64d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b71320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b4040 con 0x565307b5ca90 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b5b80 con 0x565307b65670 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b4710 con 0x565307b5c170 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b71320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54428/0 (socket says 172.21.15.44:54428) 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b71320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54422/0 (socket says 172.21.15.44:54422) 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18d4c00640 1 -- 172.21.15.44:0/2856616843 learned_addr learned my addr 172.21.15.44:0/2856616843 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.162+0000 7f18cfe00640 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b64d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18cfe00640 1 -- 172.21.15.44:0/2856616843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 msgr2=0x565307b557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18cfe00640 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18cfe00640 1 -- 172.21.15.44:0/2856616843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 msgr2=0x565307b71320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18cfe00640 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b71320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18cfe00640 1 -- 172.21.15.44:0/2856616843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653077d33f0 con 0x565307b5c170 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18cfe00640 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b64d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f18c00baf50 tx=0x7f18c0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a15ec15224e0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18bfe00640 1 -- 172.21.15.44:0/2856616843 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00be630 con 0x565307b5c170 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18bfe00640 1 -- 172.21.15.44:0/2856616843 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18c00f2e50 con 0x565307b5c170 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.163+0000 7f18bfe00640 1 -- 172.21.15.44:0/2856616843 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00c72d0 con 0x565307b5c170 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 -- 172.21.15.44:0/2856616843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 msgr2=0x565307b64d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b64d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f18c00baf50 tx=0x7f18c0000f40 comp rx=0 tx=0).stop 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 -- 172.21.15.44:0/2856616843 shutdown_connections 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b71320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/2856616843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b64d50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 -- 172.21.15.44:0/2856616843 >> 172.21.15.44:0/2856616843 conn(0x565307b568a0 msgr2=0x565307b550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 -- 172.21.15.44:0/2856616843 shutdown_connections 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 -- 172.21.15.44:0/2856616843 wait complete. 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 -- start start 2024-04-20T11:32:54.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.164+0000 7f18d9a2c740 1 Processor -- start 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b5f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b5f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b5fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b54b0 con 0x565307b5ca90 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b4de0 con 0x565307b65670 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b2030 con 0x565307b5c170 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d9a2c740 10 client.0 _ll_register_callbacks cb 0x7f18c00f6200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b5f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b5f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b5fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b5f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54426/0 (socket says 172.21.15.44:54426) 2024-04-20T11:32:54.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.165+0000 7f18cf400640 1 -- 172.21.15.44:0/1141522863 learned_addr learned my addr 172.21.15.44:0/1141522863 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cf400640 1 -- 172.21.15.44:0/1141522863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 msgr2=0x565307b5f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cf400640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b5f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cf400640 1 -- 172.21.15.44:0/1141522863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 msgr2=0x565307b5fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cf400640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b5fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cf400640 1 -- 172.21.15.44:0/1141522863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56530789d7d0 con 0x565307b5ca90 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cfe00640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b5f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18d4c00640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b5fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18cf400640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b5f8a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009710 tx=0x7f18b8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b7d4ad64fab621 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18bd600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18bd600640 1 -- 172.21.15.44:0/1141522863 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b801e040 con 0x565307b5ca90 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18bd600640 1 -- 172.21.15.44:0/1141522863 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f18b8004800 con 0x565307b5ca90 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.166+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c00f2e50 con 0x565307b5ca90 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18bd600640 1 -- 172.21.15.44:0/1141522863 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f18b8004250 con 0x565307b5ca90 2024-04-20T11:32:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 10 client.18090 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x565307bdbd50 con 0x565307b5ca90 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18bd600640 1 -- 172.21.15.44:0/1141522863 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18c00f2e50 con 0x565307b5ca90 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18bd600640 1 -- 172.21.15.44:0/1141522863 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f18b8018070 con 0x565307b5ca90 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 10 client.18090 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 10 client.18090 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 20 client.18090 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307bdc370 0x565307bde790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 4 client.18090 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.167+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 --> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565307a445a0 con 0x565307bdc370 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.168+0000 7f18cfe00640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307bdc370 0x565307bde790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.168+0000 7f18cfe00640 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307bdc370 0x565307bde790 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5250206b4bc1bc9b server_cookie=55ddf5e8e700af07 in_seq=0 out_seq=0 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.168+0000 7f18bd600640 10 client.18090 ms_handle_connect on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.168+0000 7f18bd600640 1 -- 172.21.15.44:0/1141522863 <== mds.4 v2:172.21.15.44:6836/2623237731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565307a445a0 con 0x565307bdc370 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.168+0000 7f18bd600640 10 client.18090 handle_command_reply: tid=0 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.168+0000 7f18d9a2c740 1 client.18090 shutdown 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 msgr2=0x565307b5f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b5f8a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009710 tx=0x7f18b8009810 comp rx=0 tx=0).stop 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 shutdown_connections 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:6836/2623237731,v1:172.21.15.44:6837/2623237731] conn(0x565307bdc370 0x565307bde790 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565307b65670 0x565307b5fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x565307b5ca90 0x565307b5f8a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18bd600640 0 client.18090 ms_handle_reset on v2:172.21.15.44:6836/2623237731 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 --2- 172.21.15.44:0/1141522863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565307b5c170 0x565307b5f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 >> 172.21.15.44:0/1141522863 conn(0x565307b568a0 msgr2=0x565307b55960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 shutdown_connections 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 1 -- 172.21.15.44:0/1141522863 wait complete. 2024-04-20T11:32:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.169+0000 7f18d9a2c740 20 client.18090 trim_cache size 0 max 16384 2024-04-20T11:32:54.173 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f18b40a3d20 msgr2=0x7f18b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f18b40a3d20 0x7f18b40a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f18c80099b0 tx=0x7f18c8005f90 comp rx=0 tx=0).stop 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 msgr2=0x7f18d019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d019ee10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005e30 tx=0x7f18c00346f0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 shutdown_connections 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f18b40a3d20 0x7f18b40a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18d0112e10 0x7f18d01a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18d0074c20 0x7f18d019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 --2- 172.21.15.44:0/1995386183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18d0074300 0x7f18d019ee10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 >> 172.21.15.44:0/1995386183 conn(0x7f18d00fd930 msgr2=0x7f18d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 shutdown_connections 2024-04-20T11:32:54.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.170+0000 7f18d5600640 1 -- 172.21.15.44:0/1995386183 wait complete. 2024-04-20T11:32:54.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 -- 172.21.15.44:0/2479782965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3068104d60 msgr2=0x7f3068108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 --2- 172.21.15.44:0/2479782965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3068104d60 0x7f3068108da0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3058009900 tx=0x7f305802bc40 comp rx=0 tx=0).stop 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 -- 172.21.15.44:0/2479782965 shutdown_connections 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 --2- 172.21.15.44:0/2479782965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3068109480 0x7f3068111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 --2- 172.21.15.44:0/2479782965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3068104d60 0x7f3068108da0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 --2- 172.21.15.44:0/2479782965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f3068104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 -- 172.21.15.44:0/2479782965 >> 172.21.15.44:0/2479782965 conn(0x7f30680ffbd0 msgr2=0x7f3068101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 -- 172.21.15.44:0/2479782965 shutdown_connections 2024-04-20T11:32:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.396+0000 7f306f400640 1 -- 172.21.15.44:0/2479782965 wait complete. 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.397+0000 7f306f400640 1 Processor -- start 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.397+0000 7f306f400640 1 -- start start 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f30681a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3068109480 0x7f30681a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f306819a990 0x7f306819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3068114650 con 0x7f30681043b0 2024-04-20T11:32:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306f400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30681144d0 con 0x7f306819a990 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306f400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30681147d0 con 0x7f3068109480 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f30681a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f30681a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54440/0 (socket says 172.21.15.44:54440) 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306ca00640 1 -- 172.21.15.44:0/4098472487 learned_addr learned my addr 172.21.15.44:0/4098472487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f3067e00640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3068109480 0x7f30681a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.398+0000 7f306d400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f306819a990 0x7f306819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306ca00640 1 -- 172.21.15.44:0/4098472487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3068109480 msgr2=0x7f30681a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306ca00640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3068109480 0x7f30681a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306ca00640 1 -- 172.21.15.44:0/4098472487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f306819a990 msgr2=0x7f306819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306ca00640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f306819a990 0x7f306819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306ca00640 1 -- 172.21.15.44:0/4098472487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306819b4b0 con 0x7f30681043b0 2024-04-20T11:32:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306ca00640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f30681a07e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f305c00e9f0 tx=0x7f305c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f3065600640 1 -- 172.21.15.44:0/4098472487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c00cd80 con 0x7f30681043b0 2024-04-20T11:32:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f3065600640 1 -- 172.21.15.44:0/4098472487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f305c010930 con 0x7f30681043b0 2024-04-20T11:32:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f3065600640 1 -- 172.21.15.44:0/4098472487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c018b00 con 0x7f30681043b0 2024-04-20T11:32:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306819b7a0 con 0x7f30681043b0 2024-04-20T11:32:54.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.399+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30681a79f0 con 0x7f30681043b0 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 Processor -- start 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3065600640 1 -- 172.21.15.44:0/4098472487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f305c018ca0 con 0x7f30681043b0 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 -- start start 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf5b83a050 con 0x55bf5b8de4b0 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf5b7e0490 con 0x55bf5b8d58d0 2024-04-20T11:32:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.403+0000 7f3072197740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf5b836010 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f3067e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8d5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44356/0 (socket says 172.21.15.44:44356) 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 -- 172.21.15.44:0/1709475948 learned_addr learned my addr 172.21.15.44:0/1709475948 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306ca00640 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 -- 172.21.15.44:0/1709475948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 msgr2=0x55bf5b8ddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ddf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 -- 172.21.15.44:0/1709475948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 msgr2=0x55bf5b8e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 -- 172.21.15.44:0/1709475948 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5b7e8070 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.404+0000 7f306d400640 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8d5390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3060009f50 tx=0x7f30600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28264c20b4074f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.405+0000 7f3065600640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f30500a3d20 0x7f30500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.405+0000 7f3055c00640 1 -- 172.21.15.44:0/1709475948 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060004d20 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.405+0000 7f3055c00640 1 -- 172.21.15.44:0/1709475948 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3060004050 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.405+0000 7f3055c00640 1 -- 172.21.15.44:0/1709475948 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060005260 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.405+0000 7f3065600640 1 -- 172.21.15.44:0/4098472487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f305c0d0ba0 con 0x7f30681043b0 2024-04-20T11:32:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.405+0000 7f306ca00640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f30500a3d20 0x7f30500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 -- 172.21.15.44:0/1709475948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 msgr2=0x55bf5b8d5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f306ca00640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f30500a3d20 0x7f30500a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55bf5b8d4a80 tx=0x7f305c0d4650 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8d5390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3060009f50 tx=0x7f30600118d0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 -- 172.21.15.44:0/1709475948 shutdown_connections 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 --2- 172.21.15.44:0/1709475948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8d5390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 -- 172.21.15.44:0/1709475948 >> 172.21.15.44:0/1709475948 conn(0x55bf5b8cf6e0 msgr2=0x55bf5b8cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 -- 172.21.15.44:0/1709475948 shutdown_connections 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 -- 172.21.15.44:0/1709475948 wait complete. 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.406+0000 7f3072197740 1 -- start start 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 Processor -- start 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8db520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8dba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf5b8e12a0 con 0x55bf5b8de4b0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf5b7f3860 con 0x55bf5b8d58d0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf5b8e1420 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3072197740 10 client.0 _ll_register_callbacks cb 0x55bf5b7c1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f306d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f306d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8dbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54448/0 (socket says 172.21.15.44:54448) 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f3067e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8dba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.407+0000 7f306d400640 1 -- 172.21.15.44:0/2017706788 learned_addr learned my addr 172.21.15.44:0/2017706788 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f306d400640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f3067e00640 1 -- 172.21.15.44:0/2017706788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 msgr2=0x55bf5b8db520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f3067e00640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8db520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f3067e00640 1 -- 172.21.15.44:0/2017706788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 msgr2=0x55bf5b8dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f3067e00640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f3067e00640 1 -- 172.21.15.44:0/2017706788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5b7f7950 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f306d400640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f3067e00640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8dba60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3058009230 tx=0x7f305802dd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63bc2d801fe46ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f304a000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.408+0000 7f304a000640 1 -- 172.21.15.44:0/2017706788 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058044070 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3060004050 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f304a000640 1 -- 172.21.15.44:0/2017706788 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f3058004310 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f304a000640 1 -- 172.21.15.44:0/2017706788 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f305804f050 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f3072197740 10 client.28211 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55bf5b7c5820 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f304a000640 1 -- 172.21.15.44:0/2017706788 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3060004050 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.409+0000 7f304a000640 1 -- 172.21.15.44:0/2017706788 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3058004c10 con 0x55bf5b8d58d0 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f3072197740 10 client.28211 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f3072197740 10 client.28211 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:32:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f3072197740 20 client.28211 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f3072197740 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b8ee490 0x55bf5b8f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f3072197740 4 client.28211 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf5b847a70 con 0x55bf5b8ee490 2024-04-20T11:32:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.410+0000 7f306d400640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b8ee490 0x55bf5b8f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.411+0000 7f306d400640 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b8ee490 0x55bf5b8f08d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7b7d7746e6852f0 server_cookie=11b350b660230c10 in_seq=0 out_seq=0 2024-04-20T11:32:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.411+0000 7f304a000640 10 client.28211 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.411+0000 7f304a000640 1 -- 172.21.15.44:0/2017706788 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf5b847a70 con 0x55bf5b8ee490 2024-04-20T11:32:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.411+0000 7f304a000640 10 client.28211 handle_command_reply: tid=0 2024-04-20T11:32:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.411+0000 7f3072197740 1 client.28211 shutdown 2024-04-20T11:32:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 msgr2=0x55bf5b8dba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8dba60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3058009230 tx=0x7f305802dd60 comp rx=0 tx=0).stop 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 shutdown_connections 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b8ee490 0x55bf5b8f08d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8dba60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 --2- 172.21.15.44:0/2017706788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8db520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f304a000640 0 client.28211 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 >> 172.21.15.44:0/2017706788 conn(0x55bf5b8cf6e0 msgr2=0x55bf5b8d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 shutdown_connections 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 1 -- 172.21.15.44:0/2017706788 wait complete. 2024-04-20T11:32:54.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.412+0000 7f3072197740 20 client.28211 trim_cache size 0 max 16384 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 Processor -- start 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 -- start start 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30681aff60 con 0x55bf5b8de4b0 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30681b1aa0 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.427+0000 7f3072197740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30681b0630 con 0x55bf5b8d58d0 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f306d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f306d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ea160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44370/0 (socket says 172.21.15.44:44370) 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 -- 172.21.15.44:0/660921968 learned_addr learned my addr 172.21.15.44:0/660921968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 -- 172.21.15.44:0/660921968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d4fb0 msgr2=0x55bf5b8ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 -- 172.21.15.44:0/660921968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 msgr2=0x55bf5b8d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 -- 172.21.15.44:0/660921968 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5b54c3f0 con 0x55bf5b8d58d0 2024-04-20T11:32:54.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3067e00640 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ea160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f305802c250 tx=0x7f3058005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8459ad1025b1155 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3055200640 1 -- 172.21.15.44:0/660921968 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058000e60 con 0x55bf5b8d58d0 2024-04-20T11:32:54.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3055200640 1 -- 172.21.15.44:0/660921968 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f30580054e0 con 0x55bf5b8d58d0 2024-04-20T11:32:54.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.428+0000 7f3055200640 1 -- 172.21.15.44:0/660921968 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058044040 con 0x55bf5b8d58d0 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 -- 172.21.15.44:0/660921968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 msgr2=0x55bf5b8ea160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ea160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f305802c250 tx=0x7f3058005fd0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 -- 172.21.15.44:0/660921968 shutdown_connections 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8ea160 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 --2- 172.21.15.44:0/660921968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 -- 172.21.15.44:0/660921968 >> 172.21.15.44:0/660921968 conn(0x55bf5b8cf6e0 msgr2=0x55bf5b8d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 -- 172.21.15.44:0/660921968 shutdown_connections 2024-04-20T11:32:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.429+0000 7f3072197740 1 -- 172.21.15.44:0/660921968 wait complete. 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 -- start start 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 Processor -- start 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8e2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30681b13d0 con 0x55bf5b8d58d0 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30681b0d00 con 0x55bf5b8de4b0 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30681adf50 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.430+0000 7f3072197740 10 client.0 _ll_register_callbacks cb 0x7f305802d9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8e20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44376/0 (socket says 172.21.15.44:44376) 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 -- 172.21.15.44:0/2525300109 learned_addr learned my addr 172.21.15.44:0/2525300109 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f3067e00640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 -- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8de4b0 msgr2=0x55bf5b8e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 -- 172.21.15.44:0/2525300109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8d58d0 msgr2=0x55bf5b8e2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8e2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 -- 172.21.15.44:0/2525300109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5b6167d0 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f306d400640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8e20e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3060011dd0 tx=0x7f30600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8f234efe7255eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f304a000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.431+0000 7f304a000640 1 -- 172.21.15.44:0/2525300109 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600043b0 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30580054e0 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f304a000640 1 -- 172.21.15.44:0/2525300109 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f3060004550 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f304a000640 1 -- 172.21.15.44:0/2525300109 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3060027440 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f3072197740 10 client.28225 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55bf5b954be0 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f304a000640 1 -- 172.21.15.44:0/2525300109 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f30580054e0 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f304a000640 1 -- 172.21.15.44:0/2525300109 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f3060039070 con 0x55bf5b8d4fb0 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f3072197740 10 client.28225 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f3072197740 10 client.28225 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.432+0000 7f3072197740 20 client.28225 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.433+0000 7f3072197740 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b9551c0 0x55bf5b9575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.433+0000 7f3072197740 4 client.28225 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.433+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 --> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf5b8418e0 con 0x55bf5b9551c0 2024-04-20T11:32:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.433+0000 7f3067e00640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b9551c0 0x55bf5b9575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3067e00640 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b9551c0 0x55bf5b9575b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4e4467f5a130a38 server_cookie=8436f0d0551e5f49 in_seq=0 out_seq=0 2024-04-20T11:32:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f304a000640 10 client.28225 ms_handle_connect on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f304a000640 1 -- 172.21.15.44:0/2525300109 <== mds.3 v2:172.21.15.78:6834/3770087097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf5b8418e0 con 0x55bf5b9551c0 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f304a000640 10 client.28225 handle_command_reply: tid=0 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 client.28225 shutdown 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 msgr2=0x55bf5b8e20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8e20e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3060011dd0 tx=0x7f30600118d0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 shutdown_connections 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf5b8de4b0 0x55bf5b8e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.78:6834/3770087097,v1:172.21.15.78:6835/3770087097] conn(0x55bf5b9551c0 0x55bf5b9575b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf5b8d58d0 0x55bf5b8e2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 --2- 172.21.15.44:0/2525300109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf5b8d4fb0 0x55bf5b8e20e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f304a000640 0 client.28225 ms_handle_reset on v2:172.21.15.78:6834/3770087097 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 >> 172.21.15.44:0/2525300109 conn(0x55bf5b8cf6e0 msgr2=0x55bf5b8dbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 shutdown_connections 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 1 -- 172.21.15.44:0/2525300109 wait complete. 2024-04-20T11:32:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.434+0000 7f3072197740 20 client.28225 trim_cache size 0 max 16384 2024-04-20T11:32:54.438 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f30500a3d20 msgr2=0x7f30500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f30500a3d20 0x7f30500a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55bf5b8d4a80 tx=0x7f305c0d4650 comp rx=0 tx=0).stop 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 msgr2=0x7f30681a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f30681a07e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f305c00e9f0 tx=0x7f305c00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 shutdown_connections 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f306819a990 0x7f306819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3068109480 0x7f30681a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f30500a3d20 0x7f30500a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 --2- 172.21.15.44:0/4098472487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30681043b0 0x7f30681a07e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 >> 172.21.15.44:0/4098472487 conn(0x7f30680ffbd0 msgr2=0x7f3068101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 shutdown_connections 2024-04-20T11:32:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.435+0000 7f306f400640 1 -- 172.21.15.44:0/4098472487 wait complete. 2024-04-20T11:32:54.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 -- 172.21.15.44:0/3402496000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 msgr2=0x7f5ac8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 --2- 172.21.15.44:0/3402496000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 0x7f5ac8105db0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009900 tx=0x7f5ab402bd60 comp rx=0 tx=0).stop 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 -- 172.21.15.44:0/3402496000 shutdown_connections 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 --2- 172.21.15.44:0/3402496000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 0x7f5ac8106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 --2- 172.21.15.44:0/3402496000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 0x7f5ac8105db0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 --2- 172.21.15.44:0/3402496000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 -- 172.21.15.44:0/3402496000 >> 172.21.15.44:0/3402496000 conn(0x7f5ac8077f60 msgr2=0x7f5ac8078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 -- 172.21.15.44:0/3402496000 shutdown_connections 2024-04-20T11:32:54.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.669+0000 7f5ad0200640 1 -- 172.21.15.44:0/3402496000 wait complete. 2024-04-20T11:32:54.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.670+0000 7f5ad0200640 1 Processor -- start 2024-04-20T11:32:54.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 -- start start 2024-04-20T11:32:54.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 0x7f5ac819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 0x7f5ac81a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8118a70 con 0x7f5ac8100fd0 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81188f0 con 0x7f5ac81062f0 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ad0200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8118bf0 con 0x7f5ac81018f0 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5acd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5acd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54470/0 (socket says 172.21.15.44:54470) 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5acd800640 1 -- 172.21.15.44:0/28415381 learned_addr learned my addr 172.21.15.44:0/28415381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.671+0000 7f5ace200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 0x7f5ac81a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5acce00640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 0x7f5ac819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5ace200640 1 -- 172.21.15.44:0/28415381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 msgr2=0x7f5ac819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5ace200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 0x7f5ac819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5ace200640 1 -- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 msgr2=0x7f5ac819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5ace200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5ace200640 1 -- 172.21.15.44:0/28415381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81a3f80 con 0x7f5ac81062f0 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5acd800640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.672+0000 7f5ace200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 0x7f5ac81a3850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000d6b0 tx=0x7f5ac000db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.673+0000 7f5ac6000640 1 -- 172.21.15.44:0/28415381 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0017070 con 0x7f5ac81062f0 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.673+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81a4270 con 0x7f5ac81062f0 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.673+0000 7f5ac6000640 1 -- 172.21.15.44:0/28415381 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ac0009d20 con 0x7f5ac81062f0 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.673+0000 7f5ac6000640 1 -- 172.21.15.44:0/28415381 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0003df0 con 0x7f5ac81062f0 2024-04-20T11:32:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.673+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81abb00 con 0x7f5ac81062f0 2024-04-20T11:32:54.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.676+0000 7f5ad2fec740 1 Processor -- start 2024-04-20T11:32:54.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.676+0000 7f5ac6000640 1 -- 172.21.15.44:0/28415381 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5ac0004220 con 0x7f5ac81062f0 2024-04-20T11:32:54.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.676+0000 7f5ad2fec740 1 -- start start 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c83390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c8bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c8e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5e9be8050 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5e9b8e490 con 0x55c5e9c82fb0 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5e9be4010 con 0x55c5e9c838d0 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c8e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c8e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54478/0 (socket says 172.21.15.44:54478) 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 -- 172.21.15.44:0/1486774338 learned_addr learned my addr 172.21.15.44:0/1486774338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c83390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acd800640 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c8bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 -- 172.21.15.44:0/1486774338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 msgr2=0x55c5e9c8bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c8bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 -- 172.21.15.44:0/1486774338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 msgr2=0x55c5e9c83390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c83390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.677+0000 7f5acce00640 1 -- 172.21.15.44:0/1486774338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e9b96070 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5acce00640 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c8e8a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009230 tx=0x7f5ab4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8d2abb29ae12a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5ac6000640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5ab00a3da0 0x7f5ab00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5abaa00640 1 -- 172.21.15.44:0/1486774338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004040 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5ac6000640 1 -- 172.21.15.44:0/28415381 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f5ac00cfef0 con 0x7f5ac81062f0 2024-04-20T11:32:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5abaa00640 1 -- 172.21.15.44:0/1486774338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ab4005550 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5abaa00640 1 -- 172.21.15.44:0/1486774338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004bc0 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.678+0000 7f5acd800640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5ab00a3da0 0x7f5ab00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 -- 172.21.15.44:0/1486774338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 msgr2=0x55c5e9c8e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c8e8a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009230 tx=0x7f5ab4005e20 comp rx=0 tx=0).stop 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 -- 172.21.15.44:0/1486774338 shutdown_connections 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c8e8a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c8bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/1486774338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c83390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 -- 172.21.15.44:0/1486774338 >> 172.21.15.44:0/1486774338 conn(0x55c5e9c7d6e0 msgr2=0x55c5e9c7dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5acd800640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5ab00a3da0 0x7f5ab00a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55c5e9c82a80 tx=0x7f5abc009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 -- 172.21.15.44:0/1486774338 shutdown_connections 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.679+0000 7f5ad2fec740 1 -- 172.21.15.44:0/1486774338 wait complete. 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 -- start start 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 Processor -- start 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c838d0 0x55c5e9c89a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c89f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5e9c8f230 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5e9ba1860 con 0x55c5e9c838d0 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.680+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5e9c8f3b0 con 0x55c5e9c82fb0 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5ad2fec740 10 client.0 _ll_register_callbacks cb 0x55c5e9b6ffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c838d0 0x55c5e9c89a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c894c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44408/0 (socket says 172.21.15.44:44408) 2024-04-20T11:32:54.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 -- 172.21.15.44:0/824931567 learned_addr learned my addr 172.21.15.44:0/824931567 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5ace200640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c89f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 -- 172.21.15.44:0/824931567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c838d0 msgr2=0x55c5e9c89a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c838d0 0x55c5e9c89a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 -- 172.21.15.44:0/824931567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 msgr2=0x55c5e9c89f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c89f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acd800640 1 -- 172.21.15.44:0/824931567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e9ba5950 con 0x55c5e9c82fb0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.681+0000 7f5acce00640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c838d0 0x55c5e9c89a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5acd800640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c894c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ffd0 tx=0x7f5abc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a33321e4589cae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5aaf400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5aaf400640 1 -- 172.21.15.44:0/824931567 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0123b0 con 0x55c5e9c82fb0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5aaf400640 1 -- 172.21.15.44:0/824931567 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f5abc029440 con 0x55c5e9c82fb0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab4005550 con 0x55c5e9c82fb0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5aaf400640 1 -- 172.21.15.44:0/824931567 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5abc029640 con 0x55c5e9c82fb0 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5ad2fec740 10 client.28237 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.682+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c5e9b73820 con 0x55c5e9c82fb0 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5aaf400640 1 -- 172.21.15.44:0/824931567 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ab4005550 con 0x55c5e9c82fb0 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5aaf400640 1 -- 172.21.15.44:0/824931567 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f5abc027020 con 0x55c5e9c82fb0 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5ad2fec740 10 client.28237 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5ad2fec740 10 client.28237 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5ad2fec740 20 client.28237 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9c9c4a0 0x55c5e9c9e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5ad2fec740 4 client.28237 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.683+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5e9bf5a70 con 0x55c5e9c9c4a0 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.684+0000 7f5acce00640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9c9c4a0 0x55c5e9c9e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.684+0000 7f5acce00640 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9c9c4a0 0x55c5e9c9e910 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57f8a1dca8b441b8 server_cookie=c700117e8efc26ae in_seq=0 out_seq=0 2024-04-20T11:32:54.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.684+0000 7f5aaf400640 10 client.28237 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5aaf400640 1 -- 172.21.15.44:0/824931567 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5e9bf5a70 con 0x55c5e9c9c4a0 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5aaf400640 10 client.28237 handle_command_reply: tid=0 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 client.28237 shutdown 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 msgr2=0x55c5e9c894c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c894c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00ffd0 tx=0x7f5abc004ab0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 shutdown_connections 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9c9c4a0 0x55c5e9c9e910 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c89f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c838d0 0x55c5e9c89a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/824931567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c894c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 >> 172.21.15.44:0/824931567 conn(0x55c5e9c7d6e0 msgr2=0x55c5e9c7f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5aaf400640 0 client.28237 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:54.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 shutdown_connections 2024-04-20T11:32:54.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.685+0000 7f5ad2fec740 1 -- 172.21.15.44:0/824931567 wait complete. 2024-04-20T11:32:54.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.686+0000 7f5ad2fec740 20 client.28237 trim_cache size 0 max 16384 2024-04-20T11:32:54.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.700+0000 7f5ad2fec740 1 Processor -- start 2024-04-20T11:32:54.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.700+0000 7f5ad2fec740 1 -- start start 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c8bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b4340 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b5e80 con 0x55c5e9c82fb0 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b4a10 con 0x55c5e9c838d0 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c7f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54498/0 (socket says 172.21.15.44:54498) 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 -- 172.21.15.44:0/674535409 learned_addr learned my addr 172.21.15.44:0/674535409 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5acce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5acd800640 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c8bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 -- 172.21.15.44:0/674535409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 msgr2=0x55c5e9c98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 -- 172.21.15.44:0/674535409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 msgr2=0x55c5e9c8bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.701+0000 7f5ace200640 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c8bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5ace200640 1 -- 172.21.15.44:0/674535409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e98fa3f0 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5acce00640 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5acd800640 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c8bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5ace200640 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c7f240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00045f0 tx=0x7f5ac009bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1df48650c9a5605 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5aba000640 1 -- 172.21.15.44:0/674535409 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0020040 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5aba000640 1 -- 172.21.15.44:0/674535409 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ac009a280 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.702+0000 7f5aba000640 1 -- 172.21.15.44:0/674535409 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00b5270 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 -- 172.21.15.44:0/674535409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 msgr2=0x55c5e9c7f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c7f240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00045f0 tx=0x7f5ac009bf80 comp rx=0 tx=0).stop 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 -- 172.21.15.44:0/674535409 shutdown_connections 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c7f240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c838d0 0x55c5e9c98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/674535409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c8bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 -- 172.21.15.44:0/674535409 >> 172.21.15.44:0/674535409 conn(0x55c5e9c7d6e0 msgr2=0x55c5e9c7eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 -- 172.21.15.44:0/674535409 shutdown_connections 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 -- 172.21.15.44:0/674535409 wait complete. 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 -- start start 2024-04-20T11:32:54.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.703+0000 7f5ad2fec740 1 Processor -- start 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 0x55c5e9c94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b57b0 con 0x55c5e9c838d0 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b50e0 con 0x55c5e9c82fb0 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b2330 con 0x55c5e9c8c4b0 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ad2fec740 10 client.0 _ll_register_callbacks cb 0x7f5abc00fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5acce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 0x55c5e9c94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5acce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 0x55c5e9c94b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54508/0 (socket says 172.21.15.44:54508) 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5acce00640 1 -- 172.21.15.44:0/2633962412 learned_addr learned my addr 172.21.15.44:0/2633962412 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.704+0000 7f5ace200640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acd800640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acce00640 1 -- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c8c4b0 msgr2=0x55c5e9c857f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acce00640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c857f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acce00640 1 -- 172.21.15.44:0/2633962412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 msgr2=0x55c5e9c94630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acce00640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c94630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acce00640 1 -- 172.21.15.44:0/2633962412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e99c47d0 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5acce00640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 0x55c5e9c94b70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40047d0 tx=0x7f5ab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a339fa47da49c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5aaf400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5aaf400640 1 -- 172.21.15.44:0/2633962412 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40543c0 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac009a280 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.705+0000 7f5aaf400640 1 -- 172.21.15.44:0/2633962412 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f5ab4054560 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5aaf400640 1 -- 172.21.15.44:0/2633962412 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5ab4054780 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 10 client.18126 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c5e9d02250 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5aaf400640 1 -- 172.21.15.44:0/2633962412 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ac009a280 con 0x55c5e9c838d0 2024-04-20T11:32:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5aaf400640 1 -- 172.21.15.44:0/2633962412 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7f5ab405c630 con 0x55c5e9c838d0 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 10 client.18126 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 10 client.18126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 20 client.18126 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9d02480 0x55c5e9d04870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 4 client.18126 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.706+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 --> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5e9bef8e0 con 0x55c5e9d02480 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.707+0000 7f5acd800640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9d02480 0x55c5e9d04870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.707+0000 7f5acd800640 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9d02480 0x55c5e9d04870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f618d56fd86c1fd server_cookie=66314e6ac562db5a in_seq=0 out_seq=0 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.707+0000 7f5aaf400640 10 client.18126 ms_handle_connect on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5aaf400640 1 -- 172.21.15.44:0/2633962412 <== mds.2 v2:172.21.15.165:6838/4179547084 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5e9bef8e0 con 0x55c5e9d02480 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5aaf400640 10 client.18126 handle_command_reply: tid=0 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 client.18126 shutdown 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 msgr2=0x55c5e9c94b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 0x55c5e9c94b70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40047d0 tx=0x7f5ab40046c0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 shutdown_connections 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:6838/4179547084,v1:172.21.15.165:6839/4179547084] conn(0x55c5e9d02480 0x55c5e9d04870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5e9c8c4b0 0x55c5e9c857f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5aaf400640 0 client.18126 ms_handle_reset on v2:172.21.15.165:6838/4179547084 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5e9c838d0 0x55c5e9c94b70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 --2- 172.21.15.44:0/2633962412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5e9c82fb0 0x55c5e9c94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 >> 172.21.15.44:0/2633962412 conn(0x55c5e9c7d6e0 msgr2=0x55c5e9c7c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 shutdown_connections 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 1 -- 172.21.15.44:0/2633962412 wait complete. 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.708+0000 7f5ad2fec740 20 client.18126 trim_cache size 0 max 16384 2024-04-20T11:32:54.711 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5ab00a3da0 msgr2=0x7f5ab00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5ab00a3da0 0x7f5ab00a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55c5e9c82a80 tx=0x7f5abc009500 comp rx=0 tx=0).stop 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 msgr2=0x7f5ac81a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 0x7f5ac81a3850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000d6b0 tx=0x7f5ac000db70 comp rx=0 tx=0).stop 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 shutdown_connections 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ac81062f0 0x7f5ac81a3850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac81018f0 0x7f5ac819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f5ab00a3da0 0x7f5ab00a6260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 --2- 172.21.15.44:0/28415381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac8100fd0 0x7f5ac819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 >> 172.21.15.44:0/28415381 conn(0x7f5ac8077f60 msgr2=0x7f5ac8103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 shutdown_connections 2024-04-20T11:32:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.709+0000 7f5ad0200640 1 -- 172.21.15.44:0/28415381 wait complete. 2024-04-20T11:32:54.724 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:54 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/4204259757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:32:54.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T11:32:54.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:54 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/4204259757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 -- 172.21.15.44:0/1598669776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc101b50 msgr2=0x7faecc10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 --2- 172.21.15.44:0/1598669776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc101b50 0x7faecc10dc90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faec4009cd0 tx=0x7faec402c0e0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 -- 172.21.15.44:0/1598669776 shutdown_connections 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 --2- 172.21.15.44:0/1598669776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc101b50 0x7faecc10dc90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 --2- 172.21.15.44:0/1598669776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faecc1011b0 0x7faecc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 --2- 172.21.15.44:0/1598669776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc107040 0x7faecc107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 -- 172.21.15.44:0/1598669776 >> 172.21.15.44:0/1598669776 conn(0x7faecc076c60 msgr2=0x7faecc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 -- 172.21.15.44:0/1598669776 shutdown_connections 2024-04-20T11:32:54.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.946+0000 7faed3c00640 1 -- 172.21.15.44:0/1598669776 wait complete. 2024-04-20T11:32:54.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.947+0000 7faed3c00640 1 Processor -- start 2024-04-20T11:32:54.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.947+0000 7faed3c00640 1 -- start start 2024-04-20T11:32:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faed3c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc1011b0 0x7faecc10e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faed3c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 0x7faecc10ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faed3c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faecc107040 0x7faecc10f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faed3c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1124e0 con 0x7faecc101b50 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faed3c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faecc112360 con 0x7faecc1011b0 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faed3c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faecc112660 con 0x7faecc107040 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.948+0000 7faec3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 0x7faecc10ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 0x7faecc10ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54528/0 (socket says 172.21.15.44:54528) 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 -- 172.21.15.44:0/762512198 learned_addr learned my addr 172.21.15.44:0/762512198 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faed1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc1011b0 0x7faecc10e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faed1c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faecc107040 0x7faecc10f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 -- 172.21.15.44:0/762512198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faecc107040 msgr2=0x7faecc10f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faecc107040 0x7faecc10f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 -- 172.21.15.44:0/762512198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc1011b0 msgr2=0x7faecc10e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc1011b0 0x7faecc10e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 -- 172.21.15.44:0/762512198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1a6e40 con 0x7faecc101b50 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faed1200640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc1011b0 0x7faecc10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.949+0000 7faec3e00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 0x7faecc10ecf0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faeb400d950 tx=0x7faeb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.950+0000 7faeca000640 1 -- 172.21.15.44:0/762512198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4017030 con 0x7faecc101b50 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.950+0000 7faeca000640 1 -- 172.21.15.44:0/762512198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faeb4004020 con 0x7faecc101b50 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.950+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1a7130 con 0x7faecc101b50 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.950+0000 7faeca000640 1 -- 172.21.15.44:0/762512198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004a00 con 0x7faecc101b50 2024-04-20T11:32:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.950+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1a7680 con 0x7faecc101b50 2024-04-20T11:32:54.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.953+0000 7faed69c3740 1 Processor -- start 2024-04-20T11:32:54.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.953+0000 7faed69c3740 1 -- start start 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.953+0000 7faed69c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afb2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.953+0000 7faed69c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afbaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.953+0000 7faed69c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afbd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed69c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56504af17050 con 0x56504afbb4b0 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed69c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56504aebd490 con 0x56504afb28d0 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed69c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56504af13010 con 0x56504afb1fb0 2024-04-20T11:32:54.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afbaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afbaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54592/0 (socket says 172.21.15.44:54592) 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 -- 172.21.15.44:0/2246999659 learned_addr learned my addr 172.21.15.44:0/2246999659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1c00640 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afb2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 -- 172.21.15.44:0/2246999659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 msgr2=0x56504afb2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afb2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 -- 172.21.15.44:0/2246999659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 msgr2=0x56504afbd8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afbd8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faed1200640 1 -- 172.21.15.44:0/2246999659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504aec5070 con 0x56504afb28d0 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.954+0000 7faeca000640 1 -- 172.21.15.44:0/762512198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7faeb40041d0 con 0x7faecc101b50 2024-04-20T11:32:54.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.955+0000 7faed1200640 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afbaf70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faebc0114f0 tx=0x7faebc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f574100ad7c61217 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.955+0000 7faec2000640 1 -- 172.21.15.44:0/2246999659 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc01e070 con 0x56504afb28d0 2024-04-20T11:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.955+0000 7faec2000640 1 -- 172.21.15.44:0/2246999659 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faebc00fac0 con 0x56504afb28d0 2024-04-20T11:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.955+0000 7faec2000640 1 -- 172.21.15.44:0/2246999659 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc019c50 con 0x56504afb28d0 2024-04-20T11:32:54.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 -- 172.21.15.44:0/2246999659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 msgr2=0x56504afbaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afbaf70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faebc0114f0 tx=0x7faebc0119b0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 -- 172.21.15.44:0/2246999659 shutdown_connections 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afbd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afbaf70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 --2- 172.21.15.44:0/2246999659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afb2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 -- 172.21.15.44:0/2246999659 >> 172.21.15.44:0/2246999659 conn(0x56504afac6e0 msgr2=0x56504afacac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 -- 172.21.15.44:0/2246999659 shutdown_connections 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faed69c3740 1 -- 172.21.15.44:0/2246999659 wait complete. 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faeca000640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7faeb00a3c30 0x7faeb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.956+0000 7faeca000640 1 -- 172.21.15.44:0/762512198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7faeb40d0080 con 0x7faecc101b50 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed1200640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7faeb00a3c30 0x7faeb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 -- start start 2024-04-20T11:32:54.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 Processor -- start 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed1200640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7faeb00a3c30 0x7faeb00a60f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faebc014040 tx=0x7faebc0096c0 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb1fb0 0x56504afb8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 0x56504afb8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56504afbe2a0 con 0x56504afb1fb0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56504aed0860 con 0x56504afbb4b0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.957+0000 7faed69c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56504afbe420 con 0x56504afb28d0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faed69c3740 10 client.0 _ll_register_callbacks cb 0x56504ae9efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faed1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb1fb0 0x56504afb8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 0x56504afb8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 0x56504afb8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44442/0 (socket says 172.21.15.44:44442) 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 -- 172.21.15.44:0/2097543840 learned_addr learned my addr 172.21.15.44:0/2097543840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faed1c00640 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 -- 172.21.15.44:0/2097543840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 msgr2=0x56504afb8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 -- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb1fb0 msgr2=0x56504afb8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb1fb0 0x56504afb8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 -- 172.21.15.44:0/2097543840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504aed4950 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faec3e00640 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 0x56504afb8ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faeb409be00 tx=0x7faeb409ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b05cb2b26bce85 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.958+0000 7faebaa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faebaa00640 1 -- 172.21.15.44:0/2097543840 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb40b5060 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faebaa00640 1 -- 172.21.15.44:0/2097543840 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7faeb40cdb30 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc00fac0 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faebaa00640 1 -- 172.21.15.44:0/2097543840 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7faeb40a04b0 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faed69c3740 10 client.28255 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56504aea2820 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faebaa00640 1 -- 172.21.15.44:0/2097543840 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faebc00fac0 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.959+0000 7faebaa00640 1 -- 172.21.15.44:0/2097543840 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7faeb409b550 con 0x56504afb28d0 2024-04-20T11:32:54.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed69c3740 10 client.28255 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed69c3740 10 client.28255 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:32:54.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed69c3740 20 client.28255 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed69c3740 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504afcb4b0 0x56504afcd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed69c3740 4 client.28255 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:54.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56504af24a70 con 0x56504afcb4b0 2024-04-20T11:32:54.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.960+0000 7faed1200640 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504afcb4b0 0x56504afcd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.961+0000 7faed1200640 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504afcb4b0 0x56504afcd8a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bc947e147f8f2e7 server_cookie=17b4dce4e751632a in_seq=0 out_seq=0 2024-04-20T11:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.961+0000 7faebaa00640 10 client.28255 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.961+0000 7faebaa00640 1 -- 172.21.15.44:0/2097543840 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56504af24a70 con 0x56504afcb4b0 2024-04-20T11:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.961+0000 7faebaa00640 10 client.28255 handle_command_reply: tid=0 2024-04-20T11:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.961+0000 7faed69c3740 1 client.28255 shutdown 2024-04-20T11:32:54.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 msgr2=0x56504afb8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 0x56504afb8ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faeb409be00 tx=0x7faeb409ba60 comp rx=0 tx=0).stop 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 shutdown_connections 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504afcb4b0 0x56504afcd8a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb28d0 0x56504afb8ab0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 --2- 172.21.15.44:0/2097543840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb1fb0 0x56504afb8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faebaa00640 0 client.28255 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 >> 172.21.15.44:0/2097543840 conn(0x56504afac6e0 msgr2=0x56504afae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 shutdown_connections 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 1 -- 172.21.15.44:0/2097543840 wait complete. 2024-04-20T11:32:54.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.962+0000 7faed69c3740 20 client.28255 trim_cache size 0 max 16384 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 Processor -- start 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 -- start start 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afd3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1aff10 con 0x56504afbb4b0 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b1a50 con 0x56504afb28d0 2024-04-20T11:32:54.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.977+0000 7faed69c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b05e0 con 0x56504afb1fb0 2024-04-20T11:32:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afd3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afd3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54550/0 (socket says 172.21.15.44:54550) 2024-04-20T11:32:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1c00640 1 -- 172.21.15.44:0/3714607008 learned_addr learned my addr 172.21.15.44:0/3714607008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faec3e00640 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afd3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 -- 172.21.15.44:0/3714607008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 msgr2=0x56504afd3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afd3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 -- 172.21.15.44:0/3714607008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 msgr2=0x56504afd3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afd3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 -- 172.21.15.44:0/3714607008 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504ac293f0 con 0x56504afb1fb0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1c00640 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afd3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.978+0000 7faed1200640 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc7160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faebc0198f0 tx=0x7faebc0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56eafadade4bf40f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faec1600640 1 -- 172.21.15.44:0/3714607008 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc01e040 con 0x56504afb1fb0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faec1600640 1 -- 172.21.15.44:0/3714607008 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faebc036070 con 0x56504afb1fb0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faec1600640 1 -- 172.21.15.44:0/3714607008 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc025450 con 0x56504afb1fb0 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 -- 172.21.15.44:0/3714607008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 msgr2=0x56504afc7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc7160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faebc0198f0 tx=0x7faebc0119b0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 -- 172.21.15.44:0/3714607008 shutdown_connections 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afbb4b0 0x56504afd3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afb28d0 0x56504afd3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 --2- 172.21.15.44:0/3714607008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc7160 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.979+0000 7faed69c3740 1 -- 172.21.15.44:0/3714607008 >> 172.21.15.44:0/3714607008 conn(0x56504afac6e0 msgr2=0x56504afb7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.980+0000 7faed69c3740 1 -- 172.21.15.44:0/3714607008 shutdown_connections 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.980+0000 7faed69c3740 1 -- 172.21.15.44:0/3714607008 wait complete. 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.980+0000 7faed69c3740 1 -- start start 2024-04-20T11:32:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.980+0000 7faed69c3740 1 Processor -- start 2024-04-20T11:32:54.982 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:54 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/4204259757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb28d0 0x56504afb4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b1380 con 0x56504afb28d0 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b0cb0 con 0x56504afbb4b0 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1adf00 con 0x56504afb1fb0 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed69c3740 10 client.0 _ll_register_callbacks cb 0x7faeb409bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faec3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb28d0 0x56504afb4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54622/0 (socket says 172.21.15.44:54622) 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 -- 172.21.15.44:0/2330765112 learned_addr learned my addr 172.21.15.44:0/2330765112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1200640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 -- 172.21.15.44:0/2330765112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 msgr2=0x56504afc3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 -- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb28d0 msgr2=0x56504afb4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb28d0 0x56504afb4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faed1c00640 1 -- 172.21.15.44:0/2330765112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504acf37d0 con 0x56504afbb4b0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.981+0000 7faec3e00640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb28d0 0x56504afb4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faed1c00640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb55b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faec402c620 tx=0x7faec4032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1063835328f7c174 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faebaa00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faebaa00640 1 -- 172.21.15.44:0/2330765112 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4039040 con 0x56504afbb4b0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faebaa00640 1 -- 172.21.15.44:0/2330765112 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7faec4003970 con 0x56504afbb4b0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc036070 con 0x56504afbb4b0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faebaa00640 1 -- 172.21.15.44:0/2330765112 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7faec4003b70 con 0x56504afbb4b0 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faed69c3740 10 client.28253 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.982+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56504b0318c0 con 0x56504afbb4b0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faebaa00640 1 -- 172.21.15.44:0/2330765112 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faebc036070 con 0x56504afbb4b0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faebaa00640 1 -- 172.21.15.44:0/2330765112 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7faec404a640 con 0x56504afbb4b0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed69c3740 10 client.28253 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed69c3740 10 client.28253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed69c3740 20 client.28253 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed69c3740 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504b031ff0 0x56504b034410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed69c3740 4 client.28253 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 --> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56504af1e8e0 con 0x56504b031ff0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.983+0000 7faed1200640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504b031ff0 0x56504b034410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed1200640 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504b031ff0 0x56504b034410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=216a3d1b05fb2061 server_cookie=66b19ac9918d8c54 in_seq=0 out_seq=0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faebaa00640 10 client.28253 ms_handle_connect on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faebaa00640 1 -- 172.21.15.44:0/2330765112 <== mds.1 v2:172.21.15.44:6838/2240418551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56504af1e8e0 con 0x56504b031ff0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faebaa00640 10 client.28253 handle_command_reply: tid=0 2024-04-20T11:32:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 client.28253 shutdown 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 msgr2=0x56504afb55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb55b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faec402c620 tx=0x7faec4032c60 comp rx=0 tx=0).stop 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 shutdown_connections 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:6838/2240418551,v1:172.21.15.44:6839/2240418551] conn(0x56504b031ff0 0x56504b034410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56504afbb4b0 0x56504afb55b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56504afb28d0 0x56504afb4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 --2- 172.21.15.44:0/2330765112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56504afb1fb0 0x56504afc3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faebaa00640 0 client.28253 ms_handle_reset on v2:172.21.15.44:6838/2240418551 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.984+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 >> 172.21.15.44:0/2330765112 conn(0x56504afac6e0 msgr2=0x56504afb8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 shutdown_connections 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed69c3740 1 -- 172.21.15.44:0/2330765112 wait complete. 2024-04-20T11:32:54.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed69c3740 20 client.28253 trim_cache size 0 max 16384 2024-04-20T11:32:54.989 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7faeb00a3c30 msgr2=0x7faeb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7faeb00a3c30 0x7faeb00a60f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faebc014040 tx=0x7faebc0096c0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 msgr2=0x7faecc10ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 0x7faecc10ecf0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faeb400d950 tx=0x7faeb400de10 comp rx=0 tx=0).stop 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 shutdown_connections 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faecc107040 0x7faecc10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7faeb00a3c30 0x7faeb00a60f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faecc101b50 0x7faecc10ecf0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.985+0000 7faed3c00640 1 --2- 172.21.15.44:0/762512198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faecc1011b0 0x7faecc10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.986+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 >> 172.21.15.44:0/762512198 conn(0x7faecc076c60 msgr2=0x7faecc0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.986+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 shutdown_connections 2024-04-20T11:32:54.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:54.986+0000 7faed3c00640 1 -- 172.21.15.44:0/762512198 wait complete. 2024-04-20T11:32:55.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T11:32:55.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.208+0000 7eff05600640 1 -- 172.21.15.44:0/1427779422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff001038e0 msgr2=0x7eff0010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.208+0000 7eff05600640 1 --2- 172.21.15.44:0/1427779422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff001038e0 0x7eff0010bdc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efee802bcc0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 -- 172.21.15.44:0/1427779422 shutdown_connections 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 --2- 172.21.15.44:0/1427779422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff0010c3e0 0x7eff0010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 --2- 172.21.15.44:0/1427779422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff001038e0 0x7eff0010bdc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 --2- 172.21.15.44:0/1427779422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff00102fc0 0x7eff001033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 -- 172.21.15.44:0/1427779422 >> 172.21.15.44:0/1427779422 conn(0x7eff000fc7e0 msgr2=0x7eff000fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 -- 172.21.15.44:0/1427779422 shutdown_connections 2024-04-20T11:32:55.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.209+0000 7eff05600640 1 -- 172.21.15.44:0/1427779422 wait complete. 2024-04-20T11:32:55.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.210+0000 7eff05600640 1 Processor -- start 2024-04-20T11:32:55.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.210+0000 7eff05600640 1 -- start start 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff05600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff00102fc0 0x7eff00100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff001038e0 0x7eff00101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff05600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff0010c3e0 0x7eff00101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff05600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff00112300 con 0x7eff0010c3e0 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff05600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff00112180 con 0x7eff00102fc0 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff05600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff00112480 con 0x7eff001038e0 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff0010c3e0 0x7eff00101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7efeffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff00102fc0 0x7eff00100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff0010c3e0 0x7eff00101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54578/0 (socket says 172.21.15.44:54578) 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7eff04c00640 1 -- 172.21.15.44:0/1396114295 learned_addr learned my addr 172.21.15.44:0/1396114295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:55.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7efeff400640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff001038e0 0x7eff00101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7efeffe00640 1 -- 172.21.15.44:0/1396114295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff001038e0 msgr2=0x7eff00101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7efeffe00640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff001038e0 0x7eff00101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7efeffe00640 1 -- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff0010c3e0 msgr2=0x7eff00101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.211+0000 7efeffe00640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff0010c3e0 0x7eff00101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7efeffe00640 1 -- 172.21.15.44:0/1396114295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff001a6c80 con 0x7eff00102fc0 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7efeffe00640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff00102fc0 0x7eff00100c10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efef000eb00 tx=0x7efef000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7efefcc00640 1 -- 172.21.15.44:0/1396114295 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0010430 con 0x7eff00102fc0 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7efefcc00640 1 -- 172.21.15.44:0/1396114295 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efef0010ad0 con 0x7eff00102fc0 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff001a6f70 con 0x7eff00102fc0 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7efefcc00640 1 -- 172.21.15.44:0/1396114295 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0017c10 con 0x7eff00102fc0 2024-04-20T11:32:55.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.212+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff001a73e0 con 0x7eff00102fc0 2024-04-20T11:32:55.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.215+0000 7eff09968740 1 Processor -- start 2024-04-20T11:32:55.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 -- start start 2024-04-20T11:32:55.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7efefcc00640 1 -- 172.21.15.44:0/1396114295 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efef0010c80 con 0x7eff00102fc0 2024-04-20T11:32:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d261b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261ba90 0x55e8d2624130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d2624670 0x55e8d2626a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e8d2580050 con 0x55e8d261ba90 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8d2526490 con 0x55e8d261b170 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff09968740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e8d257c010 con 0x55e8d2624670 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff04c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d261b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff04c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d261b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:54646/0 (socket says 172.21.15.44:54646) 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7eff04c00640 1 -- 172.21.15.44:0/179824054 learned_addr learned my addr 172.21.15.44:0/179824054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7efeffe00640 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261ba90 0x55e8d2624130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.216+0000 7efeff400640 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d2624670 0x55e8d2626a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7eff04c00640 1 -- 172.21.15.44:0/179824054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d2624670 msgr2=0x55e8d2626a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7eff04c00640 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d2624670 0x55e8d2626a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7eff04c00640 1 -- 172.21.15.44:0/179824054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261ba90 msgr2=0x55e8d2624130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7eff04c00640 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261ba90 0x55e8d2624130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7eff04c00640 1 -- 172.21.15.44:0/179824054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8d252e070 con 0x55e8d261b170 2024-04-20T11:32:55.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7eff04c00640 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d261b550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efef8009ce0 tx=0x7efef8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdd31dca876fdbe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7efefcc00640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efee40a3d50 0x7efee40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.217+0000 7efef5200640 1 -- 172.21.15.44:0/179824054 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef801e070 con 0x55e8d261b170 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7efef5200640 1 -- 172.21.15.44:0/179824054 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efef8019b30 con 0x55e8d261b170 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7efef5200640 1 -- 172.21.15.44:0/179824054 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef800fb50 con 0x55e8d261b170 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7efefcc00640 1 -- 172.21.15.44:0/1396114295 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7efef00d1130 con 0x7eff00102fc0 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7efeffe00640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efee40a3d50 0x7efee40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 -- 172.21.15.44:0/179824054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 msgr2=0x55e8d261b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d261b550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efef8009ce0 tx=0x7efef8011810 comp rx=0 tx=0).stop 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 -- 172.21.15.44:0/179824054 shutdown_connections 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d2624670 0x55e8d2626a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261ba90 0x55e8d2624130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 --2- 172.21.15.44:0/179824054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d261b550 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 -- 172.21.15.44:0/179824054 >> 172.21.15.44:0/179824054 conn(0x55e8d26158a0 msgr2=0x55e8d2615c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7eff09968740 1 -- 172.21.15.44:0/179824054 shutdown_connections 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.218+0000 7efeffe00640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efee40a3d50 0x7efee40a6210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efef00ce970 tx=0x7efef00d4c80 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.219+0000 7eff09968740 1 -- 172.21.15.44:0/179824054 wait complete. 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.219+0000 7eff09968740 1 -- start start 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.219+0000 7eff09968740 1 Processor -- start 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d2621700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2621c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2622180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e8d2627460 con 0x55e8d2624670 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8d2539860 con 0x55e8d261b170 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e8d26275e0 con 0x55e8d261ba90 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff09968740 10 client.0 _ll_register_callbacks cb 0x55e8d2507fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff04c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d2621700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2622180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2622180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54604/0 (socket says 172.21.15.44:54604) 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7eff04c00640 1 -- 172.21.15.44:0/3613382934 learned_addr learned my addr 172.21.15.44:0/3613382934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.220+0000 7efeff400640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2621c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff04c00640 1 -- 172.21.15.44:0/3613382934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 msgr2=0x55e8d2621c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff04c00640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2621c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff04c00640 1 -- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 msgr2=0x55e8d2621700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff04c00640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d2621700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff04c00640 1 -- 172.21.15.44:0/3613382934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8d253d950 con 0x55e8d2624670 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7efeff400640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2621c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff04c00640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2622180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efef8009cb0 tx=0x7efef8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1292ce84fe1e1626 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7efeed600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-20T11:32:55.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7efeed600640 1 -- 172.21.15.44:0/3613382934 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef802e060 con 0x55e8d2624670 2024-04-20T11:32:55.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef8019b30 con 0x55e8d2624670 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.221+0000 7efeed600640 1 -- 172.21.15.44:0/3613382934 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7efef8005240 con 0x55e8d2624670 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.222+0000 7efeed600640 1 -- 172.21.15.44:0/3613382934 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efef801e030 con 0x55e8d2624670 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.222+0000 7eff09968740 10 client.18153 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.222+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e8d250b820 con 0x55e8d2624670 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.222+0000 7efeed600640 1 -- 172.21.15.44:0/3613382934 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efef8019b30 con 0x55e8d2624670 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7efeed600640 1 -- 172.21.15.44:0/3613382934 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7efef80048c0 con 0x55e8d2624670 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7eff09968740 10 client.18153 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7eff09968740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7eff09968740 20 client.18153 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7eff09968740 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d2634600 0x55e8d26369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7eff09968740 4 client.18153 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8d258da70 con 0x55e8d2634600 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.223+0000 7efeff400640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d2634600 0x55e8d26369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.224+0000 7efeff400640 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d2634600 0x55e8d26369f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969e30be7cb10c7 server_cookie=88ed9751291e8869 in_seq=0 out_seq=0 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.224+0000 7efeed600640 10 client.18153 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.224+0000 7efeed600640 1 -- 172.21.15.44:0/3613382934 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8d258da70 con 0x55e8d2634600 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.224+0000 7efeed600640 10 client.18153 handle_command_reply: tid=0 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 client.18153 shutdown 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 msgr2=0x55e8d2622180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2622180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efef8009cb0 tx=0x7efef8000bc0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 shutdown_connections 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d2634600 0x55e8d26369f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2622180 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2621c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 --2- 172.21.15.44:0/3613382934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d2621700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7efeed600640 0 client.18153 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 >> 172.21.15.44:0/3613382934 conn(0x55e8d26158a0 msgr2=0x55e8d26175f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 shutdown_connections 2024-04-20T11:32:55.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 1 -- 172.21.15.44:0/3613382934 wait complete. 2024-04-20T11:32:55.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.225+0000 7eff09968740 20 client.18153 trim_cache size 0 max 16384 2024-04-20T11:32:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.240+0000 7eff09968740 1 Processor -- start 2024-04-20T11:32:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.240+0000 7eff09968740 1 -- start start 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.240+0000 7eff09968740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 0x55e8d2630320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.240+0000 7eff09968740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2623e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff09968740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d2624670 0x55e8d263c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff09968740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff001afcf0 con 0x55e8d261b170 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff09968740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff001b1830 con 0x55e8d2624670 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff09968740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff001b03c0 con 0x55e8d261ba90 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 0x55e8d2630320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 0x55e8d2630320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54614/0 (socket says 172.21.15.44:54614) 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 -- 172.21.15.44:0/3179628655 learned_addr learned my addr 172.21.15.44:0/3179628655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7efeff400640 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2623e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d2624670 0x55e8d263c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 -- 172.21.15.44:0/3179628655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 msgr2=0x55e8d2623e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2623e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 -- 172.21.15.44:0/3179628655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d2624670 msgr2=0x55e8d263c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.241+0000 7eff04c00640 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d2624670 0x55e8d263c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.242+0000 7eff04c00640 1 -- 172.21.15.44:0/3179628655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8d22923f0 con 0x55e8d261b170 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.242+0000 7eff04c00640 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 0x55e8d2630320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55e8d26308a0 tx=0x7efef8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241f77010afe91a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.242+0000 7efeefe00640 1 -- 172.21.15.44:0/3179628655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef802e040 con 0x55e8d261b170 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.242+0000 7efeefe00640 1 -- 172.21.15.44:0/3179628655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efef801f040 con 0x55e8d261b170 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.242+0000 7efeefe00640 1 -- 172.21.15.44:0/3179628655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef800f920 con 0x55e8d261b170 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 -- 172.21.15.44:0/3179628655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 msgr2=0x55e8d2630320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 0x55e8d2630320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55e8d26308a0 tx=0x7efef8005970 comp rx=0 tx=0).stop 2024-04-20T11:32:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 -- 172.21.15.44:0/3179628655 shutdown_connections 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d2624670 0x55e8d263c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2623e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 --2- 172.21.15.44:0/3179628655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d261b170 0x55e8d2630320 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 -- 172.21.15.44:0/3179628655 >> 172.21.15.44:0/3179628655 conn(0x55e8d26158a0 msgr2=0x55e8d2614960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 -- 172.21.15.44:0/3179628655 shutdown_connections 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 -- 172.21.15.44:0/3179628655 wait complete. 2024-04-20T11:32:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.243+0000 7eff09968740 1 -- start start 2024-04-20T11:32:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 Processor -- start 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d262c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2627df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2628330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff001b1160 con 0x55e8d2624670 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff001b0a90 con 0x55e8d261b170 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff001adce0 con 0x55e8d261ba90 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.244+0000 7eff09968740 10 client.0 _ll_register_callbacks cb 0x7efef8009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2628330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7eff04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2628330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54620/0 (socket says 172.21.15.44:54620) 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7eff04c00640 1 -- 172.21.15.44:0/118152798 learned_addr learned my addr 172.21.15.44:0/118152798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2627df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7eff04c00640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d262c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 -- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 msgr2=0x55e8d262c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d262c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 -- 172.21.15.44:0/118152798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 msgr2=0x55e8d2628330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2628330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 -- 172.21.15.44:0/118152798 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8d235c7d0 con 0x55e8d261ba90 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7eff04c00640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2628330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeff400640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2627df0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efee8000c00 tx=0x7efee8004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a83ecb7ac70070 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeed600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.245+0000 7efeed600640 1 -- 172.21.15.44:0/118152798 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8005040 con 0x55e8d261ba90 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7efeed600640 1 -- 172.21.15.44:0/118152798 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7efee80051e0 con 0x55e8d261ba90 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef801f040 con 0x55e8d261ba90 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7efeed600640 1 -- 172.21.15.44:0/118152798 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efee80053e0 con 0x55e8d261ba90 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 10 client.28279 fetch_fsmap learned FSMap version 55 2024-04-20T11:32:55.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e8d269a4b0 con 0x55e8d261ba90 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7efeed600640 1 -- 172.21.15.44:0/118152798 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efef801f040 con 0x55e8d261ba90 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7efeed600640 1 -- 172.21.15.44:0/118152798 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8355+0+0 (secure 0 0 0) 0x7efee8003890 con 0x55e8d261ba90 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 10 client.28279 fetch_fsmap finished waiting for FSMap version 55 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 10 client.28279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 20 client.28279 populate_metadata read hostname 'smithi044' 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d269ac10 0x55e8d269d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 4 client.28279 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.246+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 --> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8d25878e0 con 0x55e8d269ac10 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.247+0000 7eff04c00640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d269ac10 0x55e8d269d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.247+0000 7eff04c00640 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d269ac10 0x55e8d269d030 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305aa7b5de075182 server_cookie=77f03ab3197c5403 in_seq=0 out_seq=0 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.247+0000 7efeed600640 10 client.28279 ms_handle_connect on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7efeed600640 1 -- 172.21.15.44:0/118152798 <== mds.0 v2:172.21.15.78:6838/315239720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8d25878e0 con 0x55e8d269ac10 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7efeed600640 10 client.28279 handle_command_reply: tid=0 2024-04-20T11:32:55.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 client.28279 shutdown 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 msgr2=0x55e8d2627df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2627df0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efee8000c00 tx=0x7efee8004290 comp rx=0 tx=0).stop 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 shutdown_connections 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:6838/315239720,v1:172.21.15.78:6839/315239720] conn(0x55e8d269ac10 0x55e8d269d030 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8d2624670 0x55e8d2628330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e8d261ba90 0x55e8d2627df0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 --2- 172.21.15.44:0/118152798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8d261b170 0x55e8d262c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7efeed600640 0 client.28279 ms_handle_reset on v2:172.21.15.78:6838/315239720 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 >> 172.21.15.44:0/118152798 conn(0x55e8d26158a0 msgr2=0x55e8d26216c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 shutdown_connections 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 1 -- 172.21.15.44:0/118152798 wait complete. 2024-04-20T11:32:55.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.248+0000 7eff09968740 20 client.28279 trim_cache size 0 max 16384 2024-04-20T11:32:55.252 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-04-20T11:32:55.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efee40a3d50 msgr2=0x7efee40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efee40a3d50 0x7efee40a6210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efef00ce970 tx=0x7efef00d4c80 comp rx=0 tx=0).stop 2024-04-20T11:32:55.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff00102fc0 msgr2=0x7eff00100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:55.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff00102fc0 0x7eff00100c10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efef000eb00 tx=0x7efef000c2d0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 shutdown_connections 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7efee40a3d50 0x7efee40a6210 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff0010c3e0 0x7eff00101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff001038e0 0x7eff00101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 --2- 172.21.15.44:0/1396114295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff00102fc0 0x7eff00100c10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 >> 172.21.15.44:0/1396114295 conn(0x7eff000fc7e0 msgr2=0x7eff000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 shutdown_connections 2024-04-20T11:32:55.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:55.249+0000 7eff05600640 1 -- 172.21.15.44:0/1396114295 wait complete. 2024-04-20T11:32:55.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:55 smithi044 ceph-mon[16855]: pgmap v320: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:32:55.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:55 smithi078 ceph-mon[21464]: pgmap v320: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:32:55.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:55 smithi165 ceph-mon[21975]: pgmap v320: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:32:56.268 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-20T11:32:56.269 INFO:tasks.fwd_scrub:done joining 2024-04-20T11:32:56.269 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-20T11:32:56.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:32:56.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 -- 172.21.15.44:0/3813200331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4069d10 msgr2=0x7fadc41069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3813200331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4069d10 0x7fadc41069d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009900 tx=0x7fadb002bc50 comp rx=0 tx=0).stop 2024-04-20T11:32:56.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 -- 172.21.15.44:0/3813200331 shutdown_connections 2024-04-20T11:32:56.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3813200331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4107100 0x7fadc410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3813200331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4069d10 0x7fadc41069d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3813200331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc4069360 0x7fadc4069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 -- 172.21.15.44:0/3813200331 >> 172.21.15.44:0/3813200331 conn(0x7fadc40767a0 msgr2=0x7fadc4076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:56.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.509+0000 7fadc9e00640 1 -- 172.21.15.44:0/3813200331 shutdown_connections 2024-04-20T11:32:56.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.510+0000 7fadc9e00640 1 -- 172.21.15.44:0/3813200331 wait complete. 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.510+0000 7fadc9e00640 1 Processor -- start 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 -- start start 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 0x7fadc419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4107100 0x7fadc419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc41985a0 0x7fadc4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41122a0 con 0x7fadc4069360 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4112120 con 0x7fadc4107100 2024-04-20T11:32:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4112420 con 0x7fadc41985a0 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 0x7fadc419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 0x7fadc419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54634/0 (socket says 172.21.15.44:54634) 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc8a00640 1 -- 172.21.15.44:0/3267962154 learned_addr learned my addr 172.21.15.44:0/3267962154 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.511+0000 7fadc9400640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc41985a0 0x7fadc4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc3e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4107100 0x7fadc419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc8a00640 1 -- 172.21.15.44:0/3267962154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc41985a0 msgr2=0x7fadc4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc8a00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc41985a0 0x7fadc4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc8a00640 1 -- 172.21.15.44:0/3267962154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4107100 msgr2=0x7fadc419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc8a00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4107100 0x7fadc419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc8a00640 1 -- 172.21.15.44:0/3267962154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41990c0 con 0x7fadc4069360 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc8a00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 0x7fadc419e3f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ea60 tx=0x7fadb800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800cde0 con 0x7fadc4069360 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadb80035f0 con 0x7fadc4069360 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8010720 con 0x7fadc4069360 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.512+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41993b0 con 0x7fadc4069360 2024-04-20T11:32:56.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.513+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a5600 con 0x7fadc4069360 2024-04-20T11:32:56.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.514+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc4068ca0 con 0x7fadc4069360 2024-04-20T11:32:56.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.516+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fadb80037a0 con 0x7fadc4069360 2024-04-20T11:32:56.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.517+0000 7fadc1600640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadac0a3d20 0x7fadac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.517+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fadb80d0ce0 con 0x7fadc4069360 2024-04-20T11:32:56.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.517+0000 7fadc3e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadac0a3d20 0x7fadac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.518+0000 7fadc3e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadac0a3d20 0x7fadac0a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009230 tx=0x7fadb002c690 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:56.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.522+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fadb809d360 con 0x7fadc4069360 2024-04-20T11:32:56.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.730+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadc4199ad0 con 0x7fadc4069360 2024-04-20T11:32:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.734+0000 7fadc1600640 1 -- 172.21.15.44:0/3267962154 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+22977 (secure 0 0 0) 0x7fadb80bbe20 con 0x7fadc4069360 2024-04-20T11:32:56.734 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:32:56.735 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":96,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:32:13.067721+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:32:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadac0a3d20 msgr2=0x7fadac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadac0a3d20 0x7fadac0a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009230 tx=0x7fadb002c690 comp rx=0 tx=0).stop 2024-04-20T11:32:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 msgr2=0x7fadc419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 0x7fadc419e3f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ea60 tx=0x7fadb800ef20 comp rx=0 tx=0).stop 2024-04-20T11:32:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 shutdown_connections 2024-04-20T11:32:56.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc41985a0 0x7fadc4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadc4107100 0x7fadc419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fadac0a3d20 0x7fadac0a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 --2- 172.21.15.44:0/3267962154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadc4069360 0x7fadc419e3f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.737+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 >> 172.21.15.44:0/3267962154 conn(0x7fadc40767a0 msgr2=0x7fadc40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.738+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 shutdown_connections 2024-04-20T11:32:56.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.738+0000 7fadc9e00640 1 -- 172.21.15.44:0/3267962154 wait complete. 2024-04-20T11:32:56.758 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:32:56.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 -- 172.21.15.44:0/2023125512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f81041a0 msgr2=0x7fc7f8104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2023125512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f81041a0 0x7fc7f8104580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009860 tx=0x7fc7e802bb70 comp rx=0 tx=0).stop 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 -- 172.21.15.44:0/2023125512 shutdown_connections 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2023125512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8109270 0x7fc7f8110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2023125512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8104b50 0x7fc7f8108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2023125512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f81041a0 0x7fc7f8104580 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 -- 172.21.15.44:0/2023125512 >> 172.21.15.44:0/2023125512 conn(0x7fc7f80ffa00 msgr2=0x7fc7f8101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 -- 172.21.15.44:0/2023125512 shutdown_connections 2024-04-20T11:32:56.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.975+0000 7fc7fd400640 1 -- 172.21.15.44:0/2023125512 wait complete. 2024-04-20T11:32:56.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.976+0000 7fc7fd400640 1 Processor -- start 2024-04-20T11:32:56.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 -- start start 2024-04-20T11:32:56.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f81041a0 0x7fc7f8076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f8109270 0x7fc7f80769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8076ef0 0x7fc7f81aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8114460 con 0x7fc7f8076ef0 2024-04-20T11:32:56.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81142e0 con 0x7fc7f81041a0 2024-04-20T11:32:56.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.977+0000 7fc7fd400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81145e0 con 0x7fc7f8109270 2024-04-20T11:32:56.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7fca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8076ef0 0x7fc7f81aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f81041a0 0x7fc7f8076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f8109270 0x7fc7f80769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7fca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8076ef0 0x7fc7f81aaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54648/0 (socket says 172.21.15.44:54648) 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7fca00640 1 -- 172.21.15.44:0/2295521713 learned_addr learned my addr 172.21.15.44:0/2295521713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f8109270 0x7fc7f80769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44566/0 (socket says 172.21.15.44:44566) 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7f7e00640 1 -- 172.21.15.44:0/2295521713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f8109270 msgr2=0x7fc7f80769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7f7e00640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f8109270 0x7fc7f80769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7f7e00640 1 -- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8076ef0 msgr2=0x7fc7f81aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7f7e00640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8076ef0 0x7fc7f81aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.978+0000 7fc7f7e00640 1 -- 172.21.15.44:0/2295521713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81ab4b0 con 0x7fc7f81041a0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.979+0000 7fc7f7e00640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f81041a0 0x7fc7f8076470 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc7e802c0b0 tx=0x7fc7e80045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.979+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8041020 con 0x7fc7f81041a0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.979+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81ab6e0 con 0x7fc7f81041a0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.979+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc7e802ca10 con 0x7fc7f81041a0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.979+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8039040 con 0x7fc7f81041a0 2024-04-20T11:32:56.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.979+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81abbf0 con 0x7fc7f81041a0 2024-04-20T11:32:56.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.980+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8105670 con 0x7fc7f81041a0 2024-04-20T11:32:56.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.982+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc7e8044020 con 0x7fc7f81041a0 2024-04-20T11:32:56.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.984+0000 7fc7f5600640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc7dc0a3da0 0x7fc7dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:56.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.984+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fc7e80f1d00 con 0x7fc7f81041a0 2024-04-20T11:32:56.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.984+0000 7fc7ef400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc7dc0a3da0 0x7fc7dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:56.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.985+0000 7fc7ef400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc7dc0a3da0 0x7fc7dc0a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8077be0 tx=0x7fc7e0009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:56.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:56.988+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc7e804b020 con 0x7fc7f81041a0 2024-04-20T11:32:57.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.194+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7f810c500 con 0x7fc7f81041a0 2024-04-20T11:32:57.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.194+0000 7fc7f5600640 1 -- 172.21.15.44:0/2295521713 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fc7e80be490 con 0x7fc7f81041a0 2024-04-20T11:32:57.194 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:32:57.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc7dc0a3da0 msgr2=0x7fc7dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc7dc0a3da0 0x7fc7dc0a6260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8077be0 tx=0x7fc7e0009260 comp rx=0 tx=0).stop 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f81041a0 msgr2=0x7fc7f8076470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f81041a0 0x7fc7f8076470 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc7e802c0b0 tx=0x7fc7e80045b0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 shutdown_connections 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fc7dc0a3da0 0x7fc7dc0a6260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f8076ef0 0x7fc7f81aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f8109270 0x7fc7f80769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 --2- 172.21.15.44:0/2295521713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f81041a0 0x7fc7f8076470 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 >> 172.21.15.44:0/2295521713 conn(0x7fc7f80ffa00 msgr2=0x7fc7f8101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 shutdown_connections 2024-04-20T11:32:57.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.198+0000 7fc7fd400640 1 -- 172.21.15.44:0/2295521713 wait complete. 2024-04-20T11:32:57.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:32:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 -- 172.21.15.44:0/502249108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 msgr2=0x7f5594115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 --2- 172.21.15.44:0/502249108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f5594115400 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f558c009cd0 tx=0x7f558c02c180 comp rx=0 tx=0).stop 2024-04-20T11:32:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 -- 172.21.15.44:0/502249108 shutdown_connections 2024-04-20T11:32:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 --2- 172.21.15.44:0/502249108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f5594115400 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 --2- 172.21.15.44:0/502249108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594074b50 0x7f5594112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 --2- 172.21.15.44:0/502249108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5594074230 0x7f5594074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 -- 172.21.15.44:0/502249108 >> 172.21.15.44:0/502249108 conn(0x7f55940793b0 msgr2=0x7f55940797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 -- 172.21.15.44:0/502249108 shutdown_connections 2024-04-20T11:32:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.437+0000 7f559a000640 1 -- 172.21.15.44:0/502249108 wait complete. 2024-04-20T11:32:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.438+0000 7f559a000640 1 Processor -- start 2024-04-20T11:32:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.438+0000 7f559a000640 1 -- start start 2024-04-20T11:32:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f559a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594074230 0x7f559419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f559a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5594074b50 0x7f559419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f559a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f55941a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f559a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5594118a50 con 0x7f5594113010 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f559a000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55941188d0 con 0x7f5594074230 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f559a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5594118bd0 con 0x7f5594074b50 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f5593e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f55941a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f5593e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f55941a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54668/0 (socket says 172.21.15.44:54668) 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f5593e00640 1 -- 172.21.15.44:0/2779877704 learned_addr learned my addr 172.21.15.44:0/2779877704 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f5593400640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594074230 0x7f559419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.439+0000 7f558be00640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5594074b50 0x7f559419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5593e00640 1 -- 172.21.15.44:0/2779877704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5594074b50 msgr2=0x7f559419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5593e00640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5594074b50 0x7f559419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5593e00640 1 -- 172.21.15.44:0/2779877704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594074230 msgr2=0x7f559419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5593e00640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594074230 0x7f559419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5593e00640 1 -- 172.21.15.44:0/2779877704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55941a3f40 con 0x7f5594113010 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5593e00640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f55941a3810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f558c005cc0 tx=0x7f558c0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c039040 con 0x7f5594113010 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f558c044070 con 0x7f5594113010 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0043c0 con 0x7f5594113010 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55941a41d0 con 0x7f5594113010 2024-04-20T11:32:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.440+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941abac0 con 0x7f5594113010 2024-04-20T11:32:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.442+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5594075970 con 0x7f5594113010 2024-04-20T11:32:57.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.444+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f558c004560 con 0x7f5594113010 2024-04-20T11:32:57.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.445+0000 7f5590c00640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55780a3da0 0x7f55780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.445+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f558c0f2a50 con 0x7f5594113010 2024-04-20T11:32:57.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.445+0000 7f5593400640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55780a3da0 0x7f55780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.446+0000 7f5593400640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55780a3da0 0x7f55780a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5584009bd0 tx=0x7f5584005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:57.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.449+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f558c0bf1e0 con 0x7f5594113010 2024-04-20T11:32:57.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.656+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55941a02a0 con 0x7f5594113010 2024-04-20T11:32:57.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.657+0000 7f5590c00640 1 -- 172.21.15.44:0/2779877704 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f558c0beae0 con 0x7f5594113010 2024-04-20T11:32:57.657 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55780a3da0 msgr2=0x7f55780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55780a3da0 0x7f55780a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5584009bd0 tx=0x7f5584005f90 comp rx=0 tx=0).stop 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 msgr2=0x7f55941a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f55941a3810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f558c005cc0 tx=0x7f558c0346f0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 shutdown_connections 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f55780a3da0 0x7f55780a6260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594113010 0x7f55941a3810 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5594074b50 0x7f559419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 --2- 172.21.15.44:0/2779877704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594074230 0x7f559419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 >> 172.21.15.44:0/2779877704 conn(0x7f55940793b0 msgr2=0x7f55940fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:57.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 shutdown_connections 2024-04-20T11:32:57.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.660+0000 7f559a000640 1 -- 172.21.15.44:0/2779877704 wait complete. 2024-04-20T11:32:57.679 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:32:57.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:57 smithi044 ceph-mon[16855]: pgmap v321: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:32:57.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:57 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3267962154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:32:57.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:57 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2295521713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:32:57.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.900+0000 7f277b200640 1 -- 172.21.15.44:0/1644346735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 msgr2=0x7f277c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.900+0000 7f277b200640 1 --2- 172.21.15.44:0/1644346735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c10e790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2774009900 tx=0x7f277402bcc0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 -- 172.21.15.44:0/1644346735 shutdown_connections 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 --2- 172.21.15.44:0/1644346735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c10e790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 --2- 172.21.15.44:0/1644346735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 0x7f277c10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 --2- 172.21.15.44:0/1644346735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 0x7f277c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 -- 172.21.15.44:0/1644346735 >> 172.21.15.44:0/1644346735 conn(0x7f277c077a40 msgr2=0x7f277c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 -- 172.21.15.44:0/1644346735 shutdown_connections 2024-04-20T11:32:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.901+0000 7f277b200640 1 -- 172.21.15.44:0/1644346735 wait complete. 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 Processor -- start 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 -- start start 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 0x7f277c100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 0x7f277c1010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112210 con 0x7f277c102f40 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112090 con 0x7f277c10c310 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.902+0000 7f277b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112390 con 0x7f277c103860 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f2779e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 0x7f277c100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f2771400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 0x7f277c1010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f2779e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 0x7f277c100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54686/0 (socket says 172.21.15.44:54686) 2024-04-20T11:32:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f2779e00640 1 -- 172.21.15.44:0/452665225 learned_addr learned my addr 172.21.15.44:0/452665225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 -- 172.21.15.44:0/452665225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 msgr2=0x7f277c1010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 0x7f277c1010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 -- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 msgr2=0x7f277c100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 0x7f277c100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 -- 172.21.15.44:0/452665225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c1a6b90 con 0x7f277c10c310 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f2771400640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 0x7f277c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:32:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.903+0000 7f277a800640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c101610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f277402c200 tx=0x7f2774004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.904+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774043020 con 0x7f277c10c310 2024-04-20T11:32:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.904+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c1a6e20 con 0x7f277c10c310 2024-04-20T11:32:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.904+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2774004250 con 0x7f277c10c310 2024-04-20T11:32:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.904+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774003a80 con 0x7f277c10c310 2024-04-20T11:32:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.904+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1a7390 con 0x7f277c10c310 2024-04-20T11:32:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.905+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277c068ca0 con 0x7f277c10c310 2024-04-20T11:32:57.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.908+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2774046020 con 0x7f277c10c310 2024-04-20T11:32:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.909+0000 7f2773400640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f27600a3d20 0x7f27600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.909+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f27740f2c70 con 0x7f277c10c310 2024-04-20T11:32:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.909+0000 7f2779e00640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f27600a3d20 0x7f27600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.910+0000 7f2779e00640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f27600a3d20 0x7f27600a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f276c0104e0 tx=0x7f276c009800 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:57.911+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f277404d020 con 0x7f277c10c310 2024-04-20T11:32:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:57 smithi078 ceph-mon[21464]: pgmap v321: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:32:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:57 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3267962154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:32:57.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:57 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2295521713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:32:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:57 smithi165 ceph-mon[21975]: pgmap v321: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:32:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:57 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3267962154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:32:57.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:57 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2295521713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:32:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.117+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f277c1a7680 con 0x7f277c10c310 2024-04-20T11:32:58.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.118+0000 7f2773400640 1 -- 172.21.15.44:0/452665225 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f27740bf400 con 0x7f277c10c310 2024-04-20T11:32:58.118 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f27600a3d20 msgr2=0x7f27600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f27600a3d20 0x7f27600a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f276c0104e0 tx=0x7f276c009800 comp rx=0 tx=0).stop 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 msgr2=0x7f277c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c101610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f277402c200 tx=0x7f2774004960 comp rx=0 tx=0).stop 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 shutdown_connections 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f277c10c310 0x7f277c101610 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f277c103860 0x7f277c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f27600a3d20 0x7f27600a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 --2- 172.21.15.44:0/452665225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f277c102f40 0x7f277c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 >> 172.21.15.44:0/452665225 conn(0x7f277c077a40 msgr2=0x7f277c0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 shutdown_connections 2024-04-20T11:32:58.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.121+0000 7f277b200640 1 -- 172.21.15.44:0/452665225 wait complete. 2024-04-20T11:32:58.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 -- 172.21.15.44:0/800401321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c1064f0 msgr2=0x7f9e0c110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 --2- 172.21.15.44:0/800401321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c1064f0 0x7f9e0c110ef0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009930 tx=0x7f9e0402bc20 comp rx=0 tx=0).stop 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 -- 172.21.15.44:0/800401321 shutdown_connections 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 --2- 172.21.15.44:0/800401321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c1064f0 0x7f9e0c110ef0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 --2- 172.21.15.44:0/800401321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e0c105b50 0x7f9e0c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 --2- 172.21.15.44:0/800401321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e0c104270 0x7f9e0c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 -- 172.21.15.44:0/800401321 >> 172.21.15.44:0/800401321 conn(0x7f9e0c0ff9d0 msgr2=0x7f9e0c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 -- 172.21.15.44:0/800401321 shutdown_connections 2024-04-20T11:32:58.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.353+0000 7f9e0b200640 1 -- 172.21.15.44:0/800401321 wait complete. 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.354+0000 7f9e0b200640 1 Processor -- start 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.354+0000 7f9e0b200640 1 -- start start 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.354+0000 7f9e0b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 0x7f9e0c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e0b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e0c105b50 0x7f9e0c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e0b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e0c1064f0 0x7f9e0c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e0b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c114540 con 0x7f9e0c105b50 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e0b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1143c0 con 0x7f9e0c1064f0 2024-04-20T11:32:58.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e0b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1146c0 con 0x7f9e0c104270 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 0x7f9e0c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e0c105b50 0x7f9e0c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 0x7f9e0c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47496/0 (socket says 172.21.15.44:47496) 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 -- 172.21.15.44:0/3233902677 learned_addr learned my addr 172.21.15.44:0/3233902677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e0a800640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e0c1064f0 0x7f9e0c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 -- 172.21.15.44:0/3233902677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e0c1064f0 msgr2=0x7f9e0c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e0c1064f0 0x7f9e0c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 -- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e0c105b50 msgr2=0x7f9e0c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e0c105b50 0x7f9e0c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.355+0000 7f9e09e00640 1 -- 172.21.15.44:0/3233902677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c19afc0 con 0x7f9e0c104270 2024-04-20T11:32:58.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.356+0000 7f9e09e00640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 0x7f9e0c1a0640 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00e9f0 tx=0x7f9dfc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.356+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00cd80 con 0x7f9e0c104270 2024-04-20T11:32:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.356+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c19b2b0 con 0x7f9e0c104270 2024-04-20T11:32:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.356+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9dfc010930 con 0x7f9e0c104270 2024-04-20T11:32:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.356+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc018b00 con 0x7f9e0c104270 2024-04-20T11:32:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.356+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c1a75c0 con 0x7f9e0c104270 2024-04-20T11:32:58.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.357+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e0c105660 con 0x7f9e0c104270 2024-04-20T11:32:58.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.360+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9dfc018ca0 con 0x7f9e0c104270 2024-04-20T11:32:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.361+0000 7f9e02a00640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9dec0a3d20 0x7f9dec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:32:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.361+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f9dfc0d0ed0 con 0x7f9e0c104270 2024-04-20T11:32:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.361+0000 7f9e09400640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9dec0a3d20 0x7f9dec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:32:58.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.362+0000 7f9e09400640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9dec0a3d20 0x7f9dec0a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c19bf50 tx=0x7f9df4009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:32:58.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.364+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9dfc09cf00 con 0x7f9e0c104270 2024-04-20T11:32:58.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.629+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9e0c1a7790 con 0x7f9e0c104270 2024-04-20T11:32:58.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.641+0000 7f9e02a00640 1 -- 172.21.15.44:0/3233902677 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746058 (secure 0 0 0) 0x7f9dfc0a1200 con 0x7f9e0c104270 2024-04-20T11:32:58.653 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:32:58.653 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 27, 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "active_gid": 15174, 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "active_name": "x", 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addrs": { 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4029191467 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4029191467 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addr": "172.21.15.44:6801/4029191467", 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "active_change": "2024-04-20T11:22:36.090629+0000", 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "active_mgr_features": 4540701547738038271, 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "standbys": [ 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.655 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 25285, 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "z", 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540701547738038271, 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.656 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.659 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.660 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.662 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.663 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.665 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.667 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.669 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.671 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.672 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.673 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.675 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.676 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.677 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.678 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.679 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.680 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.681 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.682 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.683 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "read", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap", 2024-04-20T11:32:58.686 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap-read" 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.687 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.688 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:58.689 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-04-20T11:32:58.691 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-04-20T11:32:58.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-04-20T11:32:58.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_log_destination": { 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm_log_destination", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "file", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "file,syslog", 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "syslog" 2024-04-20T11:32:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T11:32:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T11:32:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T11:32:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T11:32:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_samba": { 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_samba", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Samba/SMB container image", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "grafana_dashboards_path": { 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "grafana_dashboards_path", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.726 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.727 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.731 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.732 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.733 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.734 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.735 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.736 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.737 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.738 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.740 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-04-20T11:32:58.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-04-20T11:32:58.742 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-04-20T11:32:58.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-04-20T11:32:58.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.746 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.747 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.748 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.749 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.750 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.751 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.752 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.753 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.754 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.758 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.759 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.760 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-04-20T11:32:58.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-04-20T11:32:58.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.764 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.765 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.766 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.767 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.770 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.773 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.778 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.779 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T11:32:58.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T11:32:58.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.782 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.784 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.785 INFO:teuthology.orchestra.run.smithi044.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "[]", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.786 INFO:teuthology.orchestra.run.smithi044.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "{}", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T11:32:58.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T11:32:58.788 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T11:32:58.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T11:32:58.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T11:32:58.791 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.792 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.796 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.797 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.798 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.800 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.801 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.803 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.805 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.807 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.808 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.809 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.810 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.811 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.813 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.814 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.816 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.817 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.818 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-04-20T11:32:58.819 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.822 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.823 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.824 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-04-20T11:32:58.825 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-04-20T11:32:58.826 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.827 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-04-20T11:32:58.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.830 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.831 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.832 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.833 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.834 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.839 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.840 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.843 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.844 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.845 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.846 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.847 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-04-20T11:32:58.848 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.849 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.851 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.852 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.853 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.854 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.855 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.856 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.857 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.858 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.859 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.860 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.861 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-04-20T11:32:58.862 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.865 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.866 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.867 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.869 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.870 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.871 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.872 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.873 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.874 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.875 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.877 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.879 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.880 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.881 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.883 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.885 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.887 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.889 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.890 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.891 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.893 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.894 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.896 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.897 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.898 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.902 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.906 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.907 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.908 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.909 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.910 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.911 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.912 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.913 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.914 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.915 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.916 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.917 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.918 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-04-20T11:32:58.919 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.920 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.921 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.922 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.923 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.924 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.926 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.927 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.929 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.930 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.931 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.932 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.933 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.935 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.938 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.939 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.940 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.942 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-04-20T11:32:58.945 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.946 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.949 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.950 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.953 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.954 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-04-20T11:32:58.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.957 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.958 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.959 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.960 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.961 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:58.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.964 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.968 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.969 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.971 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.975 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.977 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.979 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.980 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.981 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.982 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.984 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.985 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.987 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.988 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.989 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.990 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.992 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.993 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:58.994 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.995 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.996 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:58.997 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:58.998 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:58.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.002 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.004 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.006 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.007 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.008 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.010 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.011 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-04-20T11:32:59.013 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.014 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.015 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.016 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.017 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.020 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.022 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.023 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.024 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.025 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.026 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.028 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.029 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.030 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.033 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.034 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.035 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.036 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.039 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.040 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.041 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.042 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.043 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.044 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.045 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.046 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.048 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.049 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.050 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.053 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.054 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.056 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.057 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.058 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.059 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.060 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.061 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.062 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.063 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.064 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.065 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.066 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.067 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.068 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.069 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.070 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "periodic_async_work": { 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "periodic_async_work", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Periodically check for async work", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.072 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.073 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.074 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.075 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.076 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.078 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.079 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.080 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-04-20T11:32:59.082 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-04-20T11:32:59.083 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-04-20T11:32:59.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.085 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 25286, 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "y", 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540701547738038271, 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.087 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.088 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.089 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.090 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.091 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-04-20T11:32:59.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-04-20T11:32:59.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-04-20T11:32:59.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-04-20T11:32:59.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-04-20T11:32:59.096 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-04-20T11:32:59.097 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.098 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.099 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.100 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.101 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-04-20T11:32:59.102 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-04-20T11:32:59.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.104 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.105 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-04-20T11:32:59.106 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.107 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-04-20T11:32:59.108 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.109 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.110 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.111 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.112 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.113 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.114 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-04-20T11:32:59.115 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "read", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap-read" 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.116 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.117 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.118 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.120 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.122 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.123 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.124 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.125 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.126 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_log_destination": { 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm_log_destination", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.127 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "file", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "file,syslog", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "syslog" 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.129 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.131 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.133 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.134 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.135 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.136 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.137 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.138 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-04-20T11:32:59.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.140 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.141 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-04-20T11:32:59.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.144 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_samba": { 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_samba", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Samba/SMB container image", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-04-20T11:32:59.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-04-20T11:32:59.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-04-20T11:32:59.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T11:32:59.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-04-20T11:32:59.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-04-20T11:32:59.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-04-20T11:32:59.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-04-20T11:32:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "grafana_dashboards_path": { 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "grafana_dashboards_path", 2024-04-20T11:32:59.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.156 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.157 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.161 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.162 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-04-20T11:32:59.165 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.166 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.167 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.168 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.170 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.171 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.172 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.173 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.174 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.175 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.177 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.178 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.179 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.181 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.182 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-04-20T11:32:59.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.185 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.186 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.187 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.188 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.189 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-04-20T11:32:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-04-20T11:32:59.193 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.195 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.199 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.201 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.202 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.204 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.209 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.210 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.211 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.212 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.215 INFO:teuthology.orchestra.run.smithi044.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "[]", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.216 INFO:teuthology.orchestra.run.smithi044.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "{}", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.217 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.218 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.219 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.220 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.221 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.222 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.223 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.224 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.225 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.226 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.227 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.228 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.229 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.230 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T11:32:59.231 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.232 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.233 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.235 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.237 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.238 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.239 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.240 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.242 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-04-20T11:32:59.244 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-04-20T11:32:59.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.244 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.245 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-04-20T11:32:59.246 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-04-20T11:32:59.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.246 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.247 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.248 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.250 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-04-20T11:32:59.250 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-04-20T11:32:59.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.250 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.251 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.251 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.251 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.251 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.252 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.254 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.255 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.256 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.258 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.259 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.260 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.261 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.262 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-04-20T11:32:59.263 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.264 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-04-20T11:32:59.266 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-04-20T11:32:59.268 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.269 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.270 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.271 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.272 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.273 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.274 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-04-20T11:32:59.275 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.276 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.277 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.282 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.283 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.285 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.286 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-04-20T11:32:59.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-04-20T11:32:59.288 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.289 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-04-20T11:32:59.290 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.292 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.293 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.294 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.295 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.296 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.299 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-04-20T11:32:59.300 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.301 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-04-20T11:32:59.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.303 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.305 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.307 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.310 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.311 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.313 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.314 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.315 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.316 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.317 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.318 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.319 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.320 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.322 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.323 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.324 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.326 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.327 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.330 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-04-20T11:32:59.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.333 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.335 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.336 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.337 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.338 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.339 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.340 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.341 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.343 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.345 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.346 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.347 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.350 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.351 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.352 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.353 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.355 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.356 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.356 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.356 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.356 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-04-20T11:32:59.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.358 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.358 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.358 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.359 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.359 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T11:32:59.359 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.359 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.359 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.363 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.364 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-04-20T11:32:59.366 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.368 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.369 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.370 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.371 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.372 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.373 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.374 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.375 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.376 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.377 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.379 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.380 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.381 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.382 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.383 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.384 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.385 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.385 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.385 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-04-20T11:32:59.385 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-04-20T11:32:59.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.386 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.386 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.386 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-04-20T11:32:59.386 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-04-20T11:32:59.386 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.386 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.387 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-04-20T11:32:59.387 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T11:32:59.387 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.387 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-04-20T11:32:59.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.390 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.391 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.392 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.393 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.394 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.394 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.394 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.394 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.394 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.394 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.396 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.396 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.396 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.397 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.397 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.397 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.397 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.397 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.397 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.398 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.398 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.398 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.398 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.399 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.400 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.401 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.403 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.403 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.403 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.403 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.403 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.404 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.405 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-04-20T11:32:59.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-04-20T11:32:59.407 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.407 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.407 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.407 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.408 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.409 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.409 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.409 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.409 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.410 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.410 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.410 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.410 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.410 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.410 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.411 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.411 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.411 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.411 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.411 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.412 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.413 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.414 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.415 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.415 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.415 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.415 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.415 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.416 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.416 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.416 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.416 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.416 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.416 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.417 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-04-20T11:32:59.417 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-04-20T11:32:59.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.417 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.420 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-04-20T11:32:59.420 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-04-20T11:32:59.420 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.420 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.422 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:59.422 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:59.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.422 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.423 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-04-20T11:32:59.423 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.423 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.423 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.423 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T11:32:59.423 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-04-20T11:32:59.424 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.425 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.426 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.427 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-04-20T11:32:59.428 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.430 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.431 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.432 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.433 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.433 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.433 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.433 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.435 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.435 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.435 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.435 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.435 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.435 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.436 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.436 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.436 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.436 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.437 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.439 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.440 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.441 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.441 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.441 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.441 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.443 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.444 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.445 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.446 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.446 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.446 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.446 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.447 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.448 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.448 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.448 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.448 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.448 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.449 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.449 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.449 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.449 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.450 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.450 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.450 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.450 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.450 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.451 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.452 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.453 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.454 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:59.455 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.456 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.457 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.458 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.459 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.460 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.462 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.463 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.464 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.466 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.467 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.468 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.468 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.468 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.469 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.470 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.471 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.472 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.472 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.472 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.472 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.472 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.473 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.473 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-04-20T11:32:59.473 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-04-20T11:32:59.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.473 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.474 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-04-20T11:32:59.474 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.474 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.474 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.474 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.475 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.476 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.478 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.480 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.481 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.482 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.483 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.484 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.486 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.487 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.488 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-04-20T11:32:59.489 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.490 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.491 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.492 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-04-20T11:32:59.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.494 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.495 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.496 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.497 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.498 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.500 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.501 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.502 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.504 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.505 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.506 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.507 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.508 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.509 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.510 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.511 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.512 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.514 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.515 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.516 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.519 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.520 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.521 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.523 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.524 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.526 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.527 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.528 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.529 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.530 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.531 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.532 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.533 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.535 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.536 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.537 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.538 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.539 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.540 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.541 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.542 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.543 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.544 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.546 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.547 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.549 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.550 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T11:32:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "periodic_async_work": { 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "periodic_async_work", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Periodically check for async work", 2024-04-20T11:32:59.552 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T11:32:59.553 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T11:32:59.554 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.555 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.556 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.557 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.558 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.559 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.560 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.561 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.562 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-04-20T11:32:59.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-04-20T11:32:59.564 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-04-20T11:32:59.565 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard", 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "restful", 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_schedule" 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-04-20T11:32:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.568 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.569 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.570 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.571 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.572 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.573 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.574 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.575 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.576 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.580 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-04-20T11:32:59.581 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.582 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.583 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.584 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.585 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.586 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.587 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.589 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.590 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.591 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.592 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.593 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-04-20T11:32:59.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-04-20T11:32:59.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "read", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap", 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap-read" 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.596 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.597 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.598 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.599 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.600 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-04-20T11:32:59.601 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-04-20T11:32:59.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-04-20T11:32:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_log_destination": { 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm_log_destination", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.607 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "file", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "file,syslog", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "syslog" 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-04-20T11:32:59.608 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-04-20T11:32:59.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-04-20T11:32:59.610 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-04-20T11:32:59.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-04-20T11:32:59.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-04-20T11:32:59.613 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-04-20T11:32:59.614 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-04-20T11:32:59.615 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-04-20T11:32:59.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.617 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.618 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.619 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.622 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-04-20T11:32:59.623 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.624 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_samba": { 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_samba", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.625 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Samba/SMB container image", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-04-20T11:32:59.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-04-20T11:32:59.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-04-20T11:32:59.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.630 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.631 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.632 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.633 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "grafana_dashboards_path": { 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "grafana_dashboards_path", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.635 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.636 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.641 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.642 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.643 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T11:32:59.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T11:32:59.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.646 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.647 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-04-20T11:32:59.650 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-04-20T11:32:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-04-20T11:32:59.652 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.653 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.654 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.656 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.658 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.659 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.661 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.662 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.663 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.664 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.665 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.666 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.667 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.668 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.669 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-04-20T11:32:59.671 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T11:32:59.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T11:32:59.678 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T11:32:59.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T11:32:59.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T11:32:59.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T11:32:59.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T11:32:59.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.684 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.685 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.686 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.687 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.688 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.689 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.691 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.692 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "[]", 2024-04-20T11:32:59.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "{}", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.695 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.696 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.697 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.698 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.699 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.700 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.701 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.702 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.704 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.706 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.708 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.711 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.712 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.715 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.716 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.717 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.718 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.720 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.722 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.723 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-04-20T11:32:59.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.728 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.729 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.730 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-04-20T11:32:59.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-04-20T11:32:59.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.736 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.737 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.738 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.742 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.743 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.744 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T11:32:59.745 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T11:32:59.746 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T11:32:59.747 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T11:32:59.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.751 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.752 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.753 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.754 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-04-20T11:32:59.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-04-20T11:32:59.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.757 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.758 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-04-20T11:32:59.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.760 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.761 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.762 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.763 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.765 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-04-20T11:32:59.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-04-20T11:32:59.767 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.768 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.769 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.771 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.772 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.773 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.774 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.775 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.777 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.781 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.782 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.785 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.786 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.787 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.788 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.789 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.790 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T11:32:59.791 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.792 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.793 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.795 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.796 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.800 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.802 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.804 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.806 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.807 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.808 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.809 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.810 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.812 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.813 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.815 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.816 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.818 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.819 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.820 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.823 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.824 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.825 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.826 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.827 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.828 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.829 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.831 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.833 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.834 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.837 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.839 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.840 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.841 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-04-20T11:32:59.843 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-04-20T11:32:59.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.846 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.847 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.848 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.849 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.850 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.851 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-04-20T11:32:59.852 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.853 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T11:32:59.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.855 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.856 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.857 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.858 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.859 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.860 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.862 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.863 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.865 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.867 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.868 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.871 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.872 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.873 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.874 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.875 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-04-20T11:32:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-04-20T11:32:59.877 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.880 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.881 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.883 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.884 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.888 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-04-20T11:32:59.890 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.891 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.893 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.898 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.900 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.901 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.903 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-04-20T11:32:59.904 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-04-20T11:32:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-04-20T11:32:59.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-04-20T11:32:59.907 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-04-20T11:32:59.908 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-04-20T11:32:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.910 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.911 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.912 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-04-20T11:32:59.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.915 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.916 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.917 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.918 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.925 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-04-20T11:32:59.927 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.928 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.929 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.931 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.932 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.933 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.934 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.935 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.936 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.938 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.939 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.940 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.945 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.946 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.947 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-04-20T11:32:59.950 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-04-20T11:32:59.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-04-20T11:32:59.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.956 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.957 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-04-20T11:32:59.960 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.961 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.962 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.964 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.965 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.967 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.970 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-04-20T11:32:59.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "periodic_async_work": { 2024-04-20T11:32:59.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "periodic_async_work", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Periodically check for async work", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.974 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-04-20T11:32:59.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.977 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.978 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.979 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.980 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-04-20T11:32:59.981 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-04-20T11:32:59.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-04-20T11:32:59.983 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.984 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.985 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.986 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-04-20T11:32:59.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "services": { 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard": "https://172.21.15.44:8443/" 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "always_on_modules": { 2024-04-20T11:32:59.988 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus": [ 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific": [ 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-04-20T11:32:59.989 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "quincy": [ 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-04-20T11:32:59.990 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "reef": [ 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-04-20T11:32:59.991 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "squid": [ 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-04-20T11:32:59.992 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "last_failure_osd_epoch": 88, 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "active_clients": [ 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2390494875 2024-04-20T11:32:59.993 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "libcephsqlite", 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 47854415 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-04-20T11:32:59.994 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 981081444 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-04-20T11:32:59.995 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3243207278 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 54854322 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:32:59.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2472841196 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T11:32:59.997 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:33:00.001 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:58 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2779877704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:33:00.001 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:58 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/452665225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:33:00.001 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:59 smithi165 ceph-mon[21975]: pgmap v322: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T11:33:00.001 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:32:59 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3233902677' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T11:33:00.001 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:58 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2779877704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:33:00.001 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:58 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/452665225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:33:00.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:59 smithi078 ceph-mon[21464]: pgmap v322: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T11:33:00.002 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:32:59 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3233902677' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T11:33:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9dec0a3d20 msgr2=0x7f9dec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9dec0a3d20 0x7f9dec0a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c19bf50 tx=0x7f9df4009500 comp rx=0 tx=0).stop 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 msgr2=0x7f9e0c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 0x7f9e0c1a0640 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00e9f0 tx=0x7f9dfc00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 shutdown_connections 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e0c1064f0 0x7f9e0c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f9dec0a3d20 0x7f9dec0a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e0c105b50 0x7f9e0c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 --2- 172.21.15.44:0/3233902677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0c104270 0x7f9e0c1a0640 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.655+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 >> 172.21.15.44:0/3233902677 conn(0x7f9e0c0ff9d0 msgr2=0x7f9e0c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.656+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 shutdown_connections 2024-04-20T11:33:00.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:32:58.656+0000 7f9e0b200640 1 -- 172.21.15.44:0/3233902677 wait complete. 2024-04-20T11:33:00.009 DEBUG:tasks.check_counter:Getting stats from a 2024-04-20T11:33:00.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-20T11:33:00.010 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2779877704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:33:00.010 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:58 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/452665225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:33:00.010 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:59 smithi044 ceph-mon[16855]: pgmap v322: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T11:33:00.010 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:32:59 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3233902677' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T11:33:00.470 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:33:00.794 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:00 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:33:00.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:00 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:33:00.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:00 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T11:33:01.797 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:01 smithi078 ceph-mon[21464]: pgmap v323: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:33:01.797 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:01 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.797 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:01 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.797 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:01 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.797 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:01 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:01 smithi165 ceph-mon[21975]: pgmap v323: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:33:01.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:01 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:01 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:01 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.836 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:01 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:01 smithi044 ceph-mon[16855]: pgmap v323: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-20T11:33:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:01 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:01 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:01 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:01.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:01 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.899 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:33:02.899 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:02.899 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4, 2024-04-20T11:33:02.899 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 4, 2024-04-20T11:33:02.899 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3024, 2024-04-20T11:33:02.899 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 448, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.004744060, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000055617, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001163338, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001424646, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000474882 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:02.900 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 162, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 320, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 184752, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.032497637, 2024-04-20T11:33:02.901 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.024084238, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.003495446, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 159, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.027358790, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000172067 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.902 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 184752 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 224, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 212, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1728976, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 130464, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.095337539, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.033202294, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.071239893, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.009718237, 2024-04-20T11:33:02.903 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 211, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.020223529, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000095846 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1728976, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-20T11:33:02.904 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:02.905 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:02.906 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 151039, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 32, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:02.907 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-04-20T11:33:02.908 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 226, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1697394, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 226, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:02.909 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 226, 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1697394, 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:02.910 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:33:02.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:02 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:02 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:02 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:02 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:02 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:02 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:02 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:02 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:02 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:02 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:02 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:02 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:02 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:02 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:02 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:02 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:02 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:02.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:02 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:03.783 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-20T11:33:03.784 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-20T11:33:03.784 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-20T11:33:03.784 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-20T11:33:03.784 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-20T11:33:03.784 DEBUG:tasks.check_counter:Getting stats from d 2024-04-20T11:33:03.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-20T11:33:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: pgmap v324: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-04-20T11:33:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:03 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: pgmap v324: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-04-20T11:33:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:03.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:03 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: pgmap v324: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:03 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:04.653 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:33:04.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:04 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-20T11:33:04.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:04 smithi044 ceph-mon[16855]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-20T11:33:04.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:04 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-20T11:33:04.807 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:04 smithi044 ceph-mon[16855]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T11:33:04.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:04 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-20T11:33:04.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:04 smithi165 ceph-mon[21975]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-20T11:33:04.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:04 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-20T11:33:04.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:04 smithi165 ceph-mon[21975]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T11:33:05.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:04 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-20T11:33:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:04 smithi078 ceph-mon[21464]: Unable to set osd_memory_target on smithi078 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-20T11:33:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:04 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-20T11:33:05.188 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:04 smithi078 ceph-mon[21464]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T11:33:05.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:05 smithi044 ceph-mon[16855]: pgmap v325: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:33:05.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:05 smithi165 ceph-mon[21975]: pgmap v325: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:33:06.187 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:05 smithi078 ceph-mon[21464]: pgmap v325: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:33:07.020 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:33:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 10278, 2024-04-20T11:33:07.020 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 21655, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 4026153, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 27093556, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 47, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 9, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 3.583335893, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 2.238846649, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 1.614259866, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.122460374, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 21648, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2.739027664, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000126525 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.021 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 19124, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.010400363, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000543 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 2987360, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 10242880 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6826, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 8763, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 6009277, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 4231801, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 12, 2024-04-20T11:33:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 1.508855498, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.724114451, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.924268558, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.051222154, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 8766, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.824225557, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000094025 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 12789, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.006305139, 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000493 2024-04-20T11:33:07.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 6640, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 465888 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3830, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 4079, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3044219, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 12952407, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 15, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 7, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.785735923, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.344669467, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.391292398, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.098869138, 2024-04-20T11:33:07.024 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4078, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.408137477, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000100082 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3230, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001407278, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000435 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1966320, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 149952 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-20T11:33:07.025 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1791, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 10.658254196, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005951007 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6023, 2024-04-20T11:33:07.026 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 3.227542036, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000535869 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 4190, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 4154, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4154, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 21.046318751, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005066518 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 36, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:07.027 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 38, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 3, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 2129, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 1, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 2128, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 2128, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 499, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 1, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 1, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 4, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 5257, 2024-04-20T11:33:07.028 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 4457, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 145, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 15, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 455, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 189, 2024-04-20T11:33:07.029 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 4153, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 1159, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 1, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 4, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 6, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 1159, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 43, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 43, 2024-04-20T11:33:07.030 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 1116, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 66, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 625, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 4325, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 3258, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 2616, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 49, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 4123 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:07.031 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 205, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 5, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 58, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 3534, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 3476, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-20T11:33:07.032 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 15, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 3, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 11734, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 11734, 2024-04-20T11:33:07.033 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 15, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 15, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 32816235, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 694, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 6.677156436, 2024-04-20T11:33:07.034 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.009621262 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 1055, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 4673, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 3618, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 93, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 397, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 304, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 2129, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 8536, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 6407, 2024-04-20T11:33:07.035 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 1, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 3611, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 3610, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 79012, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 223608 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 6459, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 674, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 4190, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 28, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 1846, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 71, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.153570234, 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002162961 2024-04-20T11:33:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 32, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.056059984, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001751874 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 234, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.136652640, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000583985 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-20T11:33:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 282, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.450019737, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001595814 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-20T11:33:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 300, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 9.866636920, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.032888789 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.158435544, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.158435544 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3234, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 10.224943692, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.003161701 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 4, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 3452, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 1726, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 611, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 792098, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 1265, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 163, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 4243246, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 66191, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-20T11:33:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 5818, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 29905289, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 27, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 5818, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5818, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 14.817769987, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002546883 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5818, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 5964 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 5818, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 1289, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 4529, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 10, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 130, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 824, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 79, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 3, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 3510, 2024-04-20T11:33:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1159, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 20, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-20T11:33:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-20T11:33:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 96, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 11, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 12, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 12, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 179, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 50 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-04-20T11:33:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 14, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 6, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 3492, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 3476, 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 20930, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 10840866, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 20930, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 20930, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 10840866, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 5818, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 30262309, 2024-04-20T11:33:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 5818, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 30262309, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 5818, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 5818, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 5818, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 5818, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 3476, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 357020, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 3476, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 130, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 357020, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x5579f1356500": { 2024-04-20T11:33:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 11734, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 28621931, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 11734, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 694, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 28621931, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:07.065 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:33:07.905 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:07 smithi044 ceph-mon[16855]: pgmap v326: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:33:07.906 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:07 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:07.906 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:07 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:07.963 INFO:tasks.check_counter:Daemon mds.d mds.exported=15 2024-04-20T11:33:07.963 INFO:tasks.check_counter:Daemon mds.d mds.imported=189 2024-04-20T11:33:07.963 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=6 2024-04-20T11:33:07.963 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:07.963 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:07.963 DEBUG:tasks.check_counter:Getting stats from g 2024-04-20T11:33:07.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-20T11:33:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:07 smithi165 ceph-mon[21975]: pgmap v326: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:33:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:07 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:07.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:07 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:07.990 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:07 smithi078 ceph-mon[21464]: pgmap v326: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-20T11:33:07.990 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:07 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:07.990 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:07 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:08.428 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:33:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T11:33:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: Adjusting osd_memory_target on smithi044 to 177.5M 2024-04-20T11:33:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: Unable to set osd_memory_target on smithi044 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T11:33:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:33:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:33:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:09.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:09 smithi044 ceph-mon[16855]: pgmap v328: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: Adjusting osd_memory_target on smithi044 to 177.5M 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: Unable to set osd_memory_target on smithi044 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:09.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:09 smithi078 ceph-mon[21464]: pgmap v328: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:33:09.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T11:33:09.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: Adjusting osd_memory_target on smithi044 to 177.5M 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: Unable to set osd_memory_target on smithi044 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' 2024-04-20T11:33:09.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:09 smithi165 ceph-mon[21975]: pgmap v328: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 863, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 918, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 306205, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 2529456, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 20, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.243111549, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.121573551, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.110227416, 2024-04-20T11:33:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.014421674, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 911, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.150972414, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000165721 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1112, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000450492, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000405 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 315616 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2176, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2744, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1090273, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1781543, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 43, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.652379222, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.347021418, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.357615800, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.012477141, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2746, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.578348529, 2024-04-20T11:33:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000210614 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4212, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002259711, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000536 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 253024, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 862576 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 871, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 743, 2024-04-20T11:33:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 2238052, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 403065, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.216502576, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.076362941, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.163554093, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.017096878, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 743, 2024-04-20T11:33:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.097603952, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000131364 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 711, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000518044, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000728 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1990400, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 146592 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-20T11:33:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 362, 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.581941977, 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001607574 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 405, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.047547043, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000117400 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 409, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 382, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 382, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 1.197513006, 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.003134850 2024-04-20T11:33:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 27, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 11, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 79, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 2, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 2, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 75, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 75, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 318, 2024-04-20T11:33:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 4, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 820, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 521, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 26, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 17, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 15, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 238, 2024-04-20T11:33:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 16, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 168, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 41, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 4, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 6, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 41, 2024-04-20T11:33:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 1, 2024-04-20T11:33:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 1, 2024-04-20T11:33:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 40, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 2, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 36, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 267, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 169, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 108, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 15, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 381 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 17, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 10, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 4, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 200, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 196, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-20T11:33:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 15, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-20T11:33:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 755, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 755, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-20T11:33:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 6026888, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 121, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.288790449, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002386697 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 56, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 416, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 360, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 25, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 110, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 85, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 79, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 611, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 532, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 216, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 216, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 57268, 2024-04-20T11:33:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 223608 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 687, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 23, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 409, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 8, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 272, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 36, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.123659161, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.003434976 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-20T11:33:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 122, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.071944674, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000589710 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-20T11:33:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-20T11:33:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 23, 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.028461818, 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001237470 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4, 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.159372799, 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.039843199 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.087382727, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.087382727 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-20T11:33:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 196, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.726691827, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.003707611 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 1, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 0, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 293, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 293, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 1834, 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:10.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:10.848 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 316124, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 155, 2024-04-20T11:33:10.850 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 440, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 5, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 7960, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 166, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 248592, 2024-04-20T11:33:10.851 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2545, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 504, 2024-04-20T11:33:10.852 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 1899129, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 33, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 504, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 504, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2.424962388, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.004811433 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 504, 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 576 2024-04-20T11:33:10.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 504, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 65, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 439, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 10, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 24, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 145, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 26, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-20T11:33:10.854 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 208, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 41, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 20, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-20T11:33:10.855 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-20T11:33:10.856 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 97, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 12, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 12, 2024-04-20T11:33:10.857 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 12, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 80, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 22 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-20T11:33:10.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 6, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 6, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 196, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 196, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.859 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 3906, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 3235215, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 3906, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3906, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 3235215, 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:10.860 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:10.861 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 504, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 1920061, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 504, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1920061, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.862 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 504, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 504, 2024-04-20T11:33:10.863 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 504, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 504, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 196, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 20932, 2024-04-20T11:33:10.864 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 196, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 24, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 20932, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x5561d0a70500": { 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:10.865 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 755, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1832584, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 755, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 121, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1832584, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:10.866 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.867 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:10.867 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:33:11.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:11 smithi044 ceph-mon[16855]: pgmap v329: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:33:11.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:11 smithi165 ceph-mon[21975]: pgmap v329: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:33:11.860 INFO:tasks.check_counter:Daemon mds.g mds.exported=15 2024-04-20T11:33:11.860 INFO:tasks.check_counter:Daemon mds.g mds.imported=16 2024-04-20T11:33:11.860 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-20T11:33:11.860 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:11.860 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:11.860 DEBUG:tasks.check_counter:Getting stats from j 2024-04-20T11:33:11.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-20T11:33:11.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:11 smithi078 ceph-mon[21464]: pgmap v329: 97 pgs: 97 active+clean; 139 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-20T11:33:12.339 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T11:33:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:13 smithi165 ceph-mon[21975]: pgmap v330: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:13.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:13 smithi165 ceph-mon[21975]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T11:33:13.777 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:13 smithi044 ceph-mon[16855]: pgmap v330: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:13.777 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:13 smithi044 ceph-mon[16855]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T11:33:13.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:13 smithi078 ceph-mon[21464]: pgmap v330: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:13.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:13 smithi078 ceph-mon[21464]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1124, 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1126, 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 12936521, 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 357433, 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:14.695 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.277615656, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.087304049, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.100178482, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.062550938, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1124, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.100053674, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000089015 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.696 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1328, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1285, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 20654697, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 472174, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.342149224, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.080357241, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.211295062, 2024-04-20T11:33:14.697 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.068125034, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1285, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.120469240, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000093750 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1954784, 2024-04-20T11:33:14.698 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 140432 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 223, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 391, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 8444132, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 527094, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.098988246, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.039490706, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.035079656, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.014297210, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 388, 2024-04-20T11:33:14.699 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.048260549, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000124382 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 456816 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-20T11:33:14.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3854, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.204522671, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000053067 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 6971, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 6971, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 6971, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 1, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 1, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-04-20T11:33:14.703 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-04-20T11:33:14.704 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-04-20T11:33:14.705 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-04-20T11:33:14.706 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:14.707 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 16305, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-20T11:33:14.708 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 21, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 3, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 43595484, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 43592923, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.709 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 16305 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 1730, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 5396, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 3666, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 168, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 278, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 110, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 6971, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 7529, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 558, 2024-04-20T11:33:14.710 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 65548, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 215416 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.711 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.713 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:14.714 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-20T11:33:14.716 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.717 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.718 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.719 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:14.720 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.721 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.722 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 4, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 3, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 1836, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:14.723 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:14.724 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:14.725 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 283237, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 1925, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 7960, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 166, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-20T11:33:14.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 9278472, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 117725, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 2388, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 2388, 2024-04-20T11:33:14.727 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2388, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2.306964245, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000966065 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2388, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2389 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 2388, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 2388, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-20T11:33:14.728 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 1527, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 859, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-04-20T11:33:14.729 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-20T11:33:14.730 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 98, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 13, 2024-04-20T11:33:14.731 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 8, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 8, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-20T11:33:14.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.733 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2671, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 41777678, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2671, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2671, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 41777678, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.734 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2388, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 39408300, 2024-04-20T11:33:14.735 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2388, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 39408300, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:14.736 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2388, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 2388, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2388, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2388, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-20T11:33:14.737 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T11:33:14.738 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x560a7adc2780": { 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-20T11:33:14.739 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-20T11:33:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-20T11:33:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-20T11:33:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T11:33:14.741 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T11:33:15.563 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:15 smithi044 ceph-mon[16855]: pgmap v332: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:33:15.713 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-04-20T11:33:15.714 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-04-20T11:33:15.714 INFO:tasks.check_counter:Daemon mds.j mds.root_rsnaps=0 2024-04-20T11:33:15.714 INFO:tasks.check_counter:Daemon mds.j mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:15.714 INFO:tasks.check_counter:Daemon mds.j mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:15.714 DEBUG:tasks.check_counter:Getting stats from b 2024-04-20T11:33:15.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-20T11:33:15.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:15 smithi165 ceph-mon[21975]: pgmap v332: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:33:15.741 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:15 smithi078 ceph-mon[21464]: pgmap v332: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-20T11:33:15.965 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:33:17.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:17 smithi078 ceph-mon[21464]: pgmap v333: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 122 B/s wr, 6 op/s 2024-04-20T11:33:17.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:17 smithi165 ceph-mon[21975]: pgmap v333: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 122 B/s wr, 6 op/s 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 11377, 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 9808, 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 4286222, 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 32657140, 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 12, 2024-04-20T11:33:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.731435715, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.283609615, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.360090428, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.127594903, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 9807, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.149760277, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000117238 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 13125, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001984680, 2024-04-20T11:33:17.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000151 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 45584, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 2880 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 33729, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 64983, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 13657945, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 53037738, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 17, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 2.577627151, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 1.220313683, 2024-04-20T11:33:17.871 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 2.258610053, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.188921730, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 64981, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.759084315, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000027070 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 68891, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.010460907, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000151 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 10018784, 2024-04-20T11:33:17.872 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 30431488 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4142, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4184, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3142027, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 24132445, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 161, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 9, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.443578043, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.167518312, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.188665007, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.089290497, 2024-04-20T11:33:17.873 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4182, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.518875941, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000124073 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2946, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000625967, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000212 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 2091936, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 518224 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.874 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3935, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 16.532142357, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004201306 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:17.875 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7656, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.077162556, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000140695 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 15146, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 15116, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 15116, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 36.327781237, 2024-04-20T11:33:17.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002403266 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 30, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 93, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 9, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 4916, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 1, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 96, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 4819, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 4914, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 6580, 2024-04-20T11:33:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 3, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 4, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 3, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 15982, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 15410, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 7, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:17.878 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 283, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 6167, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 29, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 774, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 3826, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 6, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 3826, 2024-04-20T11:33:17.879 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 497, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 19, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 478, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 3329, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 36, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 812, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 14141, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 7464, 2024-04-20T11:33:17.880 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 6508, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 47, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 5593 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 314, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 162, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 4208, 2024-04-20T11:33:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 4046, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 19, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 284, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-04-20T11:33:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 9, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 28285, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 28285, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 37, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-04-20T11:33:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 37, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 4, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 74732788, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1495, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 85.600084600, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.057257581 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-04-20T11:33:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 2195, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 12356, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 10161, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 239, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 771, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 532, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 4916, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 15597, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 10681, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 4, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 9963, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 9959, 2024-04-20T11:33:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 97228, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 231800 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 16413, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 984, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 15146, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 298, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 2311, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 9321, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 17.501742757, 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001877667 2024-04-20T11:33:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 134, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.361524474, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002697943 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 375, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 4.467026991, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.011912071 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000357175, 2024-04-20T11:33:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000178587 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-04-20T11:33:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 11, 2024-04-20T11:33:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002478711, 2024-04-20T11:33:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000225337 2024-04-20T11:33:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 623, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.631715276, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001013989 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 11, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.412437007, 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.037494273 2024-04-20T11:33:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000266472, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000266472 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 400, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.640197303, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001600493 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002115655, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001057827 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 163, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 3.215819842, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.019728956 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.037552121, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.007510424 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 15, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.956422378, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.130428158 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 8, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.021701160, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002712645 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4045, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 7.076423915, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001749424 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 5, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 3, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 2, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-04-20T11:33:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 92, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 46, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 629, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 1712266, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 2649, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 1672, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 19, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 163, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 9162028, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 137127, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 10385, 2024-04-20T11:33:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 73619526, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 37, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 10385, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 10385, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 46.007343766, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004430172 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 10385, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 10617 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 10385, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 3934, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 6451, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 0, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 11, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 89, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 1565, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 60, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 9, 2024-04-20T11:33:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 4624, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 3845, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 22, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-04-20T11:33:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 98, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 14, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 334, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 0, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 58 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-04-20T11:33:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 40, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 6, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 4096, 2024-04-20T11:33:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 4046, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 49244, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 15395315, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 49244, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 49244, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 15395315, 2024-04-20T11:33:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 10385, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 74044996, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 10385, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 74044996, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 10385, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 10385, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 10385, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 10385, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 4046, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 425470, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 4046, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 70, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 425470, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x555bcbacc500": { 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 28285, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 70538484, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 28285, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1495, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 70538484, 2024-04-20T11:33:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:17.915 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-20T11:33:17.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:17 smithi044 ceph-mon[16855]: pgmap v333: 97 pgs: 97 active+clean; 139 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 122 B/s wr, 6 op/s 2024-04-20T11:33:18.708 INFO:tasks.check_counter:Daemon mds.b mds.exported=283 2024-04-20T11:33:18.708 INFO:tasks.check_counter:Daemon mds.b mds.imported=29 2024-04-20T11:33:18.709 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-20T11:33:18.709 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-20T11:33:18.709 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-20T11:33:18.709 DEBUG:tasks.check_counter:Getting stats from e 2024-04-20T11:33:18.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-20T11:33:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:33:19.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:19 smithi078 ceph-mon[21464]: pgmap v334: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:19.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:19 smithi165 ceph-mon[21975]: pgmap v334: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:19.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:19 smithi044 ceph-mon[16855]: pgmap v334: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:20.674 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-20T11:33:20.674 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:20.674 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 6574, 2024-04-20T11:33:20.674 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 7762, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 4476706, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 14260235, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 10, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.507052319, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.239135002, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.237289455, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.054206193, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7764, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.219350975, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000028252 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:20.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 10544, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001721218, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000163 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3220, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 3265, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 2780778, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 13091771, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 9, 2024-04-20T11:33:20.676 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.268543308, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.101748700, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.100766175, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.067935052, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3265, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.173155752, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000053033 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 691, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000113989, 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000164 2024-04-20T11:33:20.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1983936, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 154736 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 14331, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 25723, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 5595155, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 21707006, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 55, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 11, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.977815837, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.465926379, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.822614086, 2024-04-20T11:33:20.678 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.046239708, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 25716, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.557609943, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000021683 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 28046, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004383799, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000156 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 4905952, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 12979936 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.679 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2357, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 7.301176984, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003097656 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:20.680 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6245, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.098981222, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000175977 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 5848, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 5811, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5811, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 12.926146790, 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002224427 2024-04-20T11:33:20.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 37, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 46, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 4, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 1, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 3986, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 210, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 111, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 3665, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 3682, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 509, 2024-04-20T11:33:20.682 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 8, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 7090, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 6148, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 23, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 95, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 24, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 425, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 103, 2024-04-20T11:33:20.683 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 2111, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 1852, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 6, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 1852, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 144, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 144, 2024-04-20T11:33:20.684 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 1708, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 24, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 473, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 5744, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 6105, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 3840, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 53, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 4371 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:20.685 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 109, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 10, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 143, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 3533, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 3390, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-04-20T11:33:20.686 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 24, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 5, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 16305, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-04-20T11:33:20.687 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 16305, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 21, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 21, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 3, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 43595484, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-04-20T11:33:20.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 582, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 11.695406907, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.020095200 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 1744, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 5378, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 3634, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 183, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 262, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 79, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 3986, 2024-04-20T11:33:20.689 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 7533, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 3547, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 3557, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 3557, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 89416, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 223608 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 7546, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 375, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 5848, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 22, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 1492, 2024-04-20T11:33:20.690 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1661, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.582084172, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000952488 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 11, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012727093, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001157008 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.691 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 298, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.153224637, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000514176 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.693 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.695 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 293, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.491259056, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001676652 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-04-20T11:33:20.696 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.697 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 144, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.688732047, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.018671750 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.698 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 15, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.028315481, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.135221032 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3389, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 5.969804304, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001761523 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.701 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 4, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 3, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 2797, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 2797, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 2073, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.702 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:20.703 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:20.704 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 1519008, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 2142, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 792, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 9, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:20.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 163, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 7383229, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 123354, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.706 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 6143, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 40881452, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 33, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 6143, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6143, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 20.061713809, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003265784 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6143, 2024-04-20T11:33:20.707 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 6279 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 6143, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 1917, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 4226, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 0, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 10, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 65, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 647, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 42, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 4, 2024-04-20T11:33:20.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 3419, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1852, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 20, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:20.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-04-20T11:33:20.710 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 98, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 13, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 182, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 0, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 38 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2024-04-20T11:33:20.711 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 6, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 6, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 3390, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 3390, 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:20.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 24121, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 10111305, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 24121, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 24121, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 10111305, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.713 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:20.714 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 6143, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 41237154, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 6143, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 41237154, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:20.715 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 6143, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 6143, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 6143, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 6143, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 3390, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 355702, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 3390, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 65, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 355702, 2024-04-20T11:33:20.717 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x564790cc6500": { 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 16305, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 39401180, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 16305, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:20.718 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 582, 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 39401180, 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:20.719 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-20T11:33:21.560 INFO:tasks.check_counter:Daemon mds.e mds.exported=24 2024-04-20T11:33:21.560 INFO:tasks.check_counter:Daemon mds.e mds.imported=103 2024-04-20T11:33:21.560 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-20T11:33:21.560 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:21.560 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:21.560 DEBUG:tasks.check_counter:Getting stats from h 2024-04-20T11:33:21.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-20T11:33:21.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:21 smithi165 ceph-mon[21975]: pgmap v335: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:21.804 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:33:21.925 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:21 smithi078 ceph-mon[21464]: pgmap v335: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:21.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:21 smithi044 ceph-mon[16855]: pgmap v335: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-20T11:33:23.518 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:23 smithi078 ceph-mon[21464]: pgmap v336: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-20T11:33:23.523 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1698, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1698, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 33297885, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 539150, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.160968135, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.057294384, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.047384735, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.041959785, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1697, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.060379280, 2024-04-20T11:33:23.524 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000035580 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 129, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 295, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 21009611, 2024-04-20T11:33:23.525 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 498532, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 8, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.027776441, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.008292480, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.014153396, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.002724933, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 292, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.036138168, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000123760 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.526 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 458080 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1037, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 991, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 18939671, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 378001, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.120454060, 2024-04-20T11:33:23.527 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.031805319, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.064190239, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.030050077, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 990, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.033524763, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000033863 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1976768, 2024-04-20T11:33:23.528 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 138864 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4275, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.101565889, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000023758 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:23.529 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.530 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-04-20T11:33:23.531 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 9092, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 9092, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 9092, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 2, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-04-20T11:33:23.532 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 4, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 234, 2024-04-20T11:33:23.533 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 6, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:23.534 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:23.535 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 28285, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 37, 2024-04-20T11:33:23.537 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 4, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 74732788, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 74732676, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 28285 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-04-20T11:33:23.538 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 2322, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 12332, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 10010, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 235, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 740, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 505, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 9092, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 15333, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 6241, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 73812, 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215416 2024-04-20T11:33:23.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.540 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-04-20T11:33:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-04-20T11:33:23.542 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-04-20T11:33:23.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.544 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.546 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.546 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-04-20T11:33:23.546 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.550 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.551 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.552 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 5, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 3, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 4, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 3508, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-20T11:33:23.553 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:23.554 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:23.555 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 305696, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 2532, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-20T11:33:23.556 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 163, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 12242068, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 140672, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-20T11:33:23.557 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 2573, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 26, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 4, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 2573, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2573, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.369980495, 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000532444 2024-04-20T11:33:23.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2573, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2574 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 2573, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 2573, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 1627, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 943, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-04-20T11:33:23.559 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 2, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-04-20T11:33:23.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-04-20T11:33:23.561 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 98, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 13, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 8, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 8, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-04-20T11:33:23.562 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T11:33:23.563 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 2860, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 72972313, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 2860, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:23.564 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2860, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 72972313, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:23.565 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2573, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 70546951, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2573, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 70546951, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:23.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2573, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 2573, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2573, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 2573, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.568 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x561472e166e0": { 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:23.569 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:23.570 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-20T11:33:23.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:23 smithi165 ceph-mon[21975]: pgmap v336: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:33:23.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:23 smithi044 ceph-mon[16855]: pgmap v336: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-20T11:33:24.290 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-20T11:33:24.290 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-20T11:33:24.290 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-20T11:33:24.290 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:24.290 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:24.290 DEBUG:tasks.check_counter:Getting stats from k 2024-04-20T11:33:24.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-20T11:33:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T11:33:25.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:25 smithi078 ceph-mon[21464]: pgmap v337: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:33:25.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:25 smithi044 ceph-mon[16855]: pgmap v337: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:33:25.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:25 smithi165 ceph-mon[21975]: pgmap v337: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 9 op/s 2024-04-20T11:33:26.183 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 838, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 840, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 8669374, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 266485, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.100978134, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.030135814, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.029837908, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.030878577, 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:26.184 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 838, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.043891906, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000052376 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1776, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1735, 2024-04-20T11:33:26.185 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 22567663, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 616417, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.166310800, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.050969390, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.074983826, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.042611512, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1734, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.074938975, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000043217 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.186 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1952576, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 141904 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 9, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 177, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 4692, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 460197, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:26.187 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.011060098, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.007064670, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.002073532, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000047816, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 175, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.023514029, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000134365 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.188 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 459232 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3979, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.106569306, 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000026782 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.190 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 5604, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 5604, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 5604, 2024-04-20T11:33:26.191 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 1, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-04-20T11:33:26.192 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:26.193 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.194 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:26.195 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:26.196 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-04-20T11:33:26.196 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-04-20T11:33:26.196 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:26.196 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-04-20T11:33:26.198 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 11734, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 15, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 32816235, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 32812211, 2024-04-20T11:33:26.199 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 11734 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 1052, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 4678, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 3626, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 90, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 411, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 321, 2024-04-20T11:33:26.200 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 5604, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 8531, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 2927, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 61240, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215416 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-04-20T11:33:26.201 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.204 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.205 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.206 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.207 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.208 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.209 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.210 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.213 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 4, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 2, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 4, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 3508, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:26.214 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:26.215 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:26.216 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 255769, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 1164, 2024-04-20T11:33:26.217 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 163, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:26.218 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 6614782, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 72987, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 2333, 2024-04-20T11:33:26.219 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 2333, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2333, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.697975794, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000727807 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2333, 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2334 2024-04-20T11:33:26.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 2333, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 2333, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 1461, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 870, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-04-20T11:33:26.221 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-04-20T11:33:26.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-04-20T11:33:26.223 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 98, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 13, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 8, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 8, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.224 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-04-20T11:33:26.225 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 2619, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 30988581, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 2619, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:26.226 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2619, 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 30988581, 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:26.227 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2333, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 28626239, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2333, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 28626239, 2024-04-20T11:33:26.228 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:26.229 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2333, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 2333, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2333, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 2333, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:26.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55b663de46e0": { 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-20T11:33:26.232 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-20T11:33:26.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-20T11:33:26.233 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.233 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-20T11:33:26.233 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-20T11:33:27.069 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-20T11:33:27.069 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-20T11:33:27.070 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-20T11:33:27.070 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:27.070 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:27.070 DEBUG:tasks.check_counter:Getting stats from c 2024-04-20T11:33:27.070 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-20T11:33:27.325 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:33:27.710 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:27 smithi165 ceph-mon[21975]: pgmap v338: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:33:27.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:27 smithi078 ceph-mon[21464]: pgmap v338: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:33:27.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:27 smithi044 ceph-mon[16855]: pgmap v338: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-20T11:33:29.337 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1087, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1041, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2246288, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 394702, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 1, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.150943889, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.044468849, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.068522366, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.039524705, 2024-04-20T11:33:29.338 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1040, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.060989115, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000058643 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1977152, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 139984 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.339 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 172, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 876, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 462362, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.010196668, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.006637555, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.001668837, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000048730, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 169, 2024-04-20T11:33:29.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.034414519, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000203636 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 461664 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1390, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1390, 2024-04-20T11:33:29.341 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2190025, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 441203, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.101313069, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.034216480, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.025595542, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.033784299, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1389, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.053109888, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000038236 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:29.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-20T11:33:29.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3834, 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.109220392, 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000028487 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-20T11:33:29.345 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 259, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 259, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 259, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 1, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-04-20T11:33:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-20T11:33:29.348 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-20T11:33:29.349 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-20T11:33:29.350 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-20T11:33:29.351 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 755, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 6026888, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 6024327, 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-20T11:33:29.352 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 755 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 37, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 413, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 376, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 6, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 93, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 87, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 259, 2024-04-20T11:33:29.353 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 587, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 328, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 51888, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-20T11:33:29.354 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.355 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.356 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:29.357 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.358 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-20T11:33:29.359 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.359 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-20T11:33:29.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.361 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.363 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.364 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 2069, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:29.367 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:29.368 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:29.369 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 182660, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 84, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-20T11:33:29.370 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 163, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 286099, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 2708, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-20T11:33:29.371 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2189, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2189, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2189, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.078984100, 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000492911 2024-04-20T11:33:29.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2189, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2190 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2189, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2189, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1336, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 851, 2024-04-20T11:33:29.373 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:29.374 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-20T11:33:29.375 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 98, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 13, 2024-04-20T11:33:29.376 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 4, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 4, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-20T11:33:29.377 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:29.378 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 2477, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 4196747, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 2477, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2477, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 4196747, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2189, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1832589, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2189, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1832589, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.380 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2189, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2189, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2189, 2024-04-20T11:33:29.381 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2189, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:29.382 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x5609fb48c6e0": { 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:29.383 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:29.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:29.385 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T11:33:29.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:29 smithi078 ceph-mon[21464]: pgmap v339: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:33:29.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:29 smithi165 ceph-mon[21975]: pgmap v339: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:33:29.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:29 smithi044 ceph-mon[16855]: pgmap v339: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-20T11:33:30.120 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-20T11:33:30.120 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-20T11:33:30.120 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-20T11:33:30.121 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:30.121 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:30.121 DEBUG:tasks.check_counter:Getting stats from f 2024-04-20T11:33:30.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-20T11:33:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:33:31.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:31 smithi165 ceph-mon[21975]: pgmap v340: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:31.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:31 smithi078 ceph-mon[21464]: pgmap v340: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:31.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:31 smithi044 ceph-mon[16855]: pgmap v340: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:32.092 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T11:33:32.092 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 862, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 862, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2708979, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 273356, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.087513954, 2024-04-20T11:33:32.093 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.024917942, 2024-04-20T11:33:32.094 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.025912643, 2024-04-20T11:33:32.094 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.027982421, 2024-04-20T11:33:32.094 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:32.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 861, 2024-04-20T11:33:32.094 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.033062638, 2024-04-20T11:33:32.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000038400 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 734, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 902, 2024-04-20T11:33:32.095 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 180414, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 692829, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.088146438, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.026610653, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.021101119, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.032268249, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 899, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.055597103, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000061843 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:32.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 460048 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1090, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1044, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2247232, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 395978, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 3, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-20T11:33:32.097 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.151384001, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.044481128, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.071662699, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.039142541, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1043, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.059456096, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000057004 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1977424, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 140624 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4003, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.122549809, 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000030614 2024-04-20T11:33:32.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-20T11:33:32.100 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-20T11:33:32.101 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 296, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 296, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 296, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 1, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-04-20T11:33:32.102 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:32.103 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-20T11:33:32.104 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.105 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-20T11:33:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-20T11:33:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 1269, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-20T11:33:32.108 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 3, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 6675656, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 6673095, 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-20T11:33:32.109 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 1269 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 11, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 438, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 427, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 5, 2024-04-20T11:33:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 104, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 99, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 296, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 666, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 370, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 51932, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-04-20T11:33:32.111 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.115 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.116 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.116 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-20T11:33:32.116 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.116 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-20T11:33:32.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.118 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-20T11:33:32.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.120 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-20T11:33:32.122 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.123 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 5, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 4, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-20T11:33:32.124 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 2076, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:32.125 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:32.126 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:32.127 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 190435, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 60, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 163, 2024-04-20T11:33:32.128 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 248099, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 1671, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-20T11:33:32.129 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2393, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2393, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2393, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.982970743, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000828654 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2393, 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2394 2024-04-20T11:33:32.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.131 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2393, 2024-04-20T11:33:32.131 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2393, 2024-04-20T11:33:32.131 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-04-20T11:33:32.133 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-20T11:33:32.133 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1533, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 858, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-20T11:33:32.134 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-20T11:33:32.135 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-20T11:33:32.136 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 98, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 13, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 5, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 5, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-20T11:33:32.137 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T11:33:32.138 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 2682, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 4877681, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 2682, 2024-04-20T11:33:32.139 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2682, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 4877681, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2393, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2481357, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2393, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2481357, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:32.141 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2393, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2393, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2393, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2393, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.142 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:32.143 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:32.144 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x561aa1e726e0": { 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:32.145 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:32.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:32.146 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T11:33:32.891 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-20T11:33:32.891 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-20T11:33:32.891 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-20T11:33:32.891 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:32.891 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:32.891 DEBUG:tasks.check_counter:Getting stats from i 2024-04-20T11:33:32.891 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-20T11:33:33.135 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:33:33.515 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:33 smithi165 ceph-mon[21975]: pgmap v341: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:33:33.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:33 smithi078 ceph-mon[21464]: pgmap v341: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:33:33.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:33 smithi044 ceph-mon[16855]: pgmap v341: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2357, 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3128, 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 855866, 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 3944516, 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 25, 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:34.943 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.177995530, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.074197096, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.120778201, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.007546552, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3123, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.072715256, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000023283 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4049, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000658173, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000162 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.944 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 445152, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 1144592 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 732, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 947, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 261736, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1154997, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 39, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.068504782, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.026482991, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.038237082, 2024-04-20T11:33:34.945 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.005123382, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 944, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.056823112, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000060193 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1008, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000153918, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000152 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2160, 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 466880 2024-04-20T11:33:34.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1734, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1577, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2850685, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 753240, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 10, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.163792349, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.059847659, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.110606931, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.014681397, 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:34.947 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1580, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.057614174, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000036464 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2189, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000444857, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000203 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1986432, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 151360 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-20T11:33:34.948 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 481, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.331041733, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000688236 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 616, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.024219897, 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000039318 2024-04-20T11:33:34.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 608, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 578, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 578, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.741988807, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001283717 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 30, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-20T11:33:34.950 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 26, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 2, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 1, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 23, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 23, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 222, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 4, 2024-04-20T11:33:34.951 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 1217, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 801, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 28, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 21, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 16, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 148, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 16, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 227, 2024-04-20T11:33:34.952 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 139, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 139, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 2, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 2, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 137, 2024-04-20T11:33:34.953 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 44, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 397, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 409, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 323, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 33, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 569 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-20T11:33:34.954 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 21, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 13, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 2, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 286, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 284, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-20T11:33:34.955 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 16, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 1269, 2024-04-20T11:33:34.956 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 1269, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 3, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 3, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 6675656, 2024-04-20T11:33:34.957 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 128, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.257561523, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002012199 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 28, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 428, 2024-04-20T11:33:34.958 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 400, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 22, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 89, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 67, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 26, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 681, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 655, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 306, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 306, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 58336, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 223608 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 996, 2024-04-20T11:33:34.959 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 19, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 608, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 23, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 380, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 59, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.070814656, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001200248 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001794194, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000897097 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.960 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 204, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.063683452, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000312173 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-20T11:33:34.961 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.963 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.964 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.965 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 27, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.022389286, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000829232 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.050281603, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008380267 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.969 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 280, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.533025616, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001903662 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.970 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 5, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 4, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 403, 2024-04-20T11:33:34.971 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 403, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 2075, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:34.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:34.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 300741, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 131, 2024-04-20T11:33:34.974 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 704, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 8, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7840, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 163, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-20T11:33:34.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 132576, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 932, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 720, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 2567435, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 33, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 720, 2024-04-20T11:33:34.976 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 720, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.001970536, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002780514 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 720, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 799 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 720, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 167, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 553, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-04-20T11:33:34.977 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 28, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 156, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 32, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 296, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 139, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-04-20T11:33:34.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-20T11:33:34.979 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 98, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 13, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-04-20T11:33:34.980 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 90, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 28 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-20T11:33:34.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 6, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 6, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 284, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 284, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 4819, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 3464384, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 4819, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4819, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 3464384, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.983 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 720, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2596727, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 720, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2596727, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.984 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 720, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 720, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 720, 2024-04-20T11:33:34.985 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 720, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 284, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 29292, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:34.986 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 284, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 28, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 29292, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55795f7be500": { 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-20T11:33:34.987 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1269, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2481352, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1269, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 128, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2481352, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:34.988 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T11:33:35.779 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:35 smithi044 ceph-mon[16855]: pgmap v342: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:35.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:35 smithi165 ceph-mon[21975]: pgmap v342: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:35.826 INFO:tasks.check_counter:Daemon mds.i mds.exported=16 2024-04-20T11:33:35.826 INFO:tasks.check_counter:Daemon mds.i mds.imported=16 2024-04-20T11:33:35.826 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-20T11:33:35.826 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-20T11:33:35.826 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T11:33:35.826 DEBUG:tasks.check_counter:Getting stats from l 2024-04-20T11:33:35.826 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-20T11:33:35.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:35 smithi078 ceph-mon[21464]: pgmap v342: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T11:33:37.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:37 smithi044 ceph-mon[16855]: pgmap v343: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:37.743 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:37 smithi165 ceph-mon[21975]: pgmap v343: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 241, 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 229, 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1733600, 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 141344, 2024-04-20T11:33:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 0, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.051555287, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.014683618, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.039715189, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.006048731, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 228, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013096422, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000057440 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1733600, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 141344 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 176, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 320, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 187216, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.011430996, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.007836301, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.001202904, 2024-04-20T11:33:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 173, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.027776327, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000160556 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 187216 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 3024, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 448, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 2, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 1, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.000616123, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.000035737, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.000567290, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000803798, 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000267932 2024-04-20T11:33:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-20T11:33:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T11:33:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-20T11:33:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 148692, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 30, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-20T11:33:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 928, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 2, 2024-04-20T11:33:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 243, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1699604, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 243, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-20T11:33:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 243, 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1699604, 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T11:33:37.781 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T11:33:37.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:37 smithi078 ceph-mon[21464]: pgmap v343: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:38.577 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-20T11:33:38.577 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-20T11:33:38.577 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-20T11:33:38.578 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-20T11:33:38.578 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-20T11:33:38.578 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-20T11:33:38.590 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-20T11:33:38.590 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:33:38.590 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:38.653 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T11:33:38.653 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:33:38.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 -- 172.21.15.44:0/1108102461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf881014c0 msgr2=0x7fdf8810e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 --2- 172.21.15.44:0/1108102461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf881014c0 0x7fdf8810e0f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009900 tx=0x7fdf7002bc30 comp rx=0 tx=0).stop 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 -- 172.21.15.44:0/1108102461 shutdown_connections 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 --2- 172.21.15.44:0/1108102461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8810e6c0 0x7fdf88110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 --2- 172.21.15.44:0/1108102461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf881014c0 0x7fdf8810e0f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 --2- 172.21.15.44:0/1108102461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf88100b10 0x7fdf88100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 -- 172.21.15.44:0/1108102461 >> 172.21.15.44:0/1108102461 conn(0x7fdf880fc700 msgr2=0x7fdf880feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 -- 172.21.15.44:0/1108102461 shutdown_connections 2024-04-20T11:33:38.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 -- 172.21.15.44:0/1108102461 wait complete. 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.925+0000 7fdf8e600640 1 Processor -- start 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 -- start start 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 0x7fdf881a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf881014c0 0x7fdf881a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf8810e6c0 0x7fdf8819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88114440 con 0x7fdf881014c0 2024-04-20T11:33:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881142c0 con 0x7fdf88100b10 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881145c0 con 0x7fdf8810e6c0 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 0x7fdf881a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 0x7fdf881a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:50982/0 (socket says 172.21.15.44:50982) 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8d200640 1 -- 172.21.15.44:0/673040543 learned_addr learned my addr 172.21.15.44:0/673040543 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf7fe00640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf881014c0 0x7fdf881a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.926+0000 7fdf8dc00640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf8810e6c0 0x7fdf8819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8d200640 1 -- 172.21.15.44:0/673040543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf8810e6c0 msgr2=0x7fdf8819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8d200640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf8810e6c0 0x7fdf8819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8d200640 1 -- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf881014c0 msgr2=0x7fdf881a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8d200640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf881014c0 0x7fdf881a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8d200640 1 -- 172.21.15.44:0/673040543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8819b020 con 0x7fdf88100b10 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8dc00640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf8810e6c0 0x7fdf8819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8d200640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 0x7fdf881a06a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9f0 tx=0x7fdf7800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800cd80 con 0x7fdf88100b10 2024-04-20T11:33:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf78005520 con 0x7fdf88100b10 2024-04-20T11:33:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8819b310 con 0x7fdf88100b10 2024-04-20T11:33:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78010690 con 0x7fdf88100b10 2024-04-20T11:33:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.927+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881a7620 con 0x7fdf88100b10 2024-04-20T11:33:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.929+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8819b650 con 0x7fdf88100b10 2024-04-20T11:33:38.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.930+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdf780046c0 con 0x7fdf88100b10 2024-04-20T11:33:38.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.932+0000 7fdf86000640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdf6c0a3da0 0x7fdf6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:38.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.932+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fdf780d0c60 con 0x7fdf88100b10 2024-04-20T11:33:38.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.932+0000 7fdf7fe00640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdf6c0a3da0 0x7fdf6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:38.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.933+0000 7fdf7fe00640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdf6c0a3da0 0x7fdf6c0a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf70006010 tx=0x7fdf70005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:38.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:38.936+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdf7809d2e0 con 0x7fdf88100b10 2024-04-20T11:33:39.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.142+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf8819bcc0 con 0x7fdf88100b10 2024-04-20T11:33:39.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.144+0000 7fdf86000640 1 -- 172.21.15.44:0/673040543 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+22977 (secure 0 0 0) 0x7fdf8819bcc0 con 0x7fdf88100b10 2024-04-20T11:33:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:33:39.146 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":98,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:33:13.092802+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:39.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdf6c0a3da0 msgr2=0x7fdf6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdf6c0a3da0 0x7fdf6c0a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf70006010 tx=0x7fdf70005e40 comp rx=0 tx=0).stop 2024-04-20T11:33:39.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 msgr2=0x7fdf881a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 0x7fdf881a06a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9f0 tx=0x7fdf7800edf0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 shutdown_connections 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf8810e6c0 0x7fdf8819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fdf6c0a3da0 0x7fdf6c0a6260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf881014c0 0x7fdf881a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 --2- 172.21.15.44:0/673040543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf88100b10 0x7fdf881a06a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 >> 172.21.15.44:0/673040543 conn(0x7fdf880fc700 msgr2=0x7fdf880feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 shutdown_connections 2024-04-20T11:33:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.148+0000 7fdf8e600640 1 -- 172.21.15.44:0/673040543 wait complete. 2024-04-20T11:33:39.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T11:14:38.055848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T11:19:59.835807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T11:20:01.109834+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:33:39.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 -- 172.21.15.44:0/2666318761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076db0 msgr2=0x7fca7c1126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 --2- 172.21.15.44:0/2666318761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076db0 0x7fca7c1126a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fca64009900 tx=0x7fca6402bc40 comp rx=0 tx=0).stop 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 -- 172.21.15.44:0/2666318761 shutdown_connections 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 --2- 172.21.15.44:0/2666318761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c112be0 0x7fca7c114fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 --2- 172.21.15.44:0/2666318761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076db0 0x7fca7c1126a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 --2- 172.21.15.44:0/2666318761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c076490 0x7fca7c076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 -- 172.21.15.44:0/2666318761 >> 172.21.15.44:0/2666318761 conn(0x7fca7c0ff9d0 msgr2=0x7fca7c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 -- 172.21.15.44:0/2666318761 shutdown_connections 2024-04-20T11:33:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.395+0000 7fca82400640 1 -- 172.21.15.44:0/2666318761 wait complete. 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.396+0000 7fca82400640 1 Processor -- start 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.396+0000 7fca82400640 1 -- start start 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076490 0x7fca7c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c076db0 0x7fca7c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c112be0 0x7fca7c1a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c118830 con 0x7fca7c076490 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1186b0 con 0x7fca7c076db0 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1189b0 con 0x7fca7c112be0 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca81000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076490 0x7fca7c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c076db0 0x7fca7c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca81000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076490 0x7fca7c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53962/0 (socket says 172.21.15.44:53962) 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca81000640 1 -- 172.21.15.44:0/313504622 learned_addr learned my addr 172.21.15.44:0/313504622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca81a00640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c112be0 0x7fca7c1a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 -- 172.21.15.44:0/313504622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c112be0 msgr2=0x7fca7c1a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c112be0 0x7fca7c1a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 -- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076490 msgr2=0x7fca7c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076490 0x7fca7c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 -- 172.21.15.44:0/313504622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c1a3d20 con 0x7fca7c076db0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca81a00640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c112be0 0x7fca7c1a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca73e00640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c076db0 0x7fca7c19f270 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca64036040 tx=0x7fca6402dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64043020 con 0x7fca7c076db0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c1a3fb0 con 0x7fca7c076db0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca6402de00 con 0x7fca7c076db0 2024-04-20T11:33:39.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64003a10 con 0x7fca7c076db0 2024-04-20T11:33:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.398+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c1ab8a0 con 0x7fca7c076db0 2024-04-20T11:33:39.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.399+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c068ca0 con 0x7fca7c076db0 2024-04-20T11:33:39.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.401+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fca64046020 con 0x7fca7c076db0 2024-04-20T11:33:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.403+0000 7fca7a000640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fca600a3d20 0x7fca600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.403+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fca640f2d40 con 0x7fca7c076db0 2024-04-20T11:33:39.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.403+0000 7fca81000640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fca600a3d20 0x7fca600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.404+0000 7fca81000640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fca600a3d20 0x7fca600a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009cd0 tx=0x7fca6c005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:39.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.406+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fca6404d020 con 0x7fca7c076db0 2024-04-20T11:33:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.616+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca7c10d7e0 con 0x7fca7c076db0 2024-04-20T11:33:39.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.617+0000 7fca7a000640 1 -- 172.21.15.44:0/313504622 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fca640bf4d0 con 0x7fca7c076db0 2024-04-20T11:33:39.616 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fca600a3d20 msgr2=0x7fca600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fca600a3d20 0x7fca600a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009cd0 tx=0x7fca6c005f90 comp rx=0 tx=0).stop 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c076db0 msgr2=0x7fca7c19f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c076db0 0x7fca7c19f270 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca64036040 tx=0x7fca6402dc10 comp rx=0 tx=0).stop 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 shutdown_connections 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca7c112be0 0x7fca7c1a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca7c076db0 0x7fca7c19f270 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fca600a3d20 0x7fca600a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 --2- 172.21.15.44:0/313504622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c076490 0x7fca7c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 >> 172.21.15.44:0/313504622 conn(0x7fca7c0ff9d0 msgr2=0x7fca7c101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 shutdown_connections 2024-04-20T11:33:39.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.620+0000 7fca82400640 1 -- 172.21.15.44:0/313504622 wait complete. 2024-04-20T11:33:39.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:33:39.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 -- 172.21.15.44:0/1451039071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3490074cb0 msgr2=0x7f349010e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 --2- 172.21.15.44:0/1451039071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3490074cb0 0x7f349010e0e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f347c009900 tx=0x7f347c02bcc0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 -- 172.21.15.44:0/1451039071 shutdown_connections 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 --2- 172.21.15.44:0/1451039071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f349010e620 0x7f3490110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 --2- 172.21.15.44:0/1451039071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3490074cb0 0x7f349010e0e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 --2- 172.21.15.44:0/1451039071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3490074390 0x7f3490074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.854+0000 7f3496200640 1 -- 172.21.15.44:0/1451039071 >> 172.21.15.44:0/1451039071 conn(0x7f34900fd930 msgr2=0x7f34900ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.855+0000 7f3496200640 1 -- 172.21.15.44:0/1451039071 shutdown_connections 2024-04-20T11:33:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.855+0000 7f3496200640 1 -- 172.21.15.44:0/1451039071 wait complete. 2024-04-20T11:33:39.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 Processor -- start 2024-04-20T11:33:39.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 -- start start 2024-04-20T11:33:39.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3490074390 0x7f34901a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3490074cb0 0x7f34901a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 0x7f349019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34901144e0 con 0x7f3490074cb0 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3490114360 con 0x7f349010e620 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3496200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3490114660 con 0x7f3490074390 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3495800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 0x7f349019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f348fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3490074cb0 0x7f34901a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3495800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 0x7f349019a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51024/0 (socket says 172.21.15.44:51024) 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.856+0000 7f3495800640 1 -- 172.21.15.44:0/3230716951 learned_addr learned my addr 172.21.15.44:0/3230716951 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3494e00640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3490074390 0x7f34901a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3495800640 1 -- 172.21.15.44:0/3230716951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3490074390 msgr2=0x7f34901a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3495800640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3490074390 0x7f34901a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3495800640 1 -- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3490074cb0 msgr2=0x7f34901a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3495800640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3490074cb0 0x7f34901a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3495800640 1 -- 172.21.15.44:0/3230716951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349019b030 con 0x7f349010e620 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3495800640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 0x7f349019a860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f348800e970 tx=0x7f348800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348800ccc0 con 0x7f349010e620 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f348800ce60 con 0x7f349010e620 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349019b320 con 0x7f349010e620 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.857+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488010690 con 0x7f349010e620 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.858+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34901a74f0 con 0x7f349010e620 2024-04-20T11:33:39.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.859+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3490075a40 con 0x7f349010e620 2024-04-20T11:33:39.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.861+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f34880046c0 con 0x7f349010e620 2024-04-20T11:33:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.862+0000 7f348d600640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34740a3da0 0x7f34740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:39.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.862+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f34880d0bc0 con 0x7f349010e620 2024-04-20T11:33:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.862+0000 7f3494e00640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34740a3da0 0x7f34740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:39.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.863+0000 7f3494e00640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34740a3da0 0x7f34740a6260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3480005fd0 tx=0x7f3480009500 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:39.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:39.866+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f348809d240 con 0x7f349010e620 2024-04-20T11:33:39.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:39 smithi078 ceph-mon[21464]: pgmap v344: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:33:39.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:39 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/673040543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:33:39.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:39 smithi044 ceph-mon[16855]: pgmap v344: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:33:39.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:39 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/673040543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:33:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:39 smithi165 ceph-mon[21975]: pgmap v344: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-20T11:33:39.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:39 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/673040543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:33:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.071+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34901097a0 con 0x7f349010e620 2024-04-20T11:33:40.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.072+0000 7f348d600640 1 -- 172.21.15.44:0/3230716951 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f348809cb40 con 0x7f349010e620 2024-04-20T11:33:40.072 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34740a3da0 msgr2=0x7f34740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34740a3da0 0x7f34740a6260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3480005fd0 tx=0x7f3480009500 comp rx=0 tx=0).stop 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 msgr2=0x7f349019a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 0x7f349019a860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f348800e970 tx=0x7f348800ed70 comp rx=0 tx=0).stop 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 shutdown_connections 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f349010e620 0x7f349019a860 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f34740a3da0 0x7f34740a6260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3490074cb0 0x7f34901a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 --2- 172.21.15.44:0/3230716951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3490074390 0x7f34901a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.075+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 >> 172.21.15.44:0/3230716951 conn(0x7f34900fd930 msgr2=0x7f34900ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.076+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 shutdown_connections 2024-04-20T11:33:40.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.076+0000 7f3496200640 1 -- 172.21.15.44:0/3230716951 wait complete. 2024-04-20T11:33:40.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:33:40.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 -- 172.21.15.44:0/4189005277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 msgr2=0x7fee080746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 --2- 172.21.15.44:0/4189005277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee080746e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ca0 tx=0x7fedf802c190 comp rx=0 tx=0).stop 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 -- 172.21.15.44:0/4189005277 shutdown_connections 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 --2- 172.21.15.44:0/4189005277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee08112d10 0x7fee08115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 --2- 172.21.15.44:0/4189005277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 0x7fee081127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 --2- 172.21.15.44:0/4189005277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee080746e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 -- 172.21.15.44:0/4189005277 >> 172.21.15.44:0/4189005277 conn(0x7fee080fd830 msgr2=0x7fee080ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 -- 172.21.15.44:0/4189005277 shutdown_connections 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.315+0000 7fee0d200640 1 -- 172.21.15.44:0/4189005277 wait complete. 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.316+0000 7fee0d200640 1 Processor -- start 2024-04-20T11:33:40.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 -- start start 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee0819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 0x7fee0819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee08112d10 0x7fee081a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fee08118830 con 0x7fee08074c20 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee081186b0 con 0x7fee08074300 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee0d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee081189b0 con 0x7fee08112d10 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee07400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee0819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee07400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee0819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51050/0 (socket says 172.21.15.44:51050) 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.317+0000 7fee07400640 1 -- 172.21.15.44:0/154294725 learned_addr learned my addr 172.21.15.44:0/154294725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07e00640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee08112d10 0x7fee081a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee06a00640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 0x7fee0819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07400640 1 -- 172.21.15.44:0/154294725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee08112d10 msgr2=0x7fee081a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07400640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee08112d10 0x7fee081a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07400640 1 -- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 msgr2=0x7fee0819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07400640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 0x7fee0819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07400640 1 -- 172.21.15.44:0/154294725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee081a3d20 con 0x7fee08074300 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee06a00640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 0x7fee0819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.318+0000 7fee07400640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee0819ed30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fedf8005e30 tx=0x7fedf8034760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.319+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8039040 con 0x7fee08074300 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.319+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fedf8044070 con 0x7fee08074300 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.319+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee081a3fb0 con 0x7fee08074300 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.319+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80043e0 con 0x7fee08074300 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.319+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081ab8a0 con 0x7fee08074300 2024-04-20T11:33:40.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.320+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee080690f0 con 0x7fee08074300 2024-04-20T11:33:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.322+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fedf8004580 con 0x7fee08074300 2024-04-20T11:33:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.323+0000 7fedffe00640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fedec0a3da0 0x7fedec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.324+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fedf80f2c50 con 0x7fee08074300 2024-04-20T11:33:40.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.324+0000 7fee06a00640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fedec0a3da0 0x7fedec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.325+0000 7fee06a00640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fedec0a3da0 0x7fedec0a6260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fedf40058c0 tx=0x7fedf4005850 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.327+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fedf80bf3e0 con 0x7fee08074300 2024-04-20T11:33:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.535+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fee081051d0 con 0x7fee08074300 2024-04-20T11:33:40.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.535+0000 7fedffe00640 1 -- 172.21.15.44:0/154294725 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fedf80bece0 con 0x7fee08074300 2024-04-20T11:33:40.535 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fedec0a3da0 msgr2=0x7fedec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fedec0a3da0 0x7fedec0a6260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fedf40058c0 tx=0x7fedf4005850 comp rx=0 tx=0).stop 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 msgr2=0x7fee0819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee0819ed30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fedf8005e30 tx=0x7fedf8034760 comp rx=0 tx=0).stop 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 shutdown_connections 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee08112d10 0x7fee081a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fedec0a3da0 0x7fedec0a6260 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee08074c20 0x7fee0819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 --2- 172.21.15.44:0/154294725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee08074300 0x7fee0819ed30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 >> 172.21.15.44:0/154294725 conn(0x7fee080fd830 msgr2=0x7fee080ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 shutdown_connections 2024-04-20T11:33:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.539+0000 7fee0d200640 1 -- 172.21.15.44:0/154294725 wait complete. 2024-04-20T11:33:40.558 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:33:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 -- 172.21.15.44:0/276552365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c104410 msgr2=0x7f204c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 --2- 172.21.15.44:0/276552365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c104410 0x7f204c1047f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2044009900 tx=0x7f204402bc40 comp rx=0 tx=0).stop 2024-04-20T11:33:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 -- 172.21.15.44:0/276552365 shutdown_connections 2024-04-20T11:33:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 --2- 172.21.15.44:0/276552365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f204c1094e0 0x7f204c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 --2- 172.21.15.44:0/276552365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104dc0 0x7f204c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 --2- 172.21.15.44:0/276552365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c104410 0x7f204c1047f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 -- 172.21.15.44:0/276552365 >> 172.21.15.44:0/276552365 conn(0x7f204c0ffcf0 msgr2=0x7f204c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.772+0000 7f2051000640 1 -- 172.21.15.44:0/276552365 shutdown_connections 2024-04-20T11:33:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.773+0000 7f2051000640 1 -- 172.21.15.44:0/276552365 wait complete. 2024-04-20T11:33:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.773+0000 7f2051000640 1 Processor -- start 2024-04-20T11:33:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 -- start start 2024-04-20T11:33:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104410 0x7f204c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 0x7f204c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f204c19aa30 0x7f204c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1146d0 con 0x7f204c1094e0 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f204c114550 con 0x7f204c104410 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2051000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f204c114850 con 0x7f204c19aa30 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2049400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 0x7f204c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2049e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104410 0x7f204c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2049400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 0x7f204c1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54016/0 (socket says 172.21.15.44:54016) 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f2049400640 1 -- 172.21.15.44:0/2226412794 learned_addr learned my addr 172.21.15.44:0/2226412794 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.774+0000 7f204a800640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f204c19aa30 0x7f204c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049400640 1 -- 172.21.15.44:0/2226412794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f204c19aa30 msgr2=0x7f204c19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049400640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f204c19aa30 0x7f204c19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049400640 1 -- 172.21.15.44:0/2226412794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104410 msgr2=0x7f204c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049400640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104410 0x7f204c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049400640 1 -- 172.21.15.44:0/2226412794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c19b550 con 0x7f204c1094e0 2024-04-20T11:33:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049e00640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104410 0x7f204c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2049400640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 0x7f204c1a0dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f203400eb90 tx=0x7f203400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.775+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034010630 con 0x7f204c1094e0 2024-04-20T11:33:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.776+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f203400ee90 con 0x7f204c1094e0 2024-04-20T11:33:40.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.776+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c19b840 con 0x7f204c1094e0 2024-04-20T11:33:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.776+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034018cb0 con 0x7f204c1094e0 2024-04-20T11:33:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.776+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1a7a90 con 0x7f204c1094e0 2024-04-20T11:33:40.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.777+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c0691d0 con 0x7f204c1094e0 2024-04-20T11:33:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.779+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f20340107d0 con 0x7f204c1094e0 2024-04-20T11:33:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.780+0000 7f2042a00640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f20300a3d20 0x7f20300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.781+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f20340d0e00 con 0x7f204c1094e0 2024-04-20T11:33:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.781+0000 7f2049e00640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f20300a3d20 0x7f20300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.782+0000 7f2049e00640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f20300a3d20 0x7f20300a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2044005ab0 tx=0x7f2044005a20 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:40.785+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f203409ce30 con 0x7f204c1094e0 2024-04-20T11:33:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/313504622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:33:40.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:40 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3230716951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:33:40.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/313504622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:33:40.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:40 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3230716951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:33:40.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/313504622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T11:33:40.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:40 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3230716951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T11:33:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.007+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f204c19bf60 con 0x7f204c1094e0 2024-04-20T11:33:41.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.008+0000 7f2042a00640 1 -- 172.21.15.44:0/2226412794 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v98) ==== 81+0+942 (secure 0 0 0) 0x7f204c19bf60 con 0x7f204c1094e0 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2017822852 2024-04-21T11:22:36.090189+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/2425002048 2024-04-21T11:22:36.090189+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/464946469 2024-04-21T11:22:36.090189+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/748971044 2024-04-21T11:22:36.090189+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/1299451251 2024-04-21T11:11:49.892352+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/1299451251 2024-04-21T11:11:49.892352+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3474255824 2024-04-21T11:11:49.892352+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3779581802 2024-04-21T11:11:49.892352+0000 2024-04-20T11:33:41.008 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/1801045838 2024-04-21T11:11:14.579278+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/1801045838 2024-04-21T11:11:14.579278+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2316836529 2024-04-21T11:22:36.090189+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/2425002048 2024-04-21T11:22:36.090189+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2620659555 2024-04-21T11:11:14.579278+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/135755932 2024-04-21T11:11:14.579278+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/393908752 2024-04-21T11:11:49.892352+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3844329137 2024-04-21T11:11:14.579278+0000 2024-04-20T11:33:41.009 INFO:teuthology.orchestra.run.smithi044.stderr:listed 16 entries 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.011+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f20300a3d20 msgr2=0x7f20300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.011+0000 7f2051000640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f20300a3d20 0x7f20300a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2044005ab0 tx=0x7f2044005a20 comp rx=0 tx=0).stop 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.011+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 msgr2=0x7f204c1a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.011+0000 7f2051000640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 0x7f204c1a0dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f203400eb90 tx=0x7f203400c6a0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 shutdown_connections 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f204c19aa30 0x7f204c19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f20300a3d20 0x7f20300a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f204c1094e0 0x7f204c1a0dc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 --2- 172.21.15.44:0/2226412794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c104410 0x7f204c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 >> 172.21.15.44:0/2226412794 conn(0x7f204c0ffcf0 msgr2=0x7f204c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 shutdown_connections 2024-04-20T11:33:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.012+0000 7f2051000640 1 -- 172.21.15.44:0/2226412794 wait complete. 2024-04-20T11:33:41.031 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-20T11:33:41.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-20T11:33:41.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:33:41.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.199 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:33:41.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.300 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-20T11:33:41.377 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:33:41.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:33:41.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:33:41.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-20T11:33:41.595 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-20T11:33:41.605 INFO:tasks.cephadm:Teardown begin 2024-04-20T11:33:41.606 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:33:41.606 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 -- 172.21.15.44:0/59811130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108400 msgr2=0x7fcd9c108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 --2- 172.21.15.44:0/59811130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108400 0x7fcd9c108860 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd8402bcd0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 -- 172.21.15.44:0/59811130 shutdown_connections 2024-04-20T11:33:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 --2- 172.21.15.44:0/59811130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c108da0 0x7fcd9c115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 --2- 172.21.15.44:0/59811130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108400 0x7fcd9c108860 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 --2- 172.21.15.44:0/59811130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 0x7fcd9c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 -- 172.21.15.44:0/59811130 >> 172.21.15.44:0/59811130 conn(0x7fcd9c077ae0 msgr2=0x7fcd9c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 -- 172.21.15.44:0/59811130 shutdown_connections 2024-04-20T11:33:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.779+0000 7fcd9be00640 1 -- 172.21.15.44:0/59811130 wait complete. 2024-04-20T11:33:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.780+0000 7fcd9be00640 1 Processor -- start 2024-04-20T11:33:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.780+0000 7fcd9be00640 1 -- start start 2024-04-20T11:33:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c108400 0x7fcd9c1a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 0x7fcd9c1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 0x7fcd9c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c118900 con 0x7fcd9c10e470 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c118780 con 0x7fcd9c108da0 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c118a80 con 0x7fcd9c108400 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd93e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 0x7fcd9c1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd93e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 0x7fcd9c1a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51082/0 (socket says 172.21.15.44:51082) 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd93e00640 1 -- 172.21.15.44:0/1237003215 learned_addr learned my addr 172.21.15.44:0/1237003215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9aa00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c108400 0x7fcd9c1a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.781+0000 7fcd9b400640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 0x7fcd9c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93e00640 1 -- 172.21.15.44:0/1237003215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c108400 msgr2=0x7fcd9c1a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93e00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c108400 0x7fcd9c1a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93e00640 1 -- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 msgr2=0x7fcd9c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93e00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 0x7fcd9c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93e00640 1 -- 172.21.15.44:0/1237003215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c19f240 con 0x7fcd9c108da0 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd9b400640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 0x7fcd9c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93e00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 0x7fcd9c1a4e50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd84004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84043020 con 0x7fcd9c108da0 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd84004250 con 0x7fcd9c108da0 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c19f4d0 con 0x7fcd9c108da0 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84003ab0 con 0x7fcd9c108da0 2024-04-20T11:33:41.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.782+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c1ab7d0 con 0x7fcd9c108da0 2024-04-20T11:33:41.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.783+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c0690f0 con 0x7fcd9c108da0 2024-04-20T11:33:41.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.785+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcd84046020 con 0x7fcd9c108da0 2024-04-20T11:33:41.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.787+0000 7fcd93400640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd800a3da0 0x7fcd800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:41.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.787+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fcd840f2610 con 0x7fcd9c108da0 2024-04-20T11:33:41.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.787+0000 7fcd9aa00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd800a3da0 0x7fcd800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:41.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.787+0000 7fcd9aa00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd800a3da0 0x7fcd800a6260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c19fc00 tx=0x7fcd8c005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:41.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:41.791+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcd8404d020 con 0x7fcd9c108da0 2024-04-20T11:33:41.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:41 smithi078 ceph-mon[21464]: pgmap v345: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:41 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/154294725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:33:41.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:41 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2226412794' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:33:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:41 smithi044 ceph-mon[16855]: pgmap v345: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:41 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/154294725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:33:41.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:41 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2226412794' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:33:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:41 smithi165 ceph-mon[21975]: pgmap v345: 97 pgs: 97 active+clean; 139 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-20T11:33:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:41 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/154294725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T11:33:41.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:41 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2226412794' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T11:33:42.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.031+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd9c1abb00 con 0x7fcd9c108da0 2024-04-20T11:33:42.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.033+0000 7fcd93400640 1 -- 172.21.15.44:0/1237003215 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7fcd840beec0 con 0x7fcd9c108da0 2024-04-20T11:33:42.033 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:33:42.033 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:33:42.033 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd800a3da0 msgr2=0x7fcd800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd800a3da0 0x7fcd800a6260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c19fc00 tx=0x7fcd8c005f90 comp rx=0 tx=0).stop 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 msgr2=0x7fcd9c1a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 0x7fcd9c1a4e50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd84004950 comp rx=0 tx=0).stop 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 shutdown_connections 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fcd800a3da0 0x7fcd800a6260 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c10e470 0x7fcd9c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd9c108da0 0x7fcd9c1a4e50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 --2- 172.21.15.44:0/1237003215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c108400 0x7fcd9c1a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 >> 172.21.15.44:0/1237003215 conn(0x7fcd9c077ae0 msgr2=0x7fcd9c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 shutdown_connections 2024-04-20T11:33:42.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.036+0000 7fcd9be00640 1 -- 172.21.15.44:0/1237003215 wait complete. 2024-04-20T11:33:42.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:33:42.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.258+0000 7f78b3e00640 1 -- 172.21.15.44:0/1791698800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b41070c0 msgr2=0x7f78b410ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.258+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1791698800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b41070c0 0x7f78b410ec90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009860 tx=0x7f78ac02bb70 comp rx=0 tx=0).stop 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 -- 172.21.15.44:0/1791698800 shutdown_connections 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1791698800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b41070c0 0x7f78b410ec90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1791698800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41029a0 0x7f78b41069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1791698800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 -- 172.21.15.44:0/1791698800 >> 172.21.15.44:0/1791698800 conn(0x7f78b40fd890 msgr2=0x7f78b40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 -- 172.21.15.44:0/1791698800 shutdown_connections 2024-04-20T11:33:42.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.259+0000 7f78b3e00640 1 -- 172.21.15.44:0/1791698800 wait complete. 2024-04-20T11:33:42.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 Processor -- start 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 -- start start 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b4074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41070c0 0x7f78b40747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 0x7f78b41a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4112290 con 0x7f78b4074ce0 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4112110 con 0x7f78b41070c0 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4112410 con 0x7f78b4101ff0 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 0x7f78b41a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.260+0000 7f78b2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b4074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41070c0 0x7f78b40747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 0x7f78b41a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54066/0 (socket says 172.21.15.44:54066) 2024-04-20T11:33:42.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 -- 172.21.15.44:0/1109867809 learned_addr learned my addr 172.21.15.44:0/1109867809 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b4074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41864/0 (socket says 172.21.15.44:41864) 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 -- 172.21.15.44:0/1109867809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 msgr2=0x7f78b4074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b4074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 -- 172.21.15.44:0/1109867809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41070c0 msgr2=0x7f78b40747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41070c0 0x7f78b40747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 -- 172.21.15.44:0/1109867809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b41a6fc0 con 0x7f78b4074ce0 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b2a00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b4074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b2000640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41070c0 0x7f78b40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.261+0000 7f78b3400640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 0x7f78b41a68c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f78ac02c180 tx=0x7f78ac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.262+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac039040 con 0x7f78b4074ce0 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.262+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f78ac042070 con 0x7f78b4074ce0 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.262+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b41a7250 con 0x7f78b4074ce0 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.262+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0046c0 con 0x7f78b4074ce0 2024-04-20T11:33:42.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.262+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b41a7740 con 0x7f78b4074ce0 2024-04-20T11:33:42.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.263+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b4068ca0 con 0x7f78b4074ce0 2024-04-20T11:33:42.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.265+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f78ac005030 con 0x7f78b4074ce0 2024-04-20T11:33:42.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.266+0000 7f78ab400640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78980a3d50 0x7f78980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.266+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f78ac0f1c90 con 0x7f78b4074ce0 2024-04-20T11:33:42.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.267+0000 7f78b2a00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78980a3d50 0x7f78980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.267+0000 7f78b2a00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78980a3d50 0x7f78980a6210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78a4005fd0 tx=0x7f78a4010040 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.271+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f78ac0be420 con 0x7f78b4074ce0 2024-04-20T11:33:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.514+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78b41029a0 con 0x7f78b4074ce0 2024-04-20T11:33:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.515+0000 7f78ab400640 1 -- 172.21.15.44:0/1109867809 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11941 (secure 0 0 0) 0x7f78ac034920 con 0x7f78b4074ce0 2024-04-20T11:33:42.515 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:33:42.515 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/503506102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":503506102},{"type":"v1","addr":"172.21.15.44:6841","nonce":503506102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3129181151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3129181151},{"type":"v1","addr":"172.21.15.165:6833","nonce":3129181151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T11:20:02.149413+0000","modified":"2024-04-20T11:29:02.424485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24596,"mds_1":24562,"mds_2":24550,"mds_3":24506,"mds_4":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6833/715928335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":715928335},{"type":"v1","addr":"172.21.15.78:6833","nonce":715928335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/858649103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":858649103},{"type":"v1","addr":"172.21.15.44:6835","nonce":858649103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"d","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.44:6837/2623237731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":2623237731},{"type":"v1","addr":"172.21.15.44:6837","nonce":2623237731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.78:6835/3770087097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3770087097},{"type":"v1","addr":"172.21.15.78:6835","nonce":3770087097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1721181859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1721181859},{"type":"v1","addr":"172.21.15.165:6835","nonce":1721181859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2414137592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2414137592},{"type":"v1","addr":"172.21.15.165:6837","nonce":2414137592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6837/2547716992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2547716992},{"type":"v1","addr":"172.21.15.78:6837","nonce":2547716992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/4179547084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4179547084},{"type":"v1","addr":"172.21.15.165:6839","nonce":4179547084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6839/2240418551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2240418551},{"type":"v1","addr":"172.21.15.44:6839","nonce":2240418551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/315239720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":315239720},{"type":"v1","addr":"172.21.15.78:6839","nonce":315239720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24562,24506,24596,24550,24494]},"id":1}]} 2024-04-20T11:33:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 55 2024-04-20T11:33:42.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78980a3d50 msgr2=0x7f78980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78980a3d50 0x7f78980a6210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78a4005fd0 tx=0x7f78a4010040 comp rx=0 tx=0).stop 2024-04-20T11:33:42.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 msgr2=0x7f78b41a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 0x7f78b41a68c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f78ac02c180 tx=0x7f78ac0045e0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 shutdown_connections 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f78980a3d50 0x7f78980a6210 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78b4074ce0 0x7f78b41a68c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b41070c0 0x7f78b40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 --2- 172.21.15.44:0/1109867809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b4101ff0 0x7f78b4074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 >> 172.21.15.44:0/1109867809 conn(0x7f78b40fd890 msgr2=0x7f78b40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 shutdown_connections 2024-04-20T11:33:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.518+0000 7f78b3e00640 1 -- 172.21.15.44:0/1109867809 wait complete. 2024-04-20T11:33:42.535 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 -- 172.21.15.44:0/3324725875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 msgr2=0x7fe3941032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 --2- 172.21.15.44:0/3324725875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 0x7fe3941032f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe384009900 tx=0x7fe38402bcd0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 -- 172.21.15.44:0/3324725875 shutdown_connections 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 --2- 172.21.15.44:0/3324725875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe39410c330 0x7fe39410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 --2- 172.21.15.44:0/3324725875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe39410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 --2- 172.21.15.44:0/3324725875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 0x7fe3941032f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 -- 172.21.15.44:0/3324725875 >> 172.21.15.44:0/3324725875 conn(0x7fe3940fc710 msgr2=0x7fe3940feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 -- 172.21.15.44:0/3324725875 shutdown_connections 2024-04-20T11:33:42.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.737+0000 7fe399400640 1 -- 172.21.15.44:0/3324725875 wait complete. 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.738+0000 7fe399400640 1 Processor -- start 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.738+0000 7fe399400640 1 -- start start 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe399400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 0x7fe3941a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe399400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe3941a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe399400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe39410c330 0x7fe39419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe399400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe394112250 con 0x7fe39410c330 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe399400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941120d0 con 0x7fe394103830 2024-04-20T11:33:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe399400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941123d0 con 0x7fe394102f10 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe38b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe3941a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe38b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe3941a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51114/0 (socket says 172.21.15.44:51114) 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe38b400640 1 -- 172.21.15.44:0/3555070991 learned_addr learned my addr 172.21.15.44:0/3555070991 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe398a00640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe39410c330 0x7fe39419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.739+0000 7fe393e00640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 0x7fe3941a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe38b400640 1 -- 172.21.15.44:0/3555070991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 msgr2=0x7fe3941a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe38b400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 0x7fe3941a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe38b400640 1 -- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe39410c330 msgr2=0x7fe39419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe38b400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe39410c330 0x7fe39419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe38b400640 1 -- 172.21.15.44:0/3555070991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39419af50 con 0x7fe394103830 2024-04-20T11:33:42.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe38b400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe3941a0b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00eb10 tx=0x7fe37c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00ce20 con 0x7fe394103830 2024-04-20T11:33:42.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe37c010930 con 0x7fe394103830 2024-04-20T11:33:42.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39419b240 con 0x7fe394103830 2024-04-20T11:33:42.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c018b90 con 0x7fe394103830 2024-04-20T11:33:42.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.740+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a7460 con 0x7fe394103830 2024-04-20T11:33:42.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.741+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39406ae20 con 0x7fe394103830 2024-04-20T11:33:42.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.743+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe37c010ae0 con 0x7fe394103830 2024-04-20T11:33:42.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.745+0000 7fe391600640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3780a3da0 0x7fe3780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:42.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.745+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fe37c0d0ca0 con 0x7fe394103830 2024-04-20T11:33:42.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.745+0000 7fe393e00640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3780a3da0 0x7fe3780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:42.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.745+0000 7fe393e00640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3780a3da0 0x7fe3780a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe384006010 tx=0x7fe384005e40 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:42.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.749+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe37c09d320 con 0x7fe394103830 2024-04-20T11:33:42.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:42 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1237003215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:33:42.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:42 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1109867809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:33:42.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:42 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1237003215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:33:42.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:42 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1109867809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:33:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.950+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3941a7760 con 0x7fe394103830 2024-04-20T11:33:42.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.952+0000 7fe391600640 1 -- 172.21.15.44:0/3555070991 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+22977 (secure 0 0 0) 0x7fe37c09cc20 con 0x7fe394103830 2024-04-20T11:33:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T11:33:42.952 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":98,"fsid":"759aac0c-ff06-11ee-bc92-c7b262605968","created":"2024-04-20T11:10:40.176355+0000","modified":"2024-04-20T11:33:13.092802+0000","last_up_change":"2024-04-20T11:19:02.167754+0000","last_in_change":"2024-04-20T11:18:42.672374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T11:14:38.055848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T11:19:59.835807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T11:20:01.109834+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2d4d49b-1218-474f-8e22-6d528d61f28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6803","nonce":694121710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6805","nonce":694121710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6809","nonce":694121710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":694121710},{"type":"v1","addr":"172.21.15.44:6807","nonce":694121710}]},"public_addr":"172.21.15.44:6803/694121710","cluster_addr":"172.21.15.44:6805/694121710","heartbeat_back_addr":"172.21.15.44:6809/694121710","heartbeat_front_addr":"172.21.15.44:6807/694121710","state":["exists","up"]},{"osd":1,"uuid":"7e99f802-a969-42c1-8c47-75ac8cc78349","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6801","nonce":2898064203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6803","nonce":2898064203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6807","nonce":2898064203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2898064203},{"type":"v1","addr":"172.21.15.78:6805","nonce":2898064203}]},"public_addr":"172.21.15.78:6801/2898064203","cluster_addr":"172.21.15.78:6803/2898064203","heartbeat_back_addr":"172.21.15.78:6807/2898064203","heartbeat_front_addr":"172.21.15.78:6805/2898064203","state":["exists","up"]},{"osd":2,"uuid":"25612bf3-e5b7-49e1-bf3c-929ddf669fc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6801","nonce":1225247316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6803","nonce":1225247316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6807","nonce":1225247316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1225247316},{"type":"v1","addr":"172.21.15.165:6805","nonce":1225247316}]},"public_addr":"172.21.15.165:6801/1225247316","cluster_addr":"172.21.15.165:6803/1225247316","heartbeat_back_addr":"172.21.15.165:6807/1225247316","heartbeat_front_addr":"172.21.15.165:6805/1225247316","state":["exists","up"]},{"osd":3,"uuid":"e081b5a8-ec52-40c2-bebd-3dd47ac73986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6811","nonce":3780220117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6813","nonce":3780220117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6817","nonce":3780220117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3780220117},{"type":"v1","addr":"172.21.15.44:6815","nonce":3780220117}]},"public_addr":"172.21.15.44:6811/3780220117","cluster_addr":"172.21.15.44:6813/3780220117","heartbeat_back_addr":"172.21.15.44:6817/3780220117","heartbeat_front_addr":"172.21.15.44:6815/3780220117","state":["exists","up"]},{"osd":4,"uuid":"97110c0d-1f21-4f21-be1f-9b026c6d37ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6809","nonce":856268583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6811","nonce":856268583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6815","nonce":856268583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":856268583},{"type":"v1","addr":"172.21.15.78:6813","nonce":856268583}]},"public_addr":"172.21.15.78:6809/856268583","cluster_addr":"172.21.15.78:6811/856268583","heartbeat_back_addr":"172.21.15.78:6815/856268583","heartbeat_front_addr":"172.21.15.78:6813/856268583","state":["exists","up"]},{"osd":5,"uuid":"1e45eea6-98f1-48a0-aac0-8836c0e46fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6809","nonce":2633955642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6811","nonce":2633955642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6815","nonce":2633955642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2633955642},{"type":"v1","addr":"172.21.15.165:6813","nonce":2633955642}]},"public_addr":"172.21.15.165:6809/2633955642","cluster_addr":"172.21.15.165:6811/2633955642","heartbeat_back_addr":"172.21.15.165:6815/2633955642","heartbeat_front_addr":"172.21.15.165:6813/2633955642","state":["exists","up"]},{"osd":6,"uuid":"e4fd9d25-8aa7-45e3-a5da-50ccddec8555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6819","nonce":2930726508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6821","nonce":2930726508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6825","nonce":2930726508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2930726508},{"type":"v1","addr":"172.21.15.44:6823","nonce":2930726508}]},"public_addr":"172.21.15.44:6819/2930726508","cluster_addr":"172.21.15.44:6821/2930726508","heartbeat_back_addr":"172.21.15.44:6825/2930726508","heartbeat_front_addr":"172.21.15.44:6823/2930726508","state":["exists","up"]},{"osd":7,"uuid":"29b9c0d1-e314-4337-b6cd-0a7c7514f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6817","nonce":1412804643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6819","nonce":1412804643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6823","nonce":1412804643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1412804643},{"type":"v1","addr":"172.21.15.78:6821","nonce":1412804643}]},"public_addr":"172.21.15.78:6817/1412804643","cluster_addr":"172.21.15.78:6819/1412804643","heartbeat_back_addr":"172.21.15.78:6823/1412804643","heartbeat_front_addr":"172.21.15.78:6821/1412804643","state":["exists","up"]},{"osd":8,"uuid":"3fb60ca9-b2b6-4c9d-8769-5710dacbf4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6817","nonce":2878265181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6819","nonce":2878265181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6823","nonce":2878265181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2878265181},{"type":"v1","addr":"172.21.15.165:6821","nonce":2878265181}]},"public_addr":"172.21.15.165:6817/2878265181","cluster_addr":"172.21.15.165:6819/2878265181","heartbeat_back_addr":"172.21.15.165:6823/2878265181","heartbeat_front_addr":"172.21.15.165:6821/2878265181","state":["exists","up"]},{"osd":9,"uuid":"7e0db795-eae2-41b0-b0f6-38e6fc4b56b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6827","nonce":3770179466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6829","nonce":3770179466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6833","nonce":3770179466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":3770179466},{"type":"v1","addr":"172.21.15.44:6831","nonce":3770179466}]},"public_addr":"172.21.15.44:6827/3770179466","cluster_addr":"172.21.15.44:6829/3770179466","heartbeat_back_addr":"172.21.15.44:6833/3770179466","heartbeat_front_addr":"172.21.15.44:6831/3770179466","state":["exists","up"]},{"osd":10,"uuid":"35e05c9e-6f8c-4dce-ad79-fa9597cb0743","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6825","nonce":259959588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6827","nonce":259959588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6831","nonce":259959588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":259959588},{"type":"v1","addr":"172.21.15.78:6829","nonce":259959588}]},"public_addr":"172.21.15.78:6825/259959588","cluster_addr":"172.21.15.78:6827/259959588","heartbeat_back_addr":"172.21.15.78:6831/259959588","heartbeat_front_addr":"172.21.15.78:6829/259959588","state":["exists","up"]},{"osd":11,"uuid":"97d59cd2-9c3e-4ce0-aeaf-508d5d07c6ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6825","nonce":1018684667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6827","nonce":1018684667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6831","nonce":1018684667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1018684667},{"type":"v1","addr":"172.21.15.165:6829","nonce":1018684667}]},"public_addr":"172.21.15.165:6825/1018684667","cluster_addr":"172.21.15.165:6827/1018684667","heartbeat_back_addr":"172.21.15.165:6831/1018684667","heartbeat_front_addr":"172.21.15.165:6829/1018684667","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:13:45.802011+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:11.300656+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:14:35.584997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:10.026227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:15:37.113304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:01.903116+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:16:38.061625+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:05.177750+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:17:30.856120+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:06.503121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:18:34.740643+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T11:19:00.621546+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2017822852":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/464946469":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/748971044":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6801/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6800/1299451251":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3474255824":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3779581802":"2024-04-21T11:11:49.892352+0000","172.21.15.44:6801/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:6800/1801045838":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/2316836529":"2024-04-21T11:22:36.090189+0000","172.21.15.44:6800/2425002048":"2024-04-21T11:22:36.090189+0000","172.21.15.44:0/2620659555":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/135755932":"2024-04-21T11:11:14.579278+0000","172.21.15.44:0/393908752":"2024-04-21T11:11:49.892352+0000","172.21.15.44:0/3844329137":"2024-04-21T11:11:14.579278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3780a3da0 msgr2=0x7fe3780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3780a3da0 0x7fe3780a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe384006010 tx=0x7fe384005e40 comp rx=0 tx=0).stop 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 msgr2=0x7fe3941a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe3941a0b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00eb10 tx=0x7fe37c00ef10 comp rx=0 tx=0).stop 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 shutdown_connections 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fe3780a3da0 0x7fe3780a6260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe39410c330 0x7fe39419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe394103830 0x7fe3941a0b50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 --2- 172.21.15.44:0/3555070991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe394102f10 0x7fe3941a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 >> 172.21.15.44:0/3555070991 conn(0x7fe3940fc710 msgr2=0x7fe394100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 shutdown_connections 2024-04-20T11:33:42.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:42.955+0000 7fe399400640 1 -- 172.21.15.44:0/3555070991 wait complete. 2024-04-20T11:33:42.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:33:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:42 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1237003215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:33:42.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:42 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1109867809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T11:33:43.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.176+0000 7fd0caa00640 1 -- 172.21.15.44:0/2531569104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c4101350 msgr2=0x7fd0c4109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.176+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2531569104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c4101350 0x7fd0c4109e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009900 tx=0x7fd0b002bc50 comp rx=0 tx=0).stop 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 -- 172.21.15.44:0/2531569104 shutdown_connections 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2531569104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0c410a560 0x7fd0c4110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2531569104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c4101350 0x7fd0c4109e80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2531569104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c41009a0 0x7fd0c4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 -- 172.21.15.44:0/2531569104 >> 172.21.15.44:0/2531569104 conn(0x7fd0c4076080 msgr2=0x7fd0c4076490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 -- 172.21.15.44:0/2531569104 shutdown_connections 2024-04-20T11:33:43.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.177+0000 7fd0caa00640 1 -- 172.21.15.44:0/2531569104 wait complete. 2024-04-20T11:33:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.178+0000 7fd0caa00640 1 Processor -- start 2024-04-20T11:33:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.178+0000 7fd0caa00640 1 -- start start 2024-04-20T11:33:43.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.178+0000 7fd0caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0c41009a0 0x7fd0c41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.178+0000 7fd0caa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 0x7fd0c41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0caa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c419a810 0x7fd0c419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0caa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4114510 con 0x7fd0c410a560 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0caa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4114390 con 0x7fd0c419a810 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0caa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4114690 con 0x7fd0c41009a0 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 0x7fd0c41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 0x7fd0c41a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54100/0 (socket says 172.21.15.44:54100) 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 -- 172.21.15.44:0/2475687328 learned_addr learned my addr 172.21.15.44:0/2475687328 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0ca000640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c419a810 0x7fd0c419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c9600640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0c41009a0 0x7fd0c41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 -- 172.21.15.44:0/2475687328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0c41009a0 msgr2=0x7fd0c41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0c41009a0 0x7fd0c41a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 -- 172.21.15.44:0/2475687328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c419a810 msgr2=0x7fd0c419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c419a810 0x7fd0c419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0c8c00640 1 -- 172.21.15.44:0/2475687328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c419b3a0 con 0x7fd0c410a560 2024-04-20T11:33:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.179+0000 7fd0ca000640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c419a810 0x7fd0c419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.180+0000 7fd0c8c00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 0x7fd0c41a0be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009230 tx=0x7fd0b0004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.180+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0043020 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.180+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c419b630 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.180+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd0b0004e50 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.180+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c41a7860 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.180+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00039b0 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.181+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c406a080 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.183+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd0b0046020 con 0x7fd0c410a560 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.185+0000 7fd0c2000640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0ac0a3d20 0x7fd0ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.185+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fd0b00f2560 con 0x7fd0c410a560 2024-04-20T11:33:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.185+0000 7fd0c9600640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0ac0a3d20 0x7fd0ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.186+0000 7fd0c9600640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0ac0a3d20 0x7fd0ac0a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80095f0 tx=0x7fd0b8005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.190+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd0b004d020 con 0x7fd0c410a560 2024-04-20T11:33:43.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.439+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd0c410c570 con 0x7fd0c410a560 2024-04-20T11:33:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.442+0000 7fd0c2000640 1 -- 172.21.15.44:0/2475687328 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7fd0b00becf0 con 0x7fd0c410a560 2024-04-20T11:33:43.442 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:33:43.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0ac0a3d20 msgr2=0x7fd0ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0ac0a3d20 0x7fd0ac0a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80095f0 tx=0x7fd0b8005f90 comp rx=0 tx=0).stop 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 msgr2=0x7fd0c41a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 0x7fd0c41a0be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009230 tx=0x7fd0b0004c30 comp rx=0 tx=0).stop 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 shutdown_connections 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0c419a810 0x7fd0c419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7fd0ac0a3d20 0x7fd0ac0a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0c410a560 0x7fd0c41a0be0 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009230 tx=0x7fd0b0004c30 comp rx=0 tx=0).stop 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 --2- 172.21.15.44:0/2475687328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0c41009a0 0x7fd0c41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 >> 172.21.15.44:0/2475687328 conn(0x7fd0c4076080 msgr2=0x7fd0c4108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.447+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 shutdown_connections 2024-04-20T11:33:43.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.448+0000 7fd0caa00640 1 -- 172.21.15.44:0/2475687328 wait complete. 2024-04-20T11:33:43.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:33:43.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 -- 172.21.15.44:0/394817733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 msgr2=0x7f42c01011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 --2- 172.21.15.44:0/394817733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c01011f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009be0 tx=0x7f42b002bf80 comp rx=0 tx=0).stop 2024-04-20T11:33:43.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 -- 172.21.15.44:0/394817733 shutdown_connections 2024-04-20T11:33:43.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 --2- 172.21.15.44:0/394817733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42c0106210 0x7f42c01066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 --2- 172.21.15.44:0/394817733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42c0101730 0x7f42c0105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 --2- 172.21.15.44:0/394817733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c01011f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 -- 172.21.15.44:0/394817733 >> 172.21.15.44:0/394817733 conn(0x7f42c00fc880 msgr2=0x7f42c00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:43.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.675+0000 7f42c6000640 1 -- 172.21.15.44:0/394817733 shutdown_connections 2024-04-20T11:33:43.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.676+0000 7f42c6000640 1 -- 172.21.15.44:0/394817733 wait complete. 2024-04-20T11:33:43.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.676+0000 7f42c6000640 1 Processor -- start 2024-04-20T11:33:43.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 -- start start 2024-04-20T11:33:43.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42c0101730 0x7f42c019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42c0106210 0x7f42c01a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0118900 con 0x7f42c0106210 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0118780 con 0x7f42c0100e10 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0118a80 con 0x7f42c0101730 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42c0106210 0x7f42c01a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.44:51192/0 (socket says 172.21.15.44:51192) 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42c4c00640 1 -- 172.21.15.44:0/3331455176 learned_addr learned my addr 172.21.15.44:0/3331455176 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.677+0000 7f42bfe00640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42c0101730 0x7f42c019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42c4c00640 1 -- 172.21.15.44:0/3331455176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42c0101730 msgr2=0x7f42c019f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42c4c00640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42c0101730 0x7f42c019f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42c4c00640 1 -- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42c0106210 msgr2=0x7f42c01a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:43.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42c4c00640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42c0106210 0x7f42c01a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:43.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42c4c00640 1 -- 172.21.15.44:0/3331455176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c01a3df0 con 0x7f42c0100e10 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42c4c00640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c019ee00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42b0000f80 tx=0x7f42b0033960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0043020 con 0x7f42c0100e10 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.678+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42b00333e0 con 0x7f42c0100e10 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.679+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c01a4080 con 0x7f42c0100e10 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.679+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b003e800 con 0x7f42c0100e10 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.679+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c01ab970 con 0x7f42c0100e10 2024-04-20T11:33:43.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.680+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c006ae60 con 0x7f42c0100e10 2024-04-20T11:33:43.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.682+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f42b0046020 con 0x7f42c0100e10 2024-04-20T11:33:43.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.683+0000 7f42bd600640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42a40a3da0 0x7f42a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:43.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.683+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8841+0+0 (secure 0 0 0) 0x7f42b00f22e0 con 0x7f42c0100e10 2024-04-20T11:33:43.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.683+0000 7f42bfe00640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42a40a3da0 0x7f42a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:43.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.684+0000 7f42bfe00640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42a40a3da0 0x7f42a40a6260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f42a80095f0 tx=0x7f42a8005f90 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:43.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.687+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f42b00be8f0 con 0x7f42c0100e10 2024-04-20T11:33:43.694 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j[56677]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j[56677]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j[56677]: starting mds.j at 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a[16850]: 2024-04-20T11:33:43.440+0000 7f2ef5e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: pgmap v346: 97 pgs: 97 active+clean; 139 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3555070991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2475687328' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:33:43.695 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/2475687328' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mon[16855]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d[57057]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d[57057]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d[57057]: starting mds.d at 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g[57660]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g[57660]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.696 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-mds[57688]: -- 172.21.15.44:0/4156755383 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606524e7a40 con 0x5606524bac00 2024-04-20T11:33:43.697 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:33:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g[57660]: starting mds.g at 2024-04-20T11:33:43.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:43.934+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f42c01abb40 con 0x7f42c0100e10 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mds[51420]: -- 172.21.15.78:0/4284832222 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603d4f85a40 con 0x5603d4f58c00 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: starting mds.b at 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: pgmap v346: 97 pgs: 97 active+clean; 139 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3555070991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2475687328' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T11:33:43.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/2475687328' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mon[21464]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e[50784]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e[50784]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mds[50788]: -- 172.21.15.78:0/1748436232 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587df8e7a40 con 0x5587df8bb000 2024-04-20T11:33:43.939 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e[50784]: starting mds.e at 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h[51107]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h[51107]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mds[51111]: -- 172.21.15.78:0/2160883547 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5993f9a40 con 0x55f5993cc800 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h[51107]: starting mds.h at 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k[50459]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k[50459]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-mds[50463]: -- 172.21.15.78:0/2769817121 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a82cc5a40 con 0x563a82c98c00 2024-04-20T11:33:43.940 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:33:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k[50459]: starting mds.k at 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: pgmap v346: 97 pgs: 97 active+clean; 139 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3555070991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2475687328' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/2475687328' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mon[21975]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-20T11:33:43.980 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c[52691]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c[52691]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mds[52695]: -- 172.21.15.165:0/4093056514 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617bcffda40 con 0x5617bcfd0800 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c[52691]: starting mds.c at 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f[52385]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f[52385]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mds[52389]: -- 172.21.15.165:0/625688617 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561da7f17a40 con 0x561da7eeac00 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f[52385]: starting mds.f at 2024-04-20T11:33:43.981 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i[52999]: ignoring --setuser ceph since I am not root 2024-04-20T11:33:43.982 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i[52999]: ignoring --setgroup ceph since I am not root 2024-04-20T11:33:43.982 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-mds[53003]: -- 172.21.15.165:0/3863130704 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1c316fa40 con 0x55c1c3143000 2024-04-20T11:33:43.982 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:33:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i[52999]: starting mds.i at 2024-04-20T11:33:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.451+0000 7f42bd600640 1 -- 172.21.15.44:0/3331455176 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7f42b00be1f0 con 0x7f42c0100e10 2024-04-20T11:33:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.456+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42a40a3da0 msgr2=0x7f42a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.456+0000 7f42c6000640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42a40a3da0 0x7f42a40a6260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f42a80095f0 tx=0x7f42a8005f90 comp rx=0 tx=0).stop 2024-04-20T11:33:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.456+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 msgr2=0x7f42c019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.456+0000 7f42c6000640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c019ee00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42b0000f80 tx=0x7f42b0033960 comp rx=0 tx=0).stop 2024-04-20T11:33:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.457+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 shutdown_connections 2024-04-20T11:33:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.457+0000 7f42c6000640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f42a40a3da0 0x7f42a40a6260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.457+0000 7f42c6000640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42c0106210 0x7f42c01a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.457+0000 7f42c6000640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42c0101730 0x7f42c019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.457+0000 7f42c6000640 1 --2- 172.21.15.44:0/3331455176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42c0100e10 0x7f42c019ee00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.457+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 >> 172.21.15.44:0/3331455176 conn(0x7f42c00fc880 msgr2=0x7f42c00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.458+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 shutdown_connections 2024-04-20T11:33:44.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.461+0000 7f42c6000640 1 -- 172.21.15.44:0/3331455176 wait complete. 2024-04-20T11:33:44.482 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.687+0000 7f2e71800640 1 -- 172.21.15.44:0/3831827343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c104b50 msgr2=0x7f2e6c108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.687+0000 7f2e71800640 1 --2- 172.21.15.44:0/3831827343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c104b50 0x7f2e6c108b90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009be0 tx=0x7f2e5802bef0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 -- 172.21.15.44:0/3831827343 shutdown_connections 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 --2- 172.21.15.44:0/3831827343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6c109270 0x7f2e6c110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 --2- 172.21.15.44:0/3831827343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c104b50 0x7f2e6c108b90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 --2- 172.21.15.44:0/3831827343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c1041a0 0x7f2e6c104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 -- 172.21.15.44:0/3831827343 >> 172.21.15.44:0/3831827343 conn(0x7f2e6c0ffa00 msgr2=0x7f2e6c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 -- 172.21.15.44:0/3831827343 shutdown_connections 2024-04-20T11:33:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.688+0000 7f2e71800640 1 -- 172.21.15.44:0/3831827343 wait complete. 2024-04-20T11:33:44.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.689+0000 7f2e71800640 1 Processor -- start 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.689+0000 7f2e71800640 1 -- start start 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e71800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6c1041a0 0x7f2e6c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e71800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 0x7f2e6c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e71800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c19a6f0 0x7f2e6c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e71800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c114470 con 0x7f2e6c109270 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e71800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1142f0 con 0x7f2e6c19a6f0 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e71800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1145f0 con 0x7f2e6c1041a0 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e6b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 0x7f2e6c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e6b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 0x7f2e6c1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54186/0 (socket says 172.21.15.44:54186) 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e6b400640 1 -- 172.21.15.44:0/1796235074 learned_addr learned my addr 172.21.15.44:0/1796235074 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e6be00640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6c1041a0 0x7f2e6c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.690+0000 7f2e70e00640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c19a6f0 0x7f2e6c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e6b400640 1 -- 172.21.15.44:0/1796235074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6c1041a0 msgr2=0x7f2e6c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e6b400640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6c1041a0 0x7f2e6c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e6b400640 1 -- 172.21.15.44:0/1796235074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c19a6f0 msgr2=0x7f2e6c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e6b400640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c19a6f0 0x7f2e6c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e6b400640 1 -- 172.21.15.44:0/1796235074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c19b2f0 con 0x7f2e6c109270 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e6b400640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 0x7f2e6c1a0b70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e58000f80 tx=0x7f2e58033720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58043020 con 0x7f2e6c109270 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c19b580 con 0x7f2e6c109270 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e58004040 con 0x7f2e6c109270 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5803e6b0 con 0x7f2e6c109270 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.691+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c1a7770 con 0x7f2e6c109270 2024-04-20T11:33:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.693+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c105670 con 0x7f2e6c109270 2024-04-20T11:33:44.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.695+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2e58046020 con 0x7f2e6c109270 2024-04-20T11:33:44.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.696+0000 7f2e68c00640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2e500a3d20 0x7f2e500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:44.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.697+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8841+0+0 (secure 0 0 0) 0x7f2e580f3610 con 0x7f2e6c109270 2024-04-20T11:33:44.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.697+0000 7f2e6be00640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2e500a3d20 0x7f2e500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:44.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.697+0000 7f2e6be00640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2e500a3d20 0x7f2e500a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009530 tx=0x7f2e5c009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:44.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.702+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2e5804d020 con 0x7f2e6c109270 2024-04-20T11:33:44.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:44.910+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e6c0690f0 con 0x7f2e6c109270 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/3331455176' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: Cluster is now healthy 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6834/3676043162,v1:172.21.15.165:6835/3676043162] up:boot 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6833/3928063191,v1:172.21.15.78:6835/3928063191] up:boot 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6832/479424210,v1:172.21.15.78:6834/479424210] up:boot 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6836/3696661849,v1:172.21.15.165:6838/3696661849] up:boot 2024-04-20T11:33:44.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.165:6837/3143003892,v1:172.21.15.165:6839/3143003892] up:boot 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6836/3866425129,v1:172.21.15.78:6837/3866425129] up:boot 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.78:6838/3949985554,v1:172.21.15.78:6839/3949985554] up:boot 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6834/3479787644,v1:172.21.15.44:6836/3479787644] up:boot 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6835/2937690174,v1:172.21.15.44:6837/2937690174] up:boot 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: mds.? [v2:172.21.15.44:6838/2231062925,v1:172.21.15.44:6839/2231062925] up:boot 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: fsmap 12 up:standby 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:33:44.939 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:33:44.940 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:33:44.940 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:44 smithi078 ceph-mon[21464]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/3331455176' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: Cluster is now healthy 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6834/3676043162,v1:172.21.15.165:6835/3676043162] up:boot 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6833/3928063191,v1:172.21.15.78:6835/3928063191] up:boot 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6832/479424210,v1:172.21.15.78:6834/479424210] up:boot 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6836/3696661849,v1:172.21.15.165:6838/3696661849] up:boot 2024-04-20T11:33:44.946 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.165:6837/3143003892,v1:172.21.15.165:6839/3143003892] up:boot 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6836/3866425129,v1:172.21.15.78:6837/3866425129] up:boot 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.78:6838/3949985554,v1:172.21.15.78:6839/3949985554] up:boot 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6834/3479787644,v1:172.21.15.44:6836/3479787644] up:boot 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6835/2937690174,v1:172.21.15.44:6837/2937690174] up:boot 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: mds.? [v2:172.21.15.44:6838/2231062925,v1:172.21.15.44:6839/2231062925] up:boot 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: fsmap 12 up:standby 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:33:44.947 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:33:44.948 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:33:44.948 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:33:44.948 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:33:44.948 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:44 smithi044 ceph-mon[16855]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:33:44.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/3331455176' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:33:44.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: Cluster is now healthy 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6834/3676043162,v1:172.21.15.165:6835/3676043162] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6833/3928063191,v1:172.21.15.78:6835/3928063191] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6832/479424210,v1:172.21.15.78:6834/479424210] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6836/3696661849,v1:172.21.15.165:6838/3696661849] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.165:6837/3143003892,v1:172.21.15.165:6839/3143003892] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6836/3866425129,v1:172.21.15.78:6837/3866425129] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.78:6838/3949985554,v1:172.21.15.78:6839/3949985554] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6834/3479787644,v1:172.21.15.44:6836/3479787644] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6835/2937690174,v1:172.21.15.44:6837/2937690174] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: mds.? [v2:172.21.15.44:6838/2231062925,v1:172.21.15.44:6839/2231062925] up:boot 2024-04-20T11:33:44.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: fsmap 12 up:standby 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T11:33:44.981 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:44 smithi165 ceph-mon[21975]: from='mgr.15174 172.21.15.44:0/3943784221' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T11:33:45.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.556+0000 7f2e68c00640 1 -- 172.21.15.44:0/1796235074 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f2e6c0690f0 con 0x7f2e6c109270 2024-04-20T11:33:45.557 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.561+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2e500a3d20 msgr2=0x7f2e500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.561+0000 7f2e71800640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2e500a3d20 0x7f2e500a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009530 tx=0x7f2e5c009260 comp rx=0 tx=0).stop 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.561+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 msgr2=0x7f2e6c1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.561+0000 7f2e71800640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 0x7f2e6c1a0b70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e58000f80 tx=0x7f2e58033720 comp rx=0 tx=0).stop 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 shutdown_connections 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e6c19a6f0 0x7f2e6c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7f2e500a3d20 0x7f2e500a61e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e6c109270 0x7f2e6c1a0b70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 --2- 172.21.15.44:0/1796235074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6c1041a0 0x7f2e6c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 >> 172.21.15.44:0/1796235074 conn(0x7f2e6c0ffa00 msgr2=0x7f2e6c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:45.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 shutdown_connections 2024-04-20T11:33:45.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.562+0000 7f2e71800640 1 -- 172.21.15.44:0/1796235074 wait complete. 2024-04-20T11:33:45.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 -- 172.21.15.44:0/1977483315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 msgr2=0x7ff78c110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 --2- 172.21.15.44:0/1977483315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c110be0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff784009be0 tx=0x7ff78402bef0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 -- 172.21.15.44:0/1977483315 shutdown_connections 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 --2- 172.21.15.44:0/1977483315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c110be0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 --2- 172.21.15.44:0/1977483315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 0x7ff78c10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 --2- 172.21.15.44:0/1977483315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 0x7ff78c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 -- 172.21.15.44:0/1977483315 >> 172.21.15.44:0/1977483315 conn(0x7ff78c0fc870 msgr2=0x7ff78c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 -- 172.21.15.44:0/1977483315 shutdown_connections 2024-04-20T11:33:45.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.798+0000 7ff791e00640 1 -- 172.21.15.44:0/1977483315 wait complete. 2024-04-20T11:33:45.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.799+0000 7ff791e00640 1 Processor -- start 2024-04-20T11:33:45.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.799+0000 7ff791e00640 1 -- start start 2024-04-20T11:33:45.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 0x7ff78c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 0x7ff78c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c1145d0 con 0x7ff78c10e7f0 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c114450 con 0x7ff78c1015f0 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c114750 con 0x7ff78c100c40 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54210/0 (socket says 172.21.15.44:54210) 2024-04-20T11:33:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff791400640 1 -- 172.21.15.44:0/1739084156 learned_addr learned my addr 172.21.15.44:0/1739084156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff790a00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 0x7ff78c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.800+0000 7ff783e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 0x7ff78c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff790a00640 1 -- 172.21.15.44:0/1739084156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 msgr2=0x7ff78c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff790a00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 0x7ff78c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff790a00640 1 -- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 msgr2=0x7ff78c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff790a00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff790a00640 1 -- 172.21.15.44:0/1739084156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c19b030 con 0x7ff78c100c40 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff791400640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff783e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 0x7ff78c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff790a00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 0x7ff78c1a06b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00e9f0 tx=0x7ff77c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00ccc0 con 0x7ff78c100c40 2024-04-20T11:33:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c19b320 con 0x7ff78c100c40 2024-04-20T11:33:45.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff77c005020 con 0x7ff78c100c40 2024-04-20T11:33:45.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c004b40 con 0x7ff78c100c40 2024-04-20T11:33:45.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.801+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c1a7570 con 0x7ff78c100c40 2024-04-20T11:33:45.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.803+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c06ae40 con 0x7ff78c100c40 2024-04-20T11:33:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.805+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff77c004ce0 con 0x7ff78c100c40 2024-04-20T11:33:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.806+0000 7ff78a000640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7700a3d20 0x7ff7700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:33:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.806+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8345+0+0 (secure 0 0 0) 0x7ff77c0d0730 con 0x7ff78c100c40 2024-04-20T11:33:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.806+0000 7ff783e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7700a3d20 0x7ff7700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:33:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.807+0000 7ff783e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7700a3d20 0x7ff7700a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7740096c0 tx=0x7ff774009260 comp rx=0 tx=0).ready entity=mgr.15174 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:33:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:45.811+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff77c09ce20 con 0x7ff78c100c40 2024-04-20T11:33:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:45 smithi078 ceph-mon[21464]: pgmap v348: 97 pgs: 97 active+clean; 139 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-20T11:33:45.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:45 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1796235074' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:45.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:45 smithi044 ceph-mon[16855]: pgmap v348: 97 pgs: 97 active+clean; 139 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-20T11:33:45.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:45 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1796235074' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:45 smithi165 ceph-mon[21975]: pgmap v348: 97 pgs: 97 active+clean; 139 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-20T11:33:45.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:45 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1796235074' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:46.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.017+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff78c0690f0 con 0x7ff78c100c40 2024-04-20T11:33:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.572+0000 7ff78a000640 1 -- 172.21.15.44:0/1739084156 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7ff77c09c720 con 0x7ff78c100c40 2024-04-20T11:33:46.576 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-04-20T11:33:46.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7700a3d20 msgr2=0x7ff7700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7700a3d20 0x7ff7700a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7740096c0 tx=0x7ff774009260 comp rx=0 tx=0).stop 2024-04-20T11:33:46.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 msgr2=0x7ff78c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:33:46.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 0x7ff78c1a06b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00e9f0 tx=0x7ff77c00eeb0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 shutdown_connections 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:6800/4029191467,v1:172.21.15.44:6801/4029191467] conn(0x7ff7700a3d20 0x7ff7700a61e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff78c10e7f0 0x7ff78c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff78c1015f0 0x7ff78c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 --2- 172.21.15.44:0/1739084156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff78c100c40 0x7ff78c1a06b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.581+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 >> 172.21.15.44:0/1739084156 conn(0x7ff78c0fc870 msgr2=0x7ff78c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.582+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 shutdown_connections 2024-04-20T11:33:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T11:33:46.582+0000 7ff791e00640 1 -- 172.21.15.44:0/1739084156 wait complete. 2024-04-20T11:33:46.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:33:46.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:33:46.711 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:33:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:46 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1796235074' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T11:33:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:46 smithi165 ceph-mon[21975]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T11:33:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:46 smithi165 ceph-mon[21975]: from='client.? 172.21.15.44:0/1739084156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:46.729 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:46 smithi165 ceph-mon[21975]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:46.739 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-20T11:33:46.739 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T11:33:46.799 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T11:33:46.816 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T11:33:46.831 INFO:tasks.cephadm:Stopping all daemons... 2024-04-20T11:33:46.831 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-20T11:33:46.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a 2024-04-20T11:33:46.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:46 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1796235074' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T11:33:46.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:46 smithi044 ceph-mon[16855]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T11:33:46.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:46 smithi044 ceph-mon[16855]: from='client.? 172.21.15.44:0/1739084156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:46.911 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:46 smithi044 ceph-mon[16855]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:46.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:46 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1796235074' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T11:33:46.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:46 smithi078 ceph-mon[21464]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T11:33:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:46 smithi078 ceph-mon[21464]: from='client.? 172.21.15.44:0/1739084156' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:46.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:46 smithi078 ceph-mon[21464]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T11:33:47.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:46 smithi044 systemd[1]: Stopping Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:47.944 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a[16850]: 2024-04-20T11:33:47.462+0000 7f2efcc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:33:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a[16850]: 2024-04-20T11:33:47.462+0000 7f2efcc00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-20T11:33:47.945 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:47 smithi044 podman[85669]: 2024-04-20 11:33:47.617010788 +0000 UTC m=+0.328571599 container died fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T11:33:48.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:48 smithi044 podman[85669]: 2024-04-20 11:33:48.108948712 +0000 UTC m=+0.820509561 container cleanup fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T11:33:48.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:48 smithi044 bash[85669]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a 2024-04-20T11:33:48.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service' 2024-04-20T11:33:48.795 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:48 smithi044 podman[85680]: 2024-04-20 11:33:48.473446797 +0000 UTC m=+0.834776925 container remove fc8cefb00f96742c439746239cf78f5e10abac8a4c28661b9fbd07168cdfca40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-a, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:33:48.795 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:48 smithi044 systemd[1]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-20T11:33:48.795 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:48 smithi044 systemd[1]: Stopped Ceph mon.a for 759aac0c-ff06-11ee-bc92-c7b262605968. 2024-04-20T11:33:48.795 INFO:journalctl@ceph.mon.a.smithi044.stdout:Apr 20 11:33:48 smithi044 systemd[1]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.a.service: Consumed 43.959s CPU time. 2024-04-20T11:33:48.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:33:48.865 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-20T11:33:48.865 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-20T11:33:48.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.b 2024-04-20T11:33:49.186 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:48 smithi078 systemd[1]: Stopping Ceph mon.b for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:49.437 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-b[21460]: 2024-04-20T11:33:49.186+0000 7f1b5cd47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:33:49.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-b[21460]: 2024-04-20T11:33:49.186+0000 7f1b5cd47700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-20T11:33:49.438 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:49 smithi078 podman[57069]: 2024-04-20 11:33:49.360526343 +0000 UTC m=+0.301417352 container died b8f80487171f94dd18fa186d04c68465a06f3806f99cb15e75e20a5275b01d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-20T11:33:49.937 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:49 smithi078 podman[57069]: 2024-04-20 11:33:49.748400958 +0000 UTC m=+0.689291961 container cleanup b8f80487171f94dd18fa186d04c68465a06f3806f99cb15e75e20a5275b01d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-20T11:33:49.938 INFO:journalctl@ceph.mon.b.smithi078.stdout:Apr 20 11:33:49 smithi078 bash[57069]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-b 2024-04-20T11:33:50.175 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.b.service' 2024-04-20T11:33:50.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:33:50.215 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-20T11:33:50.216 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-20T11:33:50.216 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.c 2024-04-20T11:33:50.527 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:50 smithi165 systemd[1]: Stopping Ceph mon.c for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:50.979 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-c[21956]: 2024-04-20T11:33:50.526+0000 7f6ff44d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:33:50.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-c[21956]: 2024-04-20T11:33:50.526+0000 7f6ff44d7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-20T11:33:50.980 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:50 smithi165 podman[58507]: 2024-04-20 11:33:50.721647045 +0000 UTC m=+0.312592154 container died dad7540733ebcbe57ee600fe9d1456083a9fa6a49425b19e7870127e9b286ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:33:51.362 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:51 smithi165 podman[58507]: 2024-04-20 11:33:51.111294988 +0000 UTC m=+0.702240096 container cleanup dad7540733ebcbe57ee600fe9d1456083a9fa6a49425b19e7870127e9b286ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-20T11:33:51.362 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 20 11:33:51 smithi165 bash[58507]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mon-c 2024-04-20T11:33:51.541 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mon.c.service' 2024-04-20T11:33:51.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:33:51.605 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-20T11:33:51.605 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-20T11:33:51.605 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x 2024-04-20T11:33:51.945 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:51 smithi044 systemd[1]: Stopping Ceph mgr.x for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:52.695 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:33:52.293+0000 7f624a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:33:52.696 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x[17092]: 2024-04-20T11:33:52.293+0000 7f624a200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-20T11:33:52.696 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:52 smithi044 podman[85784]: 2024-04-20 11:33:52.397704712 +0000 UTC m=+0.291258805 container died afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T11:33:53.123 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:52 smithi044 podman[85784]: 2024-04-20 11:33:52.944012463 +0000 UTC m=+0.837566557 container cleanup afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T11:33:53.123 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:52 smithi044 bash[85784]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x 2024-04-20T11:33:53.444 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Apr 20 11:33:53 smithi044 podman[85798]: 2024-04-20 11:33:53.187871129 +0000 UTC m=+0.779742375 container remove afe2669f9fac28b550f8aa019f561fb95c994de8392faa5bc9dbdcd3ab7a34cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-20T11:33:53.651 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.x.service' 2024-04-20T11:33:53.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:33:53.756 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-20T11:33:53.757 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-20T11:33:53.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.y 2024-04-20T11:33:54.106 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:33:53 smithi078 systemd[1]: Stopping Ceph mgr.y for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:54.428 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:33:54 smithi078 podman[57190]: 2024-04-20 11:33:54.106932026 +0000 UTC m=+0.151309300 container died aebacfb96992f70fa229e94c9f357c26b6e60549ea19021e1b48e67bdfeb6032 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:33:54.687 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:33:54 smithi078 podman[57190]: 2024-04-20 11:33:54.528803172 +0000 UTC m=+0.573180443 container cleanup aebacfb96992f70fa229e94c9f357c26b6e60549ea19021e1b48e67bdfeb6032 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T11:33:54.688 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Apr 20 11:33:54 smithi078 bash[57190]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-y 2024-04-20T11:33:54.987 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.y.service' 2024-04-20T11:33:55.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:33:55.020 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-20T11:33:55.020 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-20T11:33:55.020 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.z 2024-04-20T11:33:55.335 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:33:55 smithi165 systemd[1]: Stopping Ceph mgr.z for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:55.666 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:33:55 smithi165 podman[58629]: 2024-04-20 11:33:55.336179026 +0000 UTC m=+0.125553732 container died caa509b16bb3a51174d99a3ea16b6055fd4dd96dacee58349c965ed115f5fba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-20T11:33:55.979 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:33:55 smithi165 podman[58629]: 2024-04-20 11:33:55.757695729 +0000 UTC m=+0.547070440 container cleanup caa509b16bb3a51174d99a3ea16b6055fd4dd96dacee58349c965ed115f5fba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-20T11:33:55.979 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 20 11:33:55 smithi165 bash[58629]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mgr-z 2024-04-20T11:33:56.179 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mgr.z.service' 2024-04-20T11:33:56.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:33:56.212 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-20T11:33:56.212 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-20T11:33:56.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.0 2024-04-20T11:33:56.695 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:33:56 smithi044 systemd[1]: Stopping Ceph osd.0 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:33:57.195 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:33:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0[31822]: 2024-04-20T11:33:56.841+0000 7fcee0400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:33:57.195 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:33:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0[31822]: 2024-04-20T11:33:56.841+0000 7fcee0400700 -1 osd.0 101 *** Got signal Terminated *** 2024-04-20T11:33:57.195 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:33:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0[31822]: 2024-04-20T11:33:56.841+0000 7fcee0400700 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:02.695 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:02 smithi044 podman[85906]: 2024-04-20 11:34:02.344213605 +0000 UTC m=+5.703674628 container died 0672bb214e03cef352e3f56acdd9f713ee738f5345b5b02971f6f4bda3db42d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T11:34:03.118 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:02 smithi044 podman[85906]: 2024-04-20 11:34:02.850189225 +0000 UTC m=+6.209650286 container cleanup 0672bb214e03cef352e3f56acdd9f713ee738f5345b5b02971f6f4bda3db42d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:34:03.118 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:02 smithi044 bash[85906]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0 2024-04-20T11:34:03.444 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:03 smithi044 podman[85924]: 2024-04-20 11:34:03.118185083 +0000 UTC m=+0.757860892 container remove 0672bb214e03cef352e3f56acdd9f713ee738f5345b5b02971f6f4bda3db42d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:34:04.343 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:03 smithi044 podman[86017]: 2024-04-20 11:34:03.877179915 +0000 UTC m=+0.039410294 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:04.343 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:04 smithi044 podman[86017]: 2024-04-20 11:34:04.028016953 +0000 UTC m=+0.190247345 container create adcc1fcf7eab6c2738d92cf7b5a1919f0ea7395156c8db5779892d8bad5d0579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T11:34:04.695 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:04 smithi044 podman[86017]: 2024-04-20 11:34:04.343849977 +0000 UTC m=+0.506080409 container init adcc1fcf7eab6c2738d92cf7b5a1919f0ea7395156c8db5779892d8bad5d0579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T11:34:04.696 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:04 smithi044 podman[86017]: 2024-04-20 11:34:04.355292043 +0000 UTC m=+0.517522441 container start adcc1fcf7eab6c2738d92cf7b5a1919f0ea7395156c8db5779892d8bad5d0579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:04.696 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:04 smithi044 podman[86017]: 2024-04-20 11:34:04.447505193 +0000 UTC m=+0.609735592 container attach adcc1fcf7eab6c2738d92cf7b5a1919f0ea7395156c8db5779892d8bad5d0579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-20T11:34:05.189 INFO:journalctl@ceph.osd.0.smithi044.stdout:Apr 20 11:34:04 smithi044 podman[86017]: 2024-04-20 11:34:04.818969488 +0000 UTC m=+0.981199926 container died adcc1fcf7eab6c2738d92cf7b5a1919f0ea7395156c8db5779892d8bad5d0579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T11:34:05.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.0.service' 2024-04-20T11:34:05.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:05.769 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-20T11:34:05.770 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-20T11:34:05.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.1 2024-04-20T11:34:06.083 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:05 smithi078 systemd[1]: Stopping Ceph osd.1 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:06.437 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1[26080]: 2024-04-20T11:34:06.082+0000 7ff8b88ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:06.437 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1[26080]: 2024-04-20T11:34:06.082+0000 7ff8b88ce700 -1 osd.1 101 *** Got signal Terminated *** 2024-04-20T11:34:06.438 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1[26080]: 2024-04-20T11:34:06.082+0000 7ff8b88ce700 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:11.749 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:11 smithi078 podman[57316]: 2024-04-20 11:34:11.447104991 +0000 UTC m=+5.486114564 container died 611ceb3023b341e9325ddac56fe70cbe27c109a23c4ad034a9138c066cc8ba13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-20T11:34:12.084 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:11 smithi078 podman[57316]: 2024-04-20 11:34:11.861640691 +0000 UTC m=+5.900650264 container cleanup 611ceb3023b341e9325ddac56fe70cbe27c109a23c4ad034a9138c066cc8ba13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T11:34:12.084 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:11 smithi078 bash[57316]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1 2024-04-20T11:34:12.437 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:12 smithi078 podman[57333]: 2024-04-20 11:34:12.084879975 +0000 UTC m=+0.638515142 container remove 611ceb3023b341e9325ddac56fe70cbe27c109a23c4ad034a9138c066cc8ba13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:12.804 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:12 smithi078 podman[57423]: 2024-04-20 11:34:12.431359724 +0000 UTC m=+0.017628461 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:12.804 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:12 smithi078 podman[57423]: 2024-04-20 11:34:12.552527593 +0000 UTC m=+0.138796368 container create 418d8e5cda866d05f3bfb2ca2c059d2ce4643892e6e74d9558272a9a4c2f16da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-20T11:34:12.804 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:12 smithi078 podman[57423]: 2024-04-20 11:34:12.805080086 +0000 UTC m=+0.391348820 container init 418d8e5cda866d05f3bfb2ca2c059d2ce4643892e6e74d9558272a9a4c2f16da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:34:13.101 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:12 smithi078 podman[57423]: 2024-04-20 11:34:12.808318022 +0000 UTC m=+0.394586753 container start 418d8e5cda866d05f3bfb2ca2c059d2ce4643892e6e74d9558272a9a4c2f16da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:13.101 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:12 smithi078 podman[57423]: 2024-04-20 11:34:12.913601373 +0000 UTC m=+0.499870121 container attach 418d8e5cda866d05f3bfb2ca2c059d2ce4643892e6e74d9558272a9a4c2f16da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:34:13.370 INFO:journalctl@ceph.osd.1.smithi078.stdout:Apr 20 11:34:13 smithi078 podman[57423]: 2024-04-20 11:34:13.102370357 +0000 UTC m=+0.688639100 container died 418d8e5cda866d05f3bfb2ca2c059d2ce4643892e6e74d9558272a9a4c2f16da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-20T11:34:13.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.1.service' 2024-04-20T11:34:13.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:13.920 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-20T11:34:13.921 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-20T11:34:13.921 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.2 2024-04-20T11:34:14.229 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:13 smithi165 systemd[1]: Stopping Ceph osd.2 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:14.494 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2[27116]: 2024-04-20T11:34:14.235+0000 7feb8b5ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:14.494 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2[27116]: 2024-04-20T11:34:14.235+0000 7feb8b5ae700 -1 osd.2 101 *** Got signal Terminated *** 2024-04-20T11:34:14.494 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2[27116]: 2024-04-20T11:34:14.235+0000 7feb8b5ae700 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:19.945 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:19 smithi165 podman[58750]: 2024-04-20 11:34:19.654360411 +0000 UTC m=+5.537784989 container died 99b5f688a9320b5c17d6abd5a29f1ce2fa5080cb39ed37291762343c2770adac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:34:20.229 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 podman[58750]: 2024-04-20 11:34:20.035846693 +0000 UTC m=+5.919271271 container cleanup 99b5f688a9320b5c17d6abd5a29f1ce2fa5080cb39ed37291762343c2770adac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T11:34:20.229 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 bash[58750]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2 2024-04-20T11:34:20.606 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 podman[58780]: 2024-04-20 11:34:20.303963015 +0000 UTC m=+0.646521136 container remove 99b5f688a9320b5c17d6abd5a29f1ce2fa5080cb39ed37291762343c2770adac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416) 2024-04-20T11:34:20.915 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 podman[58870]: 2024-04-20 11:34:20.510297107 +0000 UTC m=+0.018090609 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:20.915 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 podman[58870]: 2024-04-20 11:34:20.629849983 +0000 UTC m=+0.137643475 container create 75c534e8397e1638ce50bdfd74db10d802fe5674ea809781983e90e12472fdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:21.209 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 podman[58870]: 2024-04-20 11:34:20.915835108 +0000 UTC m=+0.423628601 container init 75c534e8397e1638ce50bdfd74db10d802fe5674ea809781983e90e12472fdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T11:34:21.209 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:20 smithi165 podman[58870]: 2024-04-20 11:34:20.91930397 +0000 UTC m=+0.427097460 container start 75c534e8397e1638ce50bdfd74db10d802fe5674ea809781983e90e12472fdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T11:34:21.209 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:21 smithi165 podman[58870]: 2024-04-20 11:34:21.016272003 +0000 UTC m=+0.524065495 container attach 75c534e8397e1638ce50bdfd74db10d802fe5674ea809781983e90e12472fdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T11:34:21.479 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 20 11:34:21 smithi165 podman[58870]: 2024-04-20 11:34:21.210907593 +0000 UTC m=+0.718701097 container died 75c534e8397e1638ce50bdfd74db10d802fe5674ea809781983e90e12472fdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-20T11:34:21.795 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.2.service' 2024-04-20T11:34:21.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:21.833 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-20T11:34:21.833 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-20T11:34:21.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.3 2024-04-20T11:34:22.195 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:21 smithi044 systemd[1]: Stopping Ceph osd.3 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:22.945 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:22 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:22.511+0000 7f3eaae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:22.945 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:22 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:22.511+0000 7f3eaae00700 -1 osd.3 101 *** Got signal Terminated *** 2024-04-20T11:34:22.945 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:22 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:22.511+0000 7f3eaae00700 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:23.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:22.944+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:23.445 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:23 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:23.258+0000 7f3ea6600700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.687596+0000 front 2024-04-20T11:34:00.688403+0000 (oldest deadline 2024-04-20T11:34:22.987708+0000) 2024-04-20T11:34:23.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:23 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:23.027+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:24.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:23.905+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:24.445 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:24 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:24.296+0000 7f3ea6600700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.687596+0000 front 2024-04-20T11:34:00.688403+0000 (oldest deadline 2024-04-20T11:34:22.987708+0000) 2024-04-20T11:34:24.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:24 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:24.051+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:25.187 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:24.809+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:25.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:24.921+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:25.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:24.836+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:25.445 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:25 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:25.276+0000 7f3ea6600700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.687596+0000 front 2024-04-20T11:34:00.688403+0000 (oldest deadline 2024-04-20T11:34:22.987708+0000) 2024-04-20T11:34:25.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:25 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:25.076+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:26.187 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:25.762+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:26.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:25.820+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:26.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:25.873+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:26.445 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:26.281+0000 7f3ea6600700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.687596+0000 front 2024-04-20T11:34:00.688403+0000 (oldest deadline 2024-04-20T11:34:22.987708+0000) 2024-04-20T11:34:26.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:26.054+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:27.075 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:26 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:26.798+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:27.187 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:26.766+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:27.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:26.835+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:27.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:26.860+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:27.444 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3[38453]: 2024-04-20T11:34:27.284+0000 7f3ea6600700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.687596+0000 front 2024-04-20T11:34:00.688403+0000 (oldest deadline 2024-04-20T11:34:22.987708+0000) 2024-04-20T11:34:27.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:27.074+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:28.062 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:27 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:27.778+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:28.187 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:27.747+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:28.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:27.850+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:28.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:27.844+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:28.428 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:28 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:28.061+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:28.429 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:28 smithi044 podman[86135]: 2024-04-20 11:34:28.123348854 +0000 UTC m=+5.759847756 container died 7e9087503ce68099fe0e393c00a0ccef9a5c493391c6259da9efa891b9f067f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:34:28.697 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:28 smithi044 podman[86135]: 2024-04-20 11:34:28.600340781 +0000 UTC m=+6.236839676 container cleanup 7e9087503ce68099fe0e393c00a0ccef9a5c493391c6259da9efa891b9f067f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-20T11:34:28.697 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:28 smithi044 bash[86135]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3 2024-04-20T11:34:28.964 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:28 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:28.810+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:29.187 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:28 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:28.796+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:29.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:28.844+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:29.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:28.847+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:29.444 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:28 smithi044 podman[86154]: 2024-04-20 11:34:28.96500497 +0000 UTC m=+0.833544092 container remove 7e9087503ce68099fe0e393c00a0ccef9a5c493391c6259da9efa891b9f067f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T11:34:29.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:29 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:29.013+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:29.856 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:29 smithi044 podman[86242]: 2024-04-20 11:34:29.462645721 +0000 UTC m=+0.033340789 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:29.856 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:29 smithi044 podman[86242]: 2024-04-20 11:34:29.583199069 +0000 UTC m=+0.153894177 container create 5f4941c320dc822358f3839c3496f316b095d6aee0f213ea9e6ddec0031d28ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-20T11:34:30.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:29 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:29.821+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:30.194 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:30 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:30.057+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:30.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:29 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:29.854+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:30.195 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:29 smithi044 podman[86242]: 2024-04-20 11:34:29.861892789 +0000 UTC m=+0.432587901 container init 5f4941c320dc822358f3839c3496f316b095d6aee0f213ea9e6ddec0031d28ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:34:30.195 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:29 smithi044 podman[86242]: 2024-04-20 11:34:29.873709943 +0000 UTC m=+0.444405055 container start 5f4941c320dc822358f3839c3496f316b095d6aee0f213ea9e6ddec0031d28ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5) 2024-04-20T11:34:30.195 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:30 smithi044 podman[86242]: 2024-04-20 11:34:30.010837076 +0000 UTC m=+0.581532238 container attach 5f4941c320dc822358f3839c3496f316b095d6aee0f213ea9e6ddec0031d28ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T11:34:30.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:29.845+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:30.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:29.878+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:30.668 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 20 11:34:30 smithi044 podman[86242]: 2024-04-20 11:34:30.344814913 +0000 UTC m=+0.915509981 container died 5f4941c320dc822358f3839c3496f316b095d6aee0f213ea9e6ddec0031d28ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:34:30.945 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:30 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:30.900+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:31.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.3.service' 2024-04-20T11:34:31.187 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:30 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:30.805+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:31.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:31.222 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-20T11:34:31.222 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-20T11:34:31.222 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.4 2024-04-20T11:34:31.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:30.905+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:31.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:30.846+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:31.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:31 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:31.014+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:31.553 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:31 smithi078 systemd[1]: Stopping Ceph osd.4 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:31.937 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:31.793+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:31.938 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:31.553+0000 7f3830a61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:31.938 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:31.553+0000 7f3830a61700 -1 osd.4 101 *** Got signal Terminated *** 2024-04-20T11:34:31.938 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:31 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:31.553+0000 7f3830a61700 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:32.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:31 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:31.887+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:32.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:31 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:31.986+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:32.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:31.919+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:32.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:31.852+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:32.437 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:32.037+0000 7f382904d700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.053495+0000 front 2024-04-20T11:34:10.053522+0000 (oldest deadline 2024-04-20T11:34:31.753619+0000) 2024-04-20T11:34:33.187 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:33 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:33.031+0000 7f382904d700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.053495+0000 front 2024-04-20T11:34:10.053522+0000 (oldest deadline 2024-04-20T11:34:31.753619+0000) 2024-04-20T11:34:33.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:32.987+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:33.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:32.828+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:33.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:32 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:32.828+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:33.194 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:32 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:32.945+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:33.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:32 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:32.844+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:33.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:32 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:32.844+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:33.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:32.921+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:33.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:32.854+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:33.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:32.854+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:33.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:32.806+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:34.187 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:34 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:34.005+0000 7f382904d700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.053495+0000 front 2024-04-20T11:34:10.053522+0000 (oldest deadline 2024-04-20T11:34:31.753619+0000) 2024-04-20T11:34:34.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:34 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:34.018+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:34.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:33 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:33.788+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:34.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:33 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:33.788+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:34.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:33 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:33.957+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:34.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:33 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:33.957+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:34.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:33 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:33.828+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:34.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:33 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:33.828+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:34.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:33.796+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:34.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:33.944+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:34.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:33.855+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:34.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:33.855+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:35.188 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:35.000+0000 7f382904d700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.053495+0000 front 2024-04-20T11:34:10.053522+0000 (oldest deadline 2024-04-20T11:34:31.753619+0000) 2024-04-20T11:34:35.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:35.027+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:35.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:34 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:34.791+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:35.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:34 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:34.791+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:35.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:34.974+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:35.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:34.974+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:35.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:34.864+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:35.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:34 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:34.864+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:35.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:34.964+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:35.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:34.872+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:35.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:34.872+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:35.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:34.838+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:36.188 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4[32580]: 2024-04-20T11:34:35.986+0000 7f382904d700 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.053495+0000 front 2024-04-20T11:34:10.053522+0000 (oldest deadline 2024-04-20T11:34:31.753619+0000) 2024-04-20T11:34:36.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:35.994+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:36.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:35.765+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:36.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:35 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:35.765+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:36.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:36.013+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:36.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:36.013+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:36.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:35 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:35.909+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:36.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:35 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:35.909+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:36.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:35 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:35.917+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:36.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:35 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:35.867+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:36.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:35 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:35.899+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:36.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:35 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:35.899+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:37.022 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:36 smithi078 podman[57544]: 2024-04-20 11:34:36.887918303 +0000 UTC m=+5.464457493 container died 3acb431990db08d58d381f1bf3f1f5e9a7757bada73e38c772e21d1e0da019dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=) 2024-04-20T11:34:37.022 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:36 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:36.728+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:37.022 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:36 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:36.728+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:37.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:37.045+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:37.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:37.045+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:37.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:36.917+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:37.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:36 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:36.917+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:37.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:36.849+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:37.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:36.897+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:37.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:36.897+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:37.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:36.873+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:37.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:36 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:36.873+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:37.274 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:37 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:37.021+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:37.274 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:37 smithi078 podman[57544]: 2024-04-20 11:34:37.262341248 +0000 UTC m=+5.838880437 container cleanup 3acb431990db08d58d381f1bf3f1f5e9a7757bada73e38c772e21d1e0da019dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T11:34:37.274 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:37 smithi078 bash[57544]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4 2024-04-20T11:34:37.866 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:37 smithi078 podman[57561]: 2024-04-20 11:34:37.513546932 +0000 UTC m=+0.623206338 container remove 3acb431990db08d58d381f1bf3f1f5e9a7757bada73e38c772e21d1e0da019dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:37.867 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:37 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:37.752+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:37.867 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:37 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:37.752+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:38.150 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:37 smithi078 podman[57652]: 2024-04-20 11:34:37.770691248 +0000 UTC m=+0.018029811 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:38.150 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:37 smithi078 podman[57652]: 2024-04-20 11:34:37.899302666 +0000 UTC m=+0.146641263 container create 2e75433f7afffd284eec4e15af3e582d71e699e3c9950cc06318576847b2d335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T11:34:38.150 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:38 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:38.059+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:38.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:37 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:37.812+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:38.229 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:37 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:37.863+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:38.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:37 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:37.863+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:38.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:37 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:37.945+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:38.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:37 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:37.945+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:38.438 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:38 smithi078 podman[57652]: 2024-04-20 11:34:38.150880642 +0000 UTC m=+0.398219196 container init 2e75433f7afffd284eec4e15af3e582d71e699e3c9950cc06318576847b2d335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-20T11:34:38.438 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:38 smithi078 podman[57652]: 2024-04-20 11:34:38.154171268 +0000 UTC m=+0.401509824 container start 2e75433f7afffd284eec4e15af3e582d71e699e3c9950cc06318576847b2d335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:34:38.438 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:38 smithi078 podman[57652]: 2024-04-20 11:34:38.234223919 +0000 UTC m=+0.481562493 container attach 2e75433f7afffd284eec4e15af3e582d71e699e3c9950cc06318576847b2d335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:38.444 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:38 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:38.094+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:38.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:38 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:38.094+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:38.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:37.947+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:38.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:37 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:37.947+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:38.752 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 20 11:34:38 smithi078 podman[57652]: 2024-04-20 11:34:38.446890654 +0000 UTC m=+0.694229209 container died 2e75433f7afffd284eec4e15af3e582d71e699e3c9950cc06318576847b2d335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True) 2024-04-20T11:34:39.070 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:38 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:38.752+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:39.070 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:38 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:38.752+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:39.092 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.4.service' 2024-04-20T11:34:39.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:39.136 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-20T11:34:39.136 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-20T11:34:39.136 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.5 2024-04-20T11:34:39.162 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:38 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:38.790+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:39.162 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:38 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:38.931+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:39.162 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:38 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:38.931+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:39.162 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:38 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:38.882+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:39.162 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:38 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:38.882+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:39.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:39 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:39.109+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:39.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:38 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:38.968+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:39.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:38 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:38.968+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:39.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:39 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:39.108+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:39.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:39 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:39.108+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:39.445 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:39 smithi165 systemd[1]: Stopping Ceph osd.5 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:39.729 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:39.445+0000 7f3fa6251700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:39.729 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:39.445+0000 7f3fa6251700 -1 osd.5 101 *** Got signal Terminated *** 2024-04-20T11:34:39.729 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:39.445+0000 7f3fa6251700 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:40.148 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:39 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:39.754+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:40.148 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:39 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:39.754+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:40.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:39.781+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:40.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:39.913+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:40.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:39.913+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:40.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:39.911+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:40.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:39 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:39.911+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:40.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:40 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:40.148+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:40.444 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:40.141+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:40.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:40.141+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:40.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:40.007+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:40.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:40 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:40.007+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:41.164 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:40 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:40.798+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:41.164 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:40 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:40.798+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:41.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:40 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:40.799+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:41.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:40 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:40.944+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:41.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:40 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:40.944+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:41.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:40 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:40.944+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:41.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:40 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:40.864+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:41.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:40 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:40.864+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:41.430 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:41.164+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:41.430 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:41.164+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:41.430 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:41.024+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:41.430 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:41 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:41.024+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:41.432 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:41 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:41.165+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:42.116 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:41 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:41.807+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:42.116 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:41 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:41.807+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:42.116 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:41 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:41.807+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:42.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:41 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:41.753+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:42.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:41 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:41.955+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:42.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:41 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:41.955+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:42.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:41 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:41.955+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:42.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:41 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:41.856+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:42.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:41 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:41.856+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:42.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:42 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:42.116+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:42.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:42 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:42.116+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:42.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:42 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:42.119+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:42.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:42 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:42.119+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:42.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:42 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:42.065+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:42.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:42 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:42.065+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:42.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:42 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:42.065+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:42.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:42.711+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:42.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:42.711+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:42.980 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:42.862+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:42.980 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:42.862+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:42.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:42.923+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:42.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:42.923+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:42.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:42 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:42.923+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:43.083 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:42 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:42.816+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:43.083 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:42 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:42.816+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:43.083 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:42 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:42.816+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:43.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:43.084+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:43.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:43.084+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:43.444 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:43.145+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:43.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:43.145+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:43.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:43.070+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:43.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:43.070+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:43.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:43 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:43.070+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:44.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:44 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:44.089+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:44.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:44 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:44.089+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:44.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:43.852+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:44.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:43.852+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:44.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:43 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:43.852+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:44.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:43.916+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:44.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:43.916+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:44.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:43.916+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:44.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:43.903+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.498609+0000 front 2024-04-20T11:33:59.498305+0000 (oldest deadline 2024-04-20T11:34:24.798167+0000) 2024-04-20T11:34:44.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:43.903+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:06.999826+0000 front 2024-04-20T11:34:06.999686+0000 (oldest deadline 2024-04-20T11:34:32.299758+0000) 2024-04-20T11:34:44.230 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5[33628]: 2024-04-20T11:34:43.903+0000 7f3fa2845700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.200686+0000 front 2024-04-20T11:34:18.200705+0000 (oldest deadline 2024-04-20T11:34:43.500892+0000) 2024-04-20T11:34:44.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:43.737+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:44.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:43 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:43.737+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:44.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:44.156+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:44.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:44.156+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:44.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:44.091+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:44.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:44.091+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:44.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:44.091+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:44.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:44 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:44.697+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:44.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:44 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:44.697+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:44.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:44 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:44.895+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:44.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:44 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:44.895+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:44.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:44 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:44.895+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:45.187 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:45 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:45.080+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:45.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:45 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:45.080+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:45.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:44 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:44.832+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:45.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:44 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:44.832+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:45.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:44 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:44.832+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:45.321 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:45 smithi165 podman[58978]: 2024-04-20 11:34:45.005702267 +0000 UTC m=+5.680434456 container died 92faf21cb1d41b85d505b2752e33c979c291242ab306314d6872bb05386db73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T11:34:45.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:45.166+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:45.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:45.166+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:45.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:45.061+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:45.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:45.061+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:45.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:45 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:45.061+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:45.655 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:45 smithi165 podman[58978]: 2024-04-20 11:34:45.421437038 +0000 UTC m=+6.096169234 container cleanup 92faf21cb1d41b85d505b2752e33c979c291242ab306314d6872bb05386db73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T11:34:45.655 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:45 smithi165 bash[58978]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5 2024-04-20T11:34:45.979 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:45 smithi165 podman[58996]: 2024-04-20 11:34:45.655805598 +0000 UTC m=+0.646590123 container remove 92faf21cb1d41b85d505b2752e33c979c291242ab306314d6872bb05386db73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:45.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:45 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:45.722+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:45.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:45 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:45.722+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:45.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:45 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:45.856+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:45.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:45 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:45.856+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:45.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:45 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:45.856+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:46.120 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:45 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:45.853+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:46.120 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:45 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:45.853+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:46.120 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:45 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:45.853+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:46.375 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:46 smithi165 podman[59083]: 2024-04-20 11:34:46.004869926 +0000 UTC m=+0.018365045 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:46.375 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:46 smithi165 podman[59083]: 2024-04-20 11:34:46.115273296 +0000 UTC m=+0.128768429 container create c7d53eefb2b1d5cfbec1ef5a75e3ae335c77ed9b07891c491b4c1bd3eedc4826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T11:34:46.375 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:46 smithi165 podman[59083]: 2024-04-20 11:34:46.376018436 +0000 UTC m=+0.389513527 container init c7d53eefb2b1d5cfbec1ef5a75e3ae335c77ed9b07891c491b4c1bd3eedc4826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T11:34:46.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:46 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:46.119+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:46.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:46 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:46.119+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:46.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:46.117+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:46.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:46.117+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:46.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:46.117+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:46.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:46.064+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:46.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:46.064+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:46.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:46 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:46.064+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:46.669 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:46 smithi165 podman[59083]: 2024-04-20 11:34:46.379230284 +0000 UTC m=+0.392725381 container start c7d53eefb2b1d5cfbec1ef5a75e3ae335c77ed9b07891c491b4c1bd3eedc4826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-20T11:34:46.669 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:46 smithi165 podman[59083]: 2024-04-20 11:34:46.492806582 +0000 UTC m=+0.506301678 container attach c7d53eefb2b1d5cfbec1ef5a75e3ae335c77ed9b07891c491b4c1bd3eedc4826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:34:46.954 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 20 11:34:46 smithi165 podman[59083]: 2024-04-20 11:34:46.670418956 +0000 UTC m=+0.683914059 container died c7d53eefb2b1d5cfbec1ef5a75e3ae335c77ed9b07891c491b4c1bd3eedc4826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T11:34:46.954 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:46 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:46.766+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:46.954 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:46 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:46.766+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:46.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:46 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:46.809+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:46.955 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:46 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:46.809+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:46.955 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:46 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:46.809+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:47.164 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:46 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:46.879+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:47.165 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:46 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:46.879+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:47.165 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:46 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:46.879+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:47.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.5.service' 2024-04-20T11:34:47.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:47.353 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-20T11:34:47.353 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-20T11:34:47.353 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.6 2024-04-20T11:34:47.435 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:47.086+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:47.435 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:47.086+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:47.435 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:47.086+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:47.436 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:47.031+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:47.436 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:47.031+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:47.436 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:47 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:47.031+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:47.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:47 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:47.164+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:47.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:47 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:47.164+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:47.695 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:47 smithi044 systemd[1]: Stopping Ceph osd.6 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:48.150 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:47 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:47.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:48.151 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:47 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:47.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:48.151 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:47 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:47.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:48.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:47 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:47.772+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:48.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:47 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:47.772+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:48.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:47 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:47.812+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:48.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:47 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:47.812+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:48.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:47 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:47.812+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:48.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:48 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:48.149+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:48.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:48 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:48.149+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:48.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:48.025+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:48.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:48.025+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:48.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:48.025+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:48.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:48.008+0000 7ff7afe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:48.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:48.008+0000 7ff7afe00700 -1 osd.6 101 *** Got signal Terminated *** 2024-04-20T11:34:48.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:48.008+0000 7ff7afe00700 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:48.446 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:48.094+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:48.446 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:48.094+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:48.446 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:48.094+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:49.141 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:48 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:48.864+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:49.141 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:48 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:48.864+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:49.141 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:48 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:48.864+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:49.141 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:48 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:48.864+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:49.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:48 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:48.741+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:49.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:48 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:48.741+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:49.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:48 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:48.827+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:49.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:48 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:48.827+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:49.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:48 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:48.827+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:49.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:49.141+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:49.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:49.141+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:49.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:49.078+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:49.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:49.078+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:49.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:49.078+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:49.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:48.978+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:49.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:48.978+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:49.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:48 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:48.978+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:50.131 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:49.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:50.131 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:49.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:50.131 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:49.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:50.131 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:49 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:49.837+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:50.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:49 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:49.771+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:50.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:49 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:49.771+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:50.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:49 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:49.804+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:50.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:49 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:49.804+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:50.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:49 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:49.804+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:50.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:50 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:50.131+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:50.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:50 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:50.131+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:50.106+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:50.106+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:50.106+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:50.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:50.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:50.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:50.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:50 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:50.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:51.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:51 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:51.105+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:51.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:51 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:51.105+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:51.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:50 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:50.858+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:51.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:50 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:50.858+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:51.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:50 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:50.858+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:51.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:50 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:50.858+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:51.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:50.765+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:51.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:50.765+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:51.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:50.765+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:51.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:50.765+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:51.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:50.765+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:51.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:50 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:50.765+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:51.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:51.082+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:51.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:51.082+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:51.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:51.082+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:51.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:51.017+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:51.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:51.017+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:51.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:51.017+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:51.446 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:51 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:51.017+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:52.138 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:51 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:51.905+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:52.139 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:51 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:51.905+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:52.139 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:51 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:51.905+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:52.139 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:51 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:51.905+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:52.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:51 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:51.796+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:52.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:51 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:51.796+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:52.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:51 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:51.796+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:52.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:51 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:51.763+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:52.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:51 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:51.763+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:52.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:51 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:51.763+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:52.423 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:52.138+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:52.424 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:52.138+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:52.424 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:52.138+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:52.079+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:59.623618+0000 front 2024-04-20T11:33:59.623848+0000 (oldest deadline 2024-04-20T11:34:23.024632+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:52.079+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:07.727066+0000 front 2024-04-20T11:34:07.727014+0000 (oldest deadline 2024-04-20T11:34:33.627267+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6[44960]: 2024-04-20T11:34:52.079+0000 7ff7a6600700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:17.729545+0000 front 2024-04-20T11:34:17.729468+0000 (oldest deadline 2024-04-20T11:34:45.331198+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:52.014+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:52.014+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:52.014+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:52.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:52 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:52.014+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:53.187 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:53.107+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:53.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:53.107+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:53.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:53.107+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:53.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:52.880+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:53.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:52.880+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:53.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:52.880+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:53.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:52 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:52.880+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:53.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:52 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:52.791+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:53.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:52 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:52.791+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:53.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:52 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:52.791+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:53.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:52 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:52.775+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:53.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:52 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:52.775+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:53.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:52 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:52.775+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:53.433 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:53.035+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:53.433 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:53.035+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:53.433 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:53.035+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:53.433 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:53.035+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:53.695 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:53 smithi044 podman[86359]: 2024-04-20 11:34:53.433151899 +0000 UTC m=+5.588146148 container died 21143043d532f30958043a45b8eaa2bde1c130f1247f54b0c603691c7ee04901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:54.045 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:53 smithi044 podman[86359]: 2024-04-20 11:34:53.859199553 +0000 UTC m=+6.014193735 container cleanup 21143043d532f30958043a45b8eaa2bde1c130f1247f54b0c603691c7ee04901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T11:34:54.045 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:53 smithi044 bash[86359]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6 2024-04-20T11:34:54.187 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:54.057+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:54.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:54.057+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:54.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:54.057+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:54.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:53.841+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:54.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:53.841+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:54.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:53.841+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:54.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:53 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:53.841+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:54.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:53 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:53.825+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:54.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:53 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:53.825+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:54.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:53 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:53.825+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:54.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:53 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:53.750+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:54.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:53 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:53.750+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:54.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:53 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:53.750+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:54.444 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:54 smithi044 podman[86378]: 2024-04-20 11:34:54.135401046 +0000 UTC m=+0.686184395 container remove 21143043d532f30958043a45b8eaa2bde1c130f1247f54b0c603691c7ee04901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:34:54.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:54.044+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:54.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:54.044+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:54.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:54.044+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:54.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:54 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:54.044+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:55.187 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:55.073+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:55.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:55.073+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:55.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:55.073+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:55.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:54.881+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:55.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:54.881+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:55.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:54.881+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:55.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:54 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:54.881+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:55.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:54 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:54.871+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:55.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:54 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:54.871+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:55.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:54 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:54.871+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:55.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:54 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:54.731+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:55.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:54 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:54.731+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:55.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:54 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:54.731+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:55.349 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:55 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:55.022+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:55.349 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:55 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:55.022+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:55.349 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:55 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:55.022+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:55.350 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:55 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:55.022+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:55.350 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:55 smithi044 podman[86467]: 2024-04-20 11:34:54.908288967 +0000 UTC m=+0.035225681 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:34:55.350 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:55 smithi044 podman[86467]: 2024-04-20 11:34:55.027999792 +0000 UTC m=+0.154936558 container create 33fec76bc2fe7e9ccc655fb4ed4617ace224b8b226d1c0ea0ddf8b6406fbfc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:34:55.350 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:55 smithi044 podman[86467]: 2024-04-20 11:34:55.349431226 +0000 UTC m=+0.476368039 container init 33fec76bc2fe7e9ccc655fb4ed4617ace224b8b226d1c0ea0ddf8b6406fbfc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-20T11:34:55.696 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:55 smithi044 podman[86467]: 2024-04-20 11:34:55.361209413 +0000 UTC m=+0.488146132 container start 33fec76bc2fe7e9ccc655fb4ed4617ace224b8b226d1c0ea0ddf8b6406fbfc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T11:34:55.696 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:55 smithi044 podman[86467]: 2024-04-20 11:34:55.455738688 +0000 UTC m=+0.582675427 container attach 33fec76bc2fe7e9ccc655fb4ed4617ace224b8b226d1c0ea0ddf8b6406fbfc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:34:55.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:55 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:55.845+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:55.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:55 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:55.845+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:55.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:55 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:55.845+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:55.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:55 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:55.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:55.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:55 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:55.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:55.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:55 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:55.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:56.100 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:55.834+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:56.100 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:55.834+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:56.100 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:55.834+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:56.100 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:55 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:55.834+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:56.195 INFO:journalctl@ceph.osd.6.smithi044.stdout:Apr 20 11:34:55 smithi044 podman[86467]: 2024-04-20 11:34:55.834360722 +0000 UTC m=+0.961297478 container died 33fec76bc2fe7e9ccc655fb4ed4617ace224b8b226d1c0ea0ddf8b6406fbfc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-20T11:34:56.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:56.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:56.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:56.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:56.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:56.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:56.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:56 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:56.005+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:56.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:56.100+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:56.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:56.100+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:56.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:56.100+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:56.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.6.service' 2024-04-20T11:34:56.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:34:56.791 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-20T11:34:56.791 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-20T11:34:56.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.7 2024-04-20T11:34:56.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:56 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:56.879+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:56.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:56 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:56.879+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:56.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:56 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:56.879+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:56.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:56 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:56.706+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:56.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:56 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:56.706+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:56.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:56 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:56.706+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:57.073 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:56 smithi078 systemd[1]: Stopping Ceph osd.7 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:34:57.074 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:57.073+0000 7fbf7de48700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:34:57.074 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:56.806+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:57.074 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:56.806+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:57.074 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:56.806+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:57.074 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:56 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:56.806+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:57.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:57.073+0000 7fbf7de48700 -1 osd.7 101 *** Got signal Terminated *** 2024-04-20T11:34:57.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:57.073+0000 7fbf7de48700 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:34:57.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:57.089+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:57.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:57.089+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:57.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:57.089+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:57.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:57.002+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:57.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:57.002+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:57.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:57.002+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:57.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:57 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:57.002+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:57.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:57 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:57.842+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:57.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:57 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:57.842+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:57.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:57 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:57.842+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:57.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:57 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:57.702+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:57.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:57 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:57.702+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:57.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:57 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:57.702+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:58.106 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:57.838+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:58.106 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:57.838+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:58.106 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:57.838+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:58.106 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:57.838+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:58.437 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:58.106+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:58.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:58.106+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:58.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:58.106+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:58.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:58.013+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:58.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:58.013+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:58.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:58.013+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:58.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:58 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:58.013+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:58.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:58 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:58.817+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:58.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:58 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:58.817+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:58.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:58 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:58.817+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:58.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:58 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:58.678+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:58.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:58 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:58.678+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:58.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:58 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:58.678+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:34:59.187 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:59.094+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:34:59.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:59.094+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:34:59.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:34:59.094+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:34:59.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:58.877+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:34:59.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:58.877+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:34:59.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:58.877+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:34:59.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:58 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:58.877+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:34:59.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:59.034+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:34:59.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:59.034+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:34:59.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:59.034+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:34:59.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:59.034+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:34:59.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:34:59 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:34:59.034+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:34:59.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:59 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:59.858+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:34:59.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:59 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:59.858+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:34:59.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:34:59 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:34:59.858+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:34:59.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:59 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:59.719+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:34:59.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:59 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:59.719+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:34:59.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:34:59 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:34:59.719+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:00.189 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:00.100+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:35:00.190 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:00.100+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:35:00.190 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:00.100+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:35:00.190 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:59.894+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:00.190 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:59.894+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:00.190 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:59.894+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:00.190 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:34:59 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:34:59.894+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:00.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:00.060+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:00.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:00.060+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:00.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:00.060+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:00.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:00.060+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:00.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:00 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:00.060+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:01.119+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:01.119+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:01.119+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:01.119+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:36.044474+0000 front 2024-04-20T11:34:36.044454+0000 (oldest deadline 2024-04-20T11:35:00.744666+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:00.929+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:00.929+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:00.929+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:01.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:00.929+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:01.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:00.885+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:01.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:00.885+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:01.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:00.885+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:01.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:00.759+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:01.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:00.759+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:01.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:00.759+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:01.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:00 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:00.759+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:01.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:01 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:01.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:01.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:01 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:01.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:01.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:01 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:01.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:01.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:01 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:01.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:01.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:01 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:01.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:01.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:01.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:01.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:01.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:01.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:01.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:01.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:01.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:01.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:01.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:01.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:01.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:01.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:01 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:01.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:02.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:02.070+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.639772+0000 front 2024-04-20T11:34:08.639791+0000 (oldest deadline 2024-04-20T11:34:32.739929+0000) 2024-04-20T11:35:02.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:02.070+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.541677+0000 front 2024-04-20T11:34:18.541757+0000 (oldest deadline 2024-04-20T11:34:42.041766+0000) 2024-04-20T11:35:02.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:02.070+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:26.043466+0000 front 2024-04-20T11:34:26.043558+0000 (oldest deadline 2024-04-20T11:34:51.943279+0000) 2024-04-20T11:35:02.438 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7[39575]: 2024-04-20T11:35:02.070+0000 7fbf7a43c700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:36.044474+0000 front 2024-04-20T11:34:36.044454+0000 (oldest deadline 2024-04-20T11:35:00.744666+0000) 2024-04-20T11:35:02.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:01.955+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:02.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:01.955+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:02.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:01.955+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:02.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:01 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:01.955+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:02.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:02.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:02.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:02.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:02.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.032+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:02.839 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 podman[57769]: 2024-04-20 11:35:02.521111039 +0000 UTC m=+5.527780281 container died 58b0a3f52806c135825e364ed11fafeac7f04c3924bd045e81466773502a4acb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:35:02.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:02.950+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:02.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:02.950+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:02.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:02.950+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:02.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:02.950+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:02.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:02.726+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:02.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:02.726+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:02.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:02.726+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:02.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:02 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:02.726+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:03.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 podman[57769]: 2024-04-20 11:35:02.839876429 +0000 UTC m=+5.846545680 container cleanup 58b0a3f52806c135825e364ed11fafeac7f04c3924bd045e81466773502a4acb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:35:03.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:02 smithi078 bash[57769]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7 2024-04-20T11:35:03.188 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57786]: 2024-04-20 11:35:03.041023119 +0000 UTC m=+0.518177748 container remove 58b0a3f52806c135825e364ed11fafeac7f04c3924bd045e81466773502a4acb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-20T11:35:03.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:02.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:03.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:02.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:03.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:02.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:03.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:02.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:03.189 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:02 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:02.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:03.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.989+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:03.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.989+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:03.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.989+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:03.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.989+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:03.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:02 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:02.989+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:03.777 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57877]: 2024-04-20 11:35:03.361701675 +0000 UTC m=+0.017613010 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:35:03.777 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57877]: 2024-04-20 11:35:03.458343546 +0000 UTC m=+0.114254973 container create 1c7d5c7c45bb7db0546c9bd68411da6ab43ff1ef481701b5ccc75ba3018172a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-20T11:35:03.777 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57877]: 2024-04-20 11:35:03.69162972 +0000 UTC m=+0.347541036 container init 1c7d5c7c45bb7db0546c9bd68411da6ab43ff1ef481701b5ccc75ba3018172a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-20T11:35:03.777 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57877]: 2024-04-20 11:35:03.695131173 +0000 UTC m=+0.351042497 container start 1c7d5c7c45bb7db0546c9bd68411da6ab43ff1ef481701b5ccc75ba3018172a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-20T11:35:03.777 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57877]: 2024-04-20 11:35:03.777252555 +0000 UTC m=+0.433163871 container attach 1c7d5c7c45bb7db0546c9bd68411da6ab43ff1ef481701b5ccc75ba3018172a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-20T11:35:03.976 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:03.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:03.977 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:03.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:03.977 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:03.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:03.977 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:03.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:04.187 INFO:journalctl@ceph.osd.7.smithi078.stdout:Apr 20 11:35:03 smithi078 podman[57877]: 2024-04-20 11:35:03.987673554 +0000 UTC m=+0.643584877 container died 1c7d5c7c45bb7db0546c9bd68411da6ab43ff1ef481701b5ccc75ba3018172a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T11:35:04.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:04.005+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:04.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:04.005+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:04.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:04.005+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:04.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:04.005+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:04.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:04.005+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:04.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:03.977+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:04.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:03.977+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:04.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:03.977+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:04.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:03 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:03.977+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:04.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:03 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:03.974+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:04.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:03 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:03.974+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:04.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:03 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:03.974+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:04.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:03 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:03.974+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:04.445 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:03 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:03.974+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:04.571 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.7.service' 2024-04-20T11:35:04.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:04.609 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-20T11:35:04.609 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-20T11:35:04.609 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.8 2024-04-20T11:35:04.924 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 systemd[1]: Stopping Ceph osd.8 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:04.925 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:04.697+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:04.925 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:04.697+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:04.925 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:04.697+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:04.925 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:04.697+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:05.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:05.013+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:05.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:05.014+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:05.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:05.014+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:05.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:05.014+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:05.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:05 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:05.014+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:05.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:04 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:04.940+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:05.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:04 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:04.940+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:05.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:04 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:04.940+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:05.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:04 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:04.940+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:05.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:04 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:04.940+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:05.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.925+0000 7f531b71c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:05.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.925+0000 7f531b71c700 -1 osd.8 101 *** Got signal Terminated *** 2024-04-20T11:35:05.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.925+0000 7f531b71c700 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:35:05.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.965+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:05.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.965+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:05.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.965+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:05.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:04 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:04.965+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:05.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:05.931+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:05.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:05.931+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:05.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:05.931+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:05.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:05.931+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:05.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:05.708+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:05.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:05.708+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:05.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:05.708+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:05.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:05.708+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:05.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:05 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:05.708+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:06.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:05.943+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:06.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:05.943+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:06.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:05.943+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:06.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:05.943+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:06.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:05 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:05.943+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:06.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:06.036+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:06.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:06.036+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:06.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:06.036+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:06.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:06.036+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:06.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:06 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:06.036+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:07.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:06.899+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:07.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:06.899+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:07.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:06.899+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:07.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:06.899+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:07.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:06.899+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:07.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:06 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:06.899+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:07.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:06.754+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:07.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:06.754+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:07.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:06.754+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:07.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:06.754+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:07.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:06.754+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:07.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:06.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:07.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:06.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:07.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:06.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:07.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:06 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:06.930+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:07.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:07.038+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:07.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:07.038+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:07.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:07.038+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:07.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:07.038+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:07.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:07 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:07.038+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:07.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:07.916+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:07.979 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:07.916+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:07.916+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:07.916+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:07.722+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:07.722+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:07.722+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:07.722+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:07.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:07 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:07.722+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:08.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:07 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:07.861+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:08.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:07 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:07.861+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:08.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:07 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:07.861+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:08.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:07 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:07.861+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:08.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:07 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:07.861+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:08.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:07 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:07.861+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:08.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.045+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:08.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.045+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:08.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.045+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:08.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.045+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:08.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.045+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:08.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:08.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:08.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:08.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:08.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:08.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:08.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:08.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:08.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:08.681+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:09.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:08.859+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:09.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:08.859+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:09.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:08.859+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:09.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:08.859+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:09.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:08.859+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:09.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:08 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:08.859+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:09.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:08.954+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:09.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:08.954+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:09.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:08.954+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:09.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:08 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:08.954+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:09.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:09.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:09.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:09.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:09.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:08.997+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:09.921 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:09.657+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:09.921 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:09.657+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:09.921 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:09.657+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:09.921 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:09.657+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:09.921 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:09.657+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:10.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:09.895+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:10.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:09.895+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:10.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:09.895+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:10.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:09.895+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:10.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:09.895+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:10.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:09 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:09.895+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:10.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:09.921+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.259473+0000 front 2024-04-20T11:34:10.259600+0000 (oldest deadline 2024-04-20T11:34:32.559366+0000) 2024-04-20T11:35:10.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:09.921+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.861311+0000 front 2024-04-20T11:34:18.861332+0000 (oldest deadline 2024-04-20T11:34:42.361399+0000) 2024-04-20T11:35:10.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:09.921+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:25.162952+0000 front 2024-04-20T11:34:25.163167+0000 (oldest deadline 2024-04-20T11:34:49.862696+0000) 2024-04-20T11:35:10.229 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:09.921+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:31.563721+0000 front 2024-04-20T11:34:31.563807+0000 (oldest deadline 2024-04-20T11:35:02.164234+0000) 2024-04-20T11:35:10.230 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:09 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8[40624]: 2024-04-20T11:35:09.921+0000 7f5317d10700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:42.665301+0000 front 2024-04-20T11:34:42.665272+0000 (oldest deadline 2024-04-20T11:35:09.065767+0000) 2024-04-20T11:35:10.437 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:10 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:10.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:10.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:10 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:10.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:10.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:10 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:10.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:10.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:10 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:10.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:10.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:10 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:10.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:10.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:10 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:10.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:10.729 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:10 smithi165 podman[59189]: 2024-04-20 11:35:10.460797405 +0000 UTC m=+5.660166120 container died 3f4b99bfbd261b94144c16f74a3305b1928c6972415ee55a5ce86ee3314067cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-20T11:35:10.729 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:10 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:10.676+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:10.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:10 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:10.676+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:10.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:10 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:10.676+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:10.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:10 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:10.676+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:10.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:10 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:10.676+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:11.108 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:10 smithi165 podman[59189]: 2024-04-20 11:35:10.849790573 +0000 UTC m=+6.049159287 container cleanup 3f4b99bfbd261b94144c16f74a3305b1928c6972415ee55a5ce86ee3314067cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:35:11.108 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:10 smithi165 bash[59189]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8 2024-04-20T11:35:11.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:10.878+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:11.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:10.878+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:11.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:10.878+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:11.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:10.878+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:11.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:10.878+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:11.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:10 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:10.878+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:11.339 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:11 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:11.057+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:11.339 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:11 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:11.057+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:11.339 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:11 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:11.057+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:11.339 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:11 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:11.057+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:11.340 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:11 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:11.057+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:11.340 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:11 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:11.057+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:11.463 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:11 smithi165 podman[59206]: 2024-04-20 11:35:11.109565201 +0000 UTC m=+0.645134071 container remove 3f4b99bfbd261b94144c16f74a3305b1928c6972415ee55a5ce86ee3314067cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T11:35:11.729 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:11 smithi165 podman[59295]: 2024-04-20 11:35:11.36680644 +0000 UTC m=+0.018124274 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:35:11.729 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:11 smithi165 podman[59295]: 2024-04-20 11:35:11.485544224 +0000 UTC m=+0.136862026 container create 10819bd61352286564eaaacafc5607eec3e5ff5b33d167f9bb6e170673ecdcfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-20T11:35:11.729 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:11 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:11.664+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:11.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:11 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:11.664+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:11.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:11 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:11.664+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:11.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:11 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:11.664+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:11.730 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:11 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:11.664+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:12.030 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:11 smithi165 podman[59295]: 2024-04-20 11:35:11.729652392 +0000 UTC m=+0.380970194 container init 10819bd61352286564eaaacafc5607eec3e5ff5b33d167f9bb6e170673ecdcfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-20T11:35:12.031 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:11 smithi165 podman[59295]: 2024-04-20 11:35:11.733825443 +0000 UTC m=+0.385143247 container start 10819bd61352286564eaaacafc5607eec3e5ff5b33d167f9bb6e170673ecdcfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:35:12.031 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:11 smithi165 podman[59295]: 2024-04-20 11:35:11.846565514 +0000 UTC m=+0.497883316 container attach 10819bd61352286564eaaacafc5607eec3e5ff5b33d167f9bb6e170673ecdcfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T11:35:12.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:11.865+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:12.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:11.865+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:12.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:11.865+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:12.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:11.865+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:12.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:11.865+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:12.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:11 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:11.865+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:12.304 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 20 11:35:12 smithi165 podman[59295]: 2024-04-20 11:35:12.031793459 +0000 UTC m=+0.683111282 container died 10819bd61352286564eaaacafc5607eec3e5ff5b33d167f9bb6e170673ecdcfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-20T11:35:12.350 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:12.010+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:12.351 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:12.010+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:12.351 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:12.010+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:12.351 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:12.010+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:12.351 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:12.010+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:12.351 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:12 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:12.010+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:12.676 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.8.service' 2024-04-20T11:35:12.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:12.714 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-20T11:35:12.714 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-20T11:35:12.715 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.9 2024-04-20T11:35:12.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:12.651+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:12.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:12.651+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:12.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:12.651+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:12.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:12.651+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:12.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:12.651+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 systemd[1]: Stopping Ceph osd.9 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:12.868+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:12.868+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:12.868+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:12.868+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:12.868+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:13.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:12 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:12.868+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:13.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:13 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:13.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:13.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:13 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:13.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:13.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:13 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:13.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:13.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:13 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:13.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:13.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:13 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:13.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:13.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:13 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:13.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:13.695 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.295+0000 7fd4fbc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:13.696 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.295+0000 7fd4fbc00700 -1 osd.9 101 *** Got signal Terminated *** 2024-04-20T11:35:13.696 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.295+0000 7fd4fbc00700 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:35:13.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:13.665+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:13.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:13.665+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:13.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:13.665+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:13.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:13.665+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:13.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:13 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:13.665+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:14.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.919+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:14.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.919+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:14.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.919+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:14.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.919+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:14.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.919+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:14.196 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:13 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:13.919+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:14.437 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:14 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:14.068+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:14.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:14 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:14.068+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:14.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:14 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:14.068+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:14.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:14 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:14.068+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:14.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:14 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:14.068+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:14.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:14 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:14.068+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:14.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:14.652+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:14.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:14.652+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:14.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:14.652+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:14.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:14.652+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:14.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:14.652+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:14.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:14 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:14.652+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:15.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:14.879+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:15.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:14.879+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:15.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:14.879+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:15.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:14.879+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:15.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:14.879+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:15.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:14 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:14.879+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:15.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:15.080+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:15.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:15.080+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:15.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:15.080+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:15.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:15.080+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:15.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:15.080+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:15.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:15 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:15.080+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:15.893 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:15.636+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:15.893 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:15.636+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:15.893 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:15.636+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:15.893 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:15.636+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:15.893 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:15.636+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:15.893 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:15 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:15.636+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:15.923 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:15 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:15.902+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:15.923 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:15 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:15.902+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:16.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:15 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:15.902+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:16.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:15 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:15.902+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:16.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:15 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:15.902+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:16.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:15 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:15.902+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:16.437 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:16.052+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:16.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:16.052+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:16.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:16.052+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:16.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:16.052+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:16.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:16.052+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:16.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:16 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:16.052+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:16.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:16.645+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:16.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:16.645+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:16.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:16.645+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:16.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:16.645+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:16.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:16.645+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:16.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:16.645+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:17.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:16 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:16.890+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:17.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:16 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:16.890+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:17.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:16 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:16.890+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:17.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:16 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:16.890+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:17.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:16 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:16.890+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:17.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:16 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:16.890+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:17.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:17.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:17.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:17.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:17.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:17.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:17.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:17 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:17.064+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:17.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:17.648+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:17.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:17.648+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:17.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:17.648+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:17.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:17.648+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:17.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:17.648+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:17.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:17 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:17.648+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:18.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:17 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:17.904+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:00.842693+0000 front 2024-04-20T11:34:00.843048+0000 (oldest deadline 2024-04-20T11:34:26.142775+0000) 2024-04-20T11:35:18.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:17 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:17.904+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:08.946792+0000 front 2024-04-20T11:34:08.946546+0000 (oldest deadline 2024-04-20T11:34:32.446279+0000) 2024-04-20T11:35:18.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:17 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:17.904+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:18.751073+0000 front 2024-04-20T11:34:18.750791+0000 (oldest deadline 2024-04-20T11:34:41.050362+0000) 2024-04-20T11:35:18.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:17 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:17.904+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.552690+0000 front 2024-04-20T11:34:24.552766+0000 (oldest deadline 2024-04-20T11:34:49.852646+0000) 2024-04-20T11:35:18.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:17 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:17.904+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:35.754290+0000 front 2024-04-20T11:34:35.754426+0000 (oldest deadline 2024-04-20T11:34:58.654486+0000) 2024-04-20T11:35:18.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:17 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9[51535]: 2024-04-20T11:35:17.904+0000 7fd4f2400700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.758419+0000 front 2024-04-20T11:34:43.758302+0000 (oldest deadline 2024-04-20T11:35:06.058218+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:18.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:18 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:18.018+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:18.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:18.659+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:18.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:18.659+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:18.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:18.659+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:18.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:18.659+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:18.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:18.659+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:18.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:18 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:18.659+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:19.195 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:18 smithi044 podman[86571]: 2024-04-20 11:35:18.858539015 +0000 UTC m=+5.703507870 container died f1a4ca5c8bb97b6a9ad20f869820a08946ab0528c1ad6ba8111ab8087dde6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:19.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:19 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:19.035+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:19.563 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:19 smithi044 podman[86571]: 2024-04-20 11:35:19.329040114 +0000 UTC m=+6.174009010 container cleanup f1a4ca5c8bb97b6a9ad20f869820a08946ab0528c1ad6ba8111ab8087dde6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T11:35:19.563 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:19 smithi044 bash[86571]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9 2024-04-20T11:35:19.944 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:19 smithi044 podman[86589]: 2024-04-20 11:35:19.563435069 +0000 UTC m=+0.688043248 container remove f1a4ca5c8bb97b6a9ad20f869820a08946ab0528c1ad6ba8111ab8087dde6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-20T11:35:19.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:19.696+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:19.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:19.696+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:19.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:19.696+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:19.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:19.696+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:19.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:19.696+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:19.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:19 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:19.696+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:20.437 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:20.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:20.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:20.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:20.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:20.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:20.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:20 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:20.034+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:20.642 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:20 smithi044 podman[86673]: 2024-04-20 11:35:20.274659866 +0000 UTC m=+0.034824545 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:35:20.643 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:20 smithi044 podman[86673]: 2024-04-20 11:35:20.380960675 +0000 UTC m=+0.141125391 container create af8bd46553d4eb943d0dfab3b746439e8c0314f2416dc140843e9841150cab6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T11:35:20.643 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:20 smithi044 podman[86673]: 2024-04-20 11:35:20.642906974 +0000 UTC m=+0.403071697 container init af8bd46553d4eb943d0dfab3b746439e8c0314f2416dc140843e9841150cab6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:35:20.945 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:20 smithi044 podman[86673]: 2024-04-20 11:35:20.655112465 +0000 UTC m=+0.415277154 container start af8bd46553d4eb943d0dfab3b746439e8c0314f2416dc140843e9841150cab6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T11:35:20.945 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:20 smithi044 podman[86673]: 2024-04-20 11:35:20.775109098 +0000 UTC m=+0.535273821 container attach af8bd46553d4eb943d0dfab3b746439e8c0314f2416dc140843e9841150cab6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-20T11:35:21.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:20.734+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:21.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:20.734+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:21.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:20.734+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:21.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:20.734+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:21.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:20.734+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:21.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:20.734+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:21.432 INFO:journalctl@ceph.osd.9.smithi044.stdout:Apr 20 11:35:21 smithi044 podman[86673]: 2024-04-20 11:35:21.126649056 +0000 UTC m=+0.886813748 container died af8bd46553d4eb943d0dfab3b746439e8c0314f2416dc140843e9841150cab6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-20T11:35:21.437 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:21.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:21.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:21.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:21.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:21.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:21.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:21 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:21.046+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:21.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.9.service' 2024-04-20T11:35:21.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:21.956 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-20T11:35:21.956 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-20T11:35:21.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.10 2024-04-20T11:35:21.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:21 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:21.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:21.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:21 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:21.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:21.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:21 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:21.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:21.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:21 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:21.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:21.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:21 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:21.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:21.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:21 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:21.693+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:22.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 systemd[1]: Stopping Ceph osd.10 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:22.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:22.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:22.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:22.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:22.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:22.259 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:22.259 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.089+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:22.259 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.258+0000 7fe56bc4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:22.259 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.258+0000 7fe56bc4d700 -1 osd.10 101 *** Got signal Terminated *** 2024-04-20T11:35:22.259 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:22 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:22.258+0000 7fe56bc4d700 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:35:22.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:22.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:22.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:22.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:22.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:22.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:22.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:22.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:22.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:22.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:22.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:22 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:22.720+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:23.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:23 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:23.091+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:23.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:23.728+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:23.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:23.728+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:23.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:23.728+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:23.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:23.728+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:23.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:23.728+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:23.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:23 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:23.728+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:24.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:24 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:24.097+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:24.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:24.698+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:24.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:24.698+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:24.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:24.698+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:24.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:24.698+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:24.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:24.698+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:24.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:24.698+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:25.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:25 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:25.103+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:26.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:25.738+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:26.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:25.738+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:26.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:25.738+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:26.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:25.738+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:26.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:25.738+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:26.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:25 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:25.738+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:26.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:26 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:26.118+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:27.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:26.779+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:27.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:26.779+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:27.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:26.779+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:27.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:26.779+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:27.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:26.779+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:27.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:26 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:26.779+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:27.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:34:01.048425+0000 front 2024-04-20T11:34:01.048318+0000 (oldest deadline 2024-04-20T11:34:23.948407+0000) 2024-04-20T11:35:27.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.349791+0000 front 2024-04-20T11:34:10.349884+0000 (oldest deadline 2024-04-20T11:34:32.649957+0000) 2024-04-20T11:35:27.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:15.451529+0000 front 2024-04-20T11:34:15.451563+0000 (oldest deadline 2024-04-20T11:34:41.351620+0000) 2024-04-20T11:35:27.438 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-04-20T11:34:24.153332+0000 front 2024-04-20T11:34:24.153165+0000 (oldest deadline 2024-04-20T11:34:48.252974+0000) 2024-04-20T11:35:27.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:32.754573+0000 front 2024-04-20T11:34:32.754557+0000 (oldest deadline 2024-04-20T11:35:02.155154+0000) 2024-04-20T11:35:27.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:43.856031+0000 front 2024-04-20T11:34:43.856155+0000 (oldest deadline 2024-04-20T11:35:09.756087+0000) 2024-04-20T11:35:27.439 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10[46190]: 2024-04-20T11:35:27.157+0000 7fe568241700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:50.257247+0000 front 2024-04-20T11:34:50.257355+0000 (oldest deadline 2024-04-20T11:35:16.057679+0000) 2024-04-20T11:35:27.938 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:27 smithi078 podman[57994]: 2024-04-20 11:35:27.652463947 +0000 UTC m=+5.505240985 container died 83e3c596f3046f0f194fca021c9f00c87517a036970e79144bc89fb0e1f2d535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-20T11:35:28.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:28.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:28.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:28.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:28.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:28.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:28.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:27 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:27.823+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:28.258 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[57994]: 2024-04-20 11:35:28.050083687 +0000 UTC m=+5.902860731 container cleanup 83e3c596f3046f0f194fca021c9f00c87517a036970e79144bc89fb0e1f2d535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T11:35:28.259 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 bash[57994]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10 2024-04-20T11:35:28.548 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[58013]: 2024-04-20 11:35:28.259551769 +0000 UTC m=+0.598829685 container remove 83e3c596f3046f0f194fca021c9f00c87517a036970e79144bc89fb0e1f2d535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T11:35:28.853 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[58101]: 2024-04-20 11:35:28.452071371 +0000 UTC m=+0.017933309 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:35:28.853 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[58101]: 2024-04-20 11:35:28.560200912 +0000 UTC m=+0.126062861 container create 9338851fc5d3f07e316bf6b943ab531aff8339b9bc1bfae07f303a13c0707682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-20T11:35:28.853 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[58101]: 2024-04-20 11:35:28.779302476 +0000 UTC m=+0.345164388 container init 9338851fc5d3f07e316bf6b943ab531aff8339b9bc1bfae07f303a13c0707682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-20T11:35:28.853 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[58101]: 2024-04-20 11:35:28.78237675 +0000 UTC m=+0.348238657 container start 9338851fc5d3f07e316bf6b943ab531aff8339b9bc1bfae07f303a13c0707682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-20T11:35:29.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:28 smithi078 podman[58101]: 2024-04-20 11:35:28.853971343 +0000 UTC m=+0.419833264 container attach 9338851fc5d3f07e316bf6b943ab531aff8339b9bc1bfae07f303a13c0707682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-20T11:35:29.188 INFO:journalctl@ceph.osd.10.smithi078.stdout:Apr 20 11:35:29 smithi078 podman[58101]: 2024-04-20 11:35:29.069117177 +0000 UTC m=+0.634979094 container died 9338851fc5d3f07e316bf6b943ab531aff8339b9bc1bfae07f303a13c0707682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=) 2024-04-20T11:35:29.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:29.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:29.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:29.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:29.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:29.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:29.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:28 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:28.868+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:29.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.10.service' 2024-04-20T11:35:29.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:29.701 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-20T11:35:29.701 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-20T11:35:29.701 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.11 2024-04-20T11:35:29.979 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 systemd[1]: Stopping Ceph osd.11 for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:29.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:29 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:29.855+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:30.479 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.011+0000 7f9ba3d66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:30.479 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.011+0000 7f9ba3d66700 -1 osd.11 101 *** Got signal Terminated *** 2024-04-20T11:35:30.480 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.011+0000 7f9ba3d66700 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T11:35:31.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:31.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:31.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:31.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:31.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:31.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:31.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:30 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:30.824+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:32.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:32.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:32.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:32.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:32.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:32.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:32.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:31 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:31.831+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:33.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:33.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:33.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:33.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:33.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:33.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:33.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:33.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:32 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:32.857+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-20T11:35:09.919503+0000 front 2024-04-20T11:35:09.919530+0000 (oldest deadline 2024-04-20T11:35:32.819514+0000) 2024-04-20T11:35:34.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:34.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:34.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:34.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:34.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:34.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:34.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:34.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:33 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:33.825+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-20T11:35:09.919503+0000 front 2024-04-20T11:35:09.919530+0000 (oldest deadline 2024-04-20T11:35:32.819514+0000) 2024-04-20T11:35:35.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-04-20T11:33:57.608988+0000 front 2024-04-20T11:33:57.608733+0000 (oldest deadline 2024-04-20T11:34:22.908783+0000) 2024-04-20T11:35:35.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2024-04-20T11:34:10.411073+0000 front 2024-04-20T11:34:10.411102+0000 (oldest deadline 2024-04-20T11:34:36.310859+0000) 2024-04-20T11:35:35.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-20T11:34:16.311191+0000 front 2024-04-20T11:34:16.311340+0000 (oldest deadline 2024-04-20T11:34:40.411385+0000) 2024-04-20T11:35:35.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2024-04-20T11:34:33.913455+0000 front 2024-04-20T11:34:33.913513+0000 (oldest deadline 2024-04-20T11:34:59.813336+0000) 2024-04-20T11:35:35.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-20T11:34:39.813688+0000 front 2024-04-20T11:34:39.813724+0000 (oldest deadline 2024-04-20T11:35:05.113807+0000) 2024-04-20T11:35:35.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-04-20T11:34:52.016077+0000 front 2024-04-20T11:34:52.016189+0000 (oldest deadline 2024-04-20T11:35:13.715786+0000) 2024-04-20T11:35:35.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2024-04-20T11:35:01.317419+0000 front 2024-04-20T11:35:01.317450+0000 (oldest deadline 2024-04-20T11:35:27.217395+0000) 2024-04-20T11:35:35.230 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:34 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11[47251]: 2024-04-20T11:35:34.803+0000 7f9ba035a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-20T11:35:09.919503+0000 front 2024-04-20T11:35:09.919530+0000 (oldest deadline 2024-04-20T11:35:32.819514+0000) 2024-04-20T11:35:35.784 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:35 smithi165 podman[59413]: 2024-04-20 11:35:35.483211668 +0000 UTC m=+5.581554309 container died 148ed31345fb31a8b89afbb1a3bf0245bbcef4fe66e72db5003c7474d3bd0437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-20T11:35:36.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:35 smithi165 podman[59413]: 2024-04-20 11:35:35.785248225 +0000 UTC m=+5.883590870 container cleanup 148ed31345fb31a8b89afbb1a3bf0245bbcef4fe66e72db5003c7474d3bd0437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T11:35:36.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:35 smithi165 bash[59413]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11 2024-04-20T11:35:36.229 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59431]: 2024-04-20 11:35:36.019611323 +0000 UTC m=+0.525780491 container remove 148ed31345fb31a8b89afbb1a3bf0245bbcef4fe66e72db5003c7474d3bd0437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T11:35:36.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59522]: 2024-04-20 11:35:36.238248836 +0000 UTC m=+0.018615871 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T11:35:36.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59522]: 2024-04-20 11:35:36.345678233 +0000 UTC m=+0.126045244 container create f8910c558fe8d822e9e830ed6a2bfebf24372d201684d07682b6c850c1bcbe27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T11:35:36.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59522]: 2024-04-20 11:35:36.581448047 +0000 UTC m=+0.361815067 container init f8910c558fe8d822e9e830ed6a2bfebf24372d201684d07682b6c850c1bcbe27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=) 2024-04-20T11:35:36.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59522]: 2024-04-20 11:35:36.584502349 +0000 UTC m=+0.364869367 container start f8910c558fe8d822e9e830ed6a2bfebf24372d201684d07682b6c850c1bcbe27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:35:36.655 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59522]: 2024-04-20 11:35:36.656233873 +0000 UTC m=+0.436600898 container attach f8910c558fe8d822e9e830ed6a2bfebf24372d201684d07682b6c850c1bcbe27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True) 2024-04-20T11:35:36.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:36 smithi165 podman[59522]: 2024-04-20 11:35:36.875035394 +0000 UTC m=+0.655402413 container died f8910c558fe8d822e9e830ed6a2bfebf24372d201684d07682b6c850c1bcbe27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:35:37.479 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 20 11:35:37 smithi165 podman[59547]: 2024-04-20 11:35:37.459381198 +0000 UTC m=+0.578499286 container remove f8910c558fe8d822e9e830ed6a2bfebf24372d201684d07682b6c850c1bcbe27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T11:35:37.486 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@osd.11.service' 2024-04-20T11:35:37.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:37.524 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-20T11:35:37.524 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-20T11:35:37.525 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.a 2024-04-20T11:35:37.945 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:37 smithi044 systemd[1]: Stopping Ceph mds.a for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:38.314 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:38 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a[58055]: 2024-04-20T11:35:38.078+0000 7f7ecb400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:38.314 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:38 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a[58055]: 2024-04-20T11:35:38.078+0000 7f7ecb400700 -1 mds.a *** got signal Terminated *** 2024-04-20T11:35:42.413 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:42 smithi044 podman[86779]: 2024-04-20 11:35:42.097648813 +0000 UTC m=+4.126970128 container died c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T11:35:42.696 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:42 smithi044 podman[86779]: 2024-04-20 11:35:42.538012055 +0000 UTC m=+4.567333393 container cleanup c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:35:42.696 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:42 smithi044 bash[86779]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a 2024-04-20T11:35:43.195 INFO:journalctl@ceph.mds.a.smithi044.stdout:Apr 20 11:35:42 smithi044 podman[86804]: 2024-04-20 11:35:42.853451494 +0000 UTC m=+0.736575820 container remove c83028cae8ad1176c6f7922e5c299febb8bc50c109cce5f03c40d818756d9c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T11:35:43.494 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.a.service' 2024-04-20T11:35:43.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:43.614 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-20T11:35:43.614 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-20T11:35:43.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.d 2024-04-20T11:35:44.195 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:43 smithi044 systemd[1]: Stopping Ceph mds.d for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:44.694 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d[57057]: 2024-04-20T11:35:44.273+0000 7fdc5c600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:44.694 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:44 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d[57057]: 2024-04-20T11:35:44.273+0000 7fdc5c600700 -1 mds.d *** got signal Terminated *** 2024-04-20T11:35:47.945 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:47 smithi044 podman[86911]: 2024-04-20 11:35:47.607272994 +0000 UTC m=+3.514951895 container died cdbcc3e69204df8e2a4dfd0dc5aa032e479179f298e7a3264827d1ae2f779485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T11:35:48.395 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:48 smithi044 podman[86911]: 2024-04-20 11:35:48.119302645 +0000 UTC m=+4.026981581 container cleanup cdbcc3e69204df8e2a4dfd0dc5aa032e479179f298e7a3264827d1ae2f779485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-20T11:35:48.396 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:48 smithi044 bash[86911]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d 2024-04-20T11:35:48.695 INFO:journalctl@ceph.mds.d.smithi044.stdout:Apr 20 11:35:48 smithi044 podman[86925]: 2024-04-20 11:35:48.39543084 +0000 UTC m=+0.776696051 container remove cdbcc3e69204df8e2a4dfd0dc5aa032e479179f298e7a3264827d1ae2f779485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-20T11:35:48.950 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.d.service' 2024-04-20T11:35:49.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:49.060 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-20T11:35:49.060 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-20T11:35:49.060 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.g 2024-04-20T11:35:49.445 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:49 smithi044 systemd[1]: Stopping Ceph mds.g for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:49.945 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g[57660]: 2024-04-20T11:35:49.685+0000 7f5a56200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:49.945 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:49 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g[57660]: 2024-04-20T11:35:49.685+0000 7f5a56200700 -1 mds.g *** got signal Terminated *** 2024-04-20T11:35:51.920 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:51 smithi044 podman[87031]: 2024-04-20 11:35:51.583562237 +0000 UTC m=+2.031202182 container died 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-20T11:35:52.195 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:52 smithi044 podman[87031]: 2024-04-20 11:35:52.056513203 +0000 UTC m=+2.504153164 container cleanup 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-20T11:35:52.195 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:52 smithi044 bash[87031]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g 2024-04-20T11:35:52.694 INFO:journalctl@ceph.mds.g.smithi044.stdout:Apr 20 11:35:52 smithi044 podman[87045]: 2024-04-20 11:35:52.341018627 +0000 UTC m=+0.735175220 container remove 0c0b088349762eb9879f0fc751dfe116f4aa5c0ce63571cd846cd168485c5fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T11:35:52.894 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.g.service' 2024-04-20T11:35:53.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:53.007 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-20T11:35:53.007 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-20T11:35:53.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.j 2024-04-20T11:35:53.194 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:53 smithi044 systemd[1]: Stopping Ceph mds.j for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:53.945 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j[56677]: 2024-04-20T11:35:53.591+0000 7fe916600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:53.945 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:53 smithi044 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j[56677]: 2024-04-20T11:35:53.591+0000 7fe916600700 -1 mds.j *** got signal Terminated *** 2024-04-20T11:35:55.848 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:55 smithi044 podman[87151]: 2024-04-20 11:35:55.592794345 +0000 UTC m=+2.123202061 container died 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-20T11:35:56.161 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:55 smithi044 podman[87151]: 2024-04-20 11:35:55.943970738 +0000 UTC m=+2.474378461 container cleanup 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-20T11:35:56.161 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:55 smithi044 bash[87151]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j 2024-04-20T11:35:56.445 INFO:journalctl@ceph.mds.j.smithi044.stdout:Apr 20 11:35:56 smithi044 podman[87165]: 2024-04-20 11:35:56.161367701 +0000 UTC m=+0.556064355 container remove 5d318443ab469efe11b5c79bf0ae883e58fe520b25ec4e533f7bbb42c97b725b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:35:56.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.j.service' 2024-04-20T11:35:56.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:35:56.753 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-20T11:35:56.753 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-20T11:35:56.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.b 2024-04-20T11:35:57.188 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:35:56 smithi078 systemd[1]: Stopping Ceph mds.b for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:35:57.687 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:35:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:35:57.189+0000 7fbc22f74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:35:57.688 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:35:57 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b[51416]: 2024-04-20T11:35:57.189+0000 7fbc22f74700 -1 mds.b *** got signal Terminated *** 2024-04-20T11:35:59.799 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:35:59 smithi078 podman[58208]: 2024-04-20 11:35:59.5061645 +0000 UTC m=+2.556525464 container died 731415ecc45a0242693f71ed4240297f802f7ade015cfd756dc864c90233b4a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T11:36:00.124 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:35:59 smithi078 podman[58208]: 2024-04-20 11:35:59.899183804 +0000 UTC m=+2.949544770 container cleanup 731415ecc45a0242693f71ed4240297f802f7ade015cfd756dc864c90233b4a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-20T11:36:00.124 INFO:journalctl@ceph.mds.b.smithi078.stdout:Apr 20 11:35:59 smithi078 bash[58208]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-b 2024-04-20T11:36:00.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.b.service' 2024-04-20T11:36:00.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:00.292 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-20T11:36:00.292 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-20T11:36:00.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.e 2024-04-20T11:36:00.437 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:00 smithi078 systemd[1]: Stopping Ceph mds.e for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:00.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e[50784]: 2024-04-20T11:36:00.609+0000 7f23a417a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:00.938 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:00 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e[50784]: 2024-04-20T11:36:00.609+0000 7f23a417a700 -1 mds.e *** got signal Terminated *** 2024-04-20T11:36:03.801 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:03 smithi078 podman[58341]: 2024-04-20 11:36:03.52257084 +0000 UTC m=+3.017028009 container died 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-20T11:36:03.802 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:03 smithi078 podman[58341]: 2024-04-20 11:36:03.802461328 +0000 UTC m=+3.296918496 container cleanup 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-20T11:36:04.143 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:03 smithi078 bash[58341]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e 2024-04-20T11:36:04.143 INFO:journalctl@ceph.mds.e.smithi078.stdout:Apr 20 11:36:04 smithi078 podman[58355]: 2024-04-20 11:36:04.045105941 +0000 UTC m=+0.523087749 container remove 8c5dd656b8ddf268a11a78faf841e87edd17709dd77ed1aaf253037c1726367c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-20T11:36:04.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.e.service' 2024-04-20T11:36:04.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:04.190 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-20T11:36:04.190 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-20T11:36:04.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.h 2024-04-20T11:36:04.438 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:04 smithi078 systemd[1]: Stopping Ceph mds.h for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:04.938 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h[51107]: 2024-04-20T11:36:04.530+0000 7f01aab23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:04.938 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:04 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h[51107]: 2024-04-20T11:36:04.530+0000 7f01aab23700 -1 mds.h *** got signal Terminated *** 2024-04-20T11:36:07.813 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:07 smithi078 podman[58461]: 2024-04-20 11:36:07.513705489 +0000 UTC m=+3.111837494 container died 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T11:36:07.814 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:07 smithi078 podman[58461]: 2024-04-20 11:36:07.814220732 +0000 UTC m=+3.412352733 container cleanup 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T11:36:08.143 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:07 smithi078 bash[58461]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h 2024-04-20T11:36:08.143 INFO:journalctl@ceph.mds.h.smithi078.stdout:Apr 20 11:36:08 smithi078 podman[58475]: 2024-04-20 11:36:08.023964504 +0000 UTC m=+0.508430211 container remove 84fbf08d33c454a5db8acc247cd6b221ddda2082c9653e80d3f8947809d72326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:36:08.163 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.h.service' 2024-04-20T11:36:08.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:08.196 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-20T11:36:08.196 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-20T11:36:08.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.k 2024-04-20T11:36:08.438 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:36:08 smithi078 systemd[1]: Stopping Ceph mds.k for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:08.938 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:36:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k[50459]: 2024-04-20T11:36:08.534+0000 7fb2c116e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:08.938 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:36:08 smithi078 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k[50459]: 2024-04-20T11:36:08.534+0000 7fb2c116e700 -1 mds.k *** got signal Terminated *** 2024-04-20T11:36:11.827 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:36:11 smithi078 podman[58581]: 2024-04-20 11:36:11.515344194 +0000 UTC m=+3.091837061 container died af72995753b688d14cb070cc27330a3be9be3edccfa7aab5e339782c30922547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T11:36:12.118 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:36:11 smithi078 podman[58581]: 2024-04-20 11:36:11.90963606 +0000 UTC m=+3.486128926 container cleanup af72995753b688d14cb070cc27330a3be9be3edccfa7aab5e339782c30922547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T11:36:12.118 INFO:journalctl@ceph.mds.k.smithi078.stdout:Apr 20 11:36:11 smithi078 bash[58581]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-k 2024-04-20T11:36:12.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.k.service' 2024-04-20T11:36:12.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:12.263 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-20T11:36:12.263 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-20T11:36:12.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.c 2024-04-20T11:36:12.567 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:36:12 smithi165 systemd[1]: Stopping Ceph mds.c for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:12.979 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:36:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c[52691]: 2024-04-20T11:36:12.568+0000 7f0948a09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:12.979 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:36:12 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c[52691]: 2024-04-20T11:36:12.568+0000 7f0948a09700 -1 mds.c *** got signal Terminated *** 2024-04-20T11:36:15.800 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:36:15 smithi165 podman[59626]: 2024-04-20 11:36:15.503591721 +0000 UTC m=+3.051416674 container died da122010b2c8fbfee2dc0c861b9cb7599040a51d933f12566a4bbe6ff51f5c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T11:36:16.118 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:36:15 smithi165 podman[59626]: 2024-04-20 11:36:15.918047059 +0000 UTC m=+3.465872013 container cleanup da122010b2c8fbfee2dc0c861b9cb7599040a51d933f12566a4bbe6ff51f5c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T11:36:16.118 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 20 11:36:15 smithi165 bash[59626]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-c 2024-04-20T11:36:16.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.c.service' 2024-04-20T11:36:16.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:16.257 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-20T11:36:16.258 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-20T11:36:16.258 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.f 2024-04-20T11:36:16.479 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:36:16 smithi165 systemd[1]: Stopping Ceph mds.f for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:16.979 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:36:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f[52385]: 2024-04-20T11:36:16.589+0000 7f939f1c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:16.979 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:36:16 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f[52385]: 2024-04-20T11:36:16.589+0000 7f939f1c4700 -1 mds.f *** got signal Terminated *** 2024-04-20T11:36:19.798 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:36:19 smithi165 podman[59758]: 2024-04-20 11:36:19.518593385 +0000 UTC m=+3.036917946 container died 76a599a47916ad8810d008eb1c18be252f30a621fea2ba22b3790d5acc69c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-20T11:36:20.131 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:36:19 smithi165 podman[59758]: 2024-04-20 11:36:19.880759265 +0000 UTC m=+3.399083827 container cleanup 76a599a47916ad8810d008eb1c18be252f30a621fea2ba22b3790d5acc69c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-20T11:36:20.131 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 20 11:36:19 smithi165 bash[59758]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-f 2024-04-20T11:36:20.311 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.f.service' 2024-04-20T11:36:20.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:20.343 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-20T11:36:20.344 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-20T11:36:20.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.i 2024-04-20T11:36:20.684 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:36:20 smithi165 systemd[1]: Stopping Ceph mds.i for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:20.979 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:36:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i[52999]: 2024-04-20T11:36:20.685+0000 7f609d1c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:20.979 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:36:20 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i[52999]: 2024-04-20T11:36:20.685+0000 7f609d1c7700 -1 mds.i *** got signal Terminated *** 2024-04-20T11:36:23.784 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:36:23 smithi165 podman[59878]: 2024-04-20 11:36:23.51144037 +0000 UTC m=+2.947213595 container died 1e7b0f13131aabd1ecea72742487511b21e02f068d690c44348ca330831466fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-20T11:36:24.127 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:36:23 smithi165 podman[59878]: 2024-04-20 11:36:23.868360658 +0000 UTC m=+3.304133882 container cleanup 1e7b0f13131aabd1ecea72742487511b21e02f068d690c44348ca330831466fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T11:36:24.127 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 20 11:36:23 smithi165 bash[59878]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-i 2024-04-20T11:36:24.300 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.i.service' 2024-04-20T11:36:24.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:24.331 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-20T11:36:24.331 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-20T11:36:24.331 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.l 2024-04-20T11:36:24.729 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:24 smithi165 systemd[1]: Stopping Ceph mds.l for 759aac0c-ff06-11ee-bc92-c7b262605968... 2024-04-20T11:36:25.229 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-l[52069]: 2024-04-20T11:36:24.772+0000 7ffb85d08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T11:36:25.229 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:24 smithi165 ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-l[52069]: 2024-04-20T11:36:24.772+0000 7ffb85d08700 -1 mds.l *** got signal Terminated *** 2024-04-20T11:36:27.721 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:27 smithi165 podman[60001]: 2024-04-20 11:36:27.401933997 +0000 UTC m=+2.854327504 container died c3f64b075a58aa6b2e198d2e0507849f4278a3a4664007336e1ded3ef7652b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T11:36:27.721 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:27 smithi165 podman[60001]: 2024-04-20 11:36:27.722345011 +0000 UTC m=+3.174738518 container cleanup c3f64b075a58aa6b2e198d2e0507849f4278a3a4664007336e1ded3ef7652b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True) 2024-04-20T11:36:27.979 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:27 smithi165 bash[60001]: ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-l 2024-04-20T11:36:27.979 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 20 11:36:27 smithi165 podman[60015]: 2024-04-20 11:36:27.8900153 +0000 UTC m=+0.486527739 container remove c3f64b075a58aa6b2e198d2e0507849f4278a3a4664007336e1ded3ef7652b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-759aac0c-ff06-11ee-bc92-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-20T11:36:28.097 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-759aac0c-ff06-11ee-bc92-c7b262605968@mds.l.service' 2024-04-20T11:36:28.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T11:36:28.128 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-20T11:36:28.128 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T11:36:28.456 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:36:33.660 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T11:36:33.855 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:36:36.272 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T11:36:36.462 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:36:38.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:36:38.904 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:36:38.935 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T11:36:38.961 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-20T11:36:38.963 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi044/crash 2024-04-20T11:36:38.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash -- . 2024-04-20T11:36:39.056 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T11:36:39.056 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-04-20T11:36:39.062 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi078/crash 2024-04-20T11:36:39.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash -- . 2024-04-20T11:36:39.104 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T11:36:39.104 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-04-20T11:36:39.106 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi165/crash 2024-04-20T11:36:39.107 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash -- . 2024-04-20T11:36:39.137 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T11:36:39.137 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-04-20T11:36:39.138 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-20T11:36:39.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-20T11:36:39.248 INFO:tasks.cephadm:Compressing logs... 2024-04-20T11:36:39.249 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:36:39.254 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:36:39.256 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:36:39.289 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:: No such file or directory 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.2.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.f.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log: /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.5.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.c.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log: /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.c.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.z.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.i.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.z.log: 87.7% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T11:36:39.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T11:36:39.291 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.11.log 2024-04-20T11:36:39.291 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T11:36:39.291 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.8.log 2024-04-20T11:36:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log 2024-04-20T11:36:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-04-20T11:36:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T11:36:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.7.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log: /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.b.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.4.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.1.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.b.log: /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.y.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.h.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.10.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.h.log: 87.7% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr: 90.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.e.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.y.log.gz 2024-04-20T11:36:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.b.log 2024-04-20T11:36:39.299 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.z.log.gz 2024-04-20T11:36:39.299 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T11:36:39.301 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.l.log 2024-04-20T11:36:39.308 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T11:36:39.308 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.k.log 2024-04-20T11:36:39.311 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T11:36:39.312 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-20T11:36:39.315 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log 2024-04-20T11:36:39.316 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T11:36:39.318 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T11:36:39.318 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.g.log 2024-04-20T11:36:39.318 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log: -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.j.log 2024-04-20T11:36:39.320 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.3.log 2024-04-20T11:36:39.320 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T11:36:39.321 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.0.log 2024-04-20T11:36:39.322 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.x.log 2024-04-20T11:36:39.322 INFO:teuthology.orchestra.run.smithi044.stderr: 90.2% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T11:36:39.323 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.a.log 2024-04-20T11:36:39.325 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.6.log 2024-04-20T11:36:39.326 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T11:36:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.l.log.gz 2024-04-20T11:36:39.329 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.6.log: /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.a.log 2024-04-20T11:36:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T11:36:39.334 INFO:teuthology.orchestra.run.smithi165.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T11:36:39.335 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.a.log: 84.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T11:36:39.335 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T11:36:39.336 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.d.log 2024-04-20T11:36:39.338 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.9.log 2024-04-20T11:36:39.340 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T11:36:39.344 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T11:36:39.349 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.a.log.gz 2024-04-20T11:36:39.362 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T11:36:39.368 INFO:teuthology.orchestra.run.smithi044.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T11:36:39.423 INFO:teuthology.orchestra.run.smithi165.stderr: 94.1% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.c.log.gz 2024-04-20T11:36:39.464 INFO:teuthology.orchestra.run.smithi165.stderr: 94.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.f.log.gz 2024-04-20T11:36:39.767 INFO:teuthology.orchestra.run.smithi044.stderr: 90.1% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mgr.x.log.gz 2024-04-20T11:36:39.814 INFO:teuthology.orchestra.run.smithi165.stderr: 91.8% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.c.log.gz 2024-04-20T11:36:39.939 INFO:teuthology.orchestra.run.smithi078.stderr: 91.9% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.b.log.gz 2024-04-20T11:36:40.225 INFO:teuthology.orchestra.run.smithi044.stderr: 93.4% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.g.log.gz 2024-04-20T11:36:40.465 INFO:teuthology.orchestra.run.smithi165.stderr: 93.7% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.i.log.gz 2024-04-20T11:36:40.557 INFO:teuthology.orchestra.run.smithi078.stderr: 95.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.k.log.gz 2024-04-20T11:36:40.720 INFO:teuthology.orchestra.run.smithi044.stderr: 90.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mon.a.log.gz 2024-04-20T11:36:40.779 INFO:teuthology.orchestra.run.smithi044.stderr: 95.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.j.log.gz 2024-04-20T11:36:41.843 INFO:teuthology.orchestra.run.smithi078.stderr: 95.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.h.log.gz 2024-04-20T11:36:42.295 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.8.log.gz 2024-04-20T11:36:42.526 INFO:teuthology.orchestra.run.smithi165.stderr: 91.6% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.11.log.gz 2024-04-20T11:36:42.886 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.2.log.gz 2024-04-20T11:36:42.900 INFO:teuthology.orchestra.run.smithi044.stderr: 92.2% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.9.log.gz 2024-04-20T11:36:42.926 INFO:teuthology.orchestra.run.smithi044.stderr: 91.9% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.6.log.gz 2024-04-20T11:36:43.103 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.5.log.gz 2024-04-20T11:36:43.104 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-20T11:36:43.105 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m3.835s 2024-04-20T11:36:43.105 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m15.196s 2024-04-20T11:36:43.105 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m0.531s 2024-04-20T11:36:43.774 INFO:teuthology.orchestra.run.smithi078.stderr: 92.2% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.4.log.gz 2024-04-20T11:36:43.792 INFO:teuthology.orchestra.run.smithi078.stderr: 91.5% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.7.log.gz 2024-04-20T11:36:43.835 INFO:teuthology.orchestra.run.smithi078.stderr: 91.8% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.10.log.gz 2024-04-20T11:36:43.884 INFO:teuthology.orchestra.run.smithi044.stderr: 91.6% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.0.log.gz 2024-04-20T11:36:44.382 INFO:teuthology.orchestra.run.smithi044.stderr: 91.6% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.3.log.gz 2024-04-20T11:36:44.462 INFO:teuthology.orchestra.run.smithi078.stderr: 92.0% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-osd.1.log.gz 2024-04-20T11:36:48.462 INFO:teuthology.orchestra.run.smithi044.stderr: 93.5% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.d.log.gz 2024-04-20T11:36:48.467 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-20T11:36:48.467 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m9.180s 2024-04-20T11:36:48.467 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m27.720s 2024-04-20T11:36:48.467 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m2.045s 2024-04-20T11:36:49.803 INFO:teuthology.orchestra.run.smithi078.stderr: 93.3% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.e.log.gz 2024-04-20T11:37:00.778 INFO:teuthology.orchestra.run.smithi078.stderr: 93.1% -- replaced with /var/log/ceph/759aac0c-ff06-11ee-bc92-c7b262605968/ceph-mds.b.log.gz 2024-04-20T11:37:00.780 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-04-20T11:37:00.780 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m21.510s 2024-04-20T11:37:00.780 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m52.896s 2024-04-20T11:37:00.780 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m1.605s 2024-04-20T11:37:00.780 INFO:tasks.cephadm:Archiving logs... 2024-04-20T11:37:00.781 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi044/log 2024-04-20T11:37:00.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:37:05.251 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi078/log 2024-04-20T11:37:05.251 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:37:12.275 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi165/log 2024-04-20T11:37:12.276 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:37:14.838 INFO:tasks.cephadm:Removing cluster... 2024-04-20T11:37:14.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --force 2024-04-20T11:37:15.166 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:37:15.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --force 2024-04-20T11:37:16.038 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:37:16.615 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 759aac0c-ff06-11ee-bc92-c7b262605968 --force 2024-04-20T11:37:16.807 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: 759aac0c-ff06-11ee-bc92-c7b262605968 2024-04-20T11:37:17.182 INFO:tasks.cephadm:Removing cephadm ... 2024-04-20T11:37:17.183 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T11:37:17.244 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T11:37:17.260 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T11:37:17.275 INFO:tasks.cephadm:Teardown complete 2024-04-20T11:37:17.275 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T11:37:17.286 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T11:37:17.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:37:17.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:37:17.303 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:37:17.436 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T11:37:17.446 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T11:37:17.446 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:37:17.454 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:37:17.456 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:37:17.467 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:37:17.469 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:37:17.472 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:37:17.472 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-20T11:37:17.472 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 34 +727us[ +745us] +/- 71ms 2024-04-20T11:37:17.472 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv02.front.sepia.ceph.com 3 8 377 31 -136us[ -119us] +/- 64ms 2024-04-20T11:37:17.472 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -612us[ -612us] +/- 89ms 2024-04-20T11:37:17.472 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:37:17.473 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:37:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-20T11:37:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +692us[ +704us] +/- 71ms 2024-04-20T11:37:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv02.front.sepia.ceph.com 3 7 377 32 -107us[ -95us] +/- 64ms 2024-04-20T11:37:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -603us[ -603us] +/- 89ms 2024-04-20T11:37:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:37:17.503 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:37:17.515 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:37:17.515 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-20T11:37:17.515 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 33 +732us[ +744us] +/- 71ms 2024-04-20T11:37:17.515 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -154us[ -141us] +/- 64ms 2024-04-20T11:37:17.515 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -594us[ -581us] +/- 89ms 2024-04-20T11:37:17.515 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:37:17.516 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T11:37:17.526 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T11:37:17.527 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T11:37:17.537 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:37:17.739 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:37:17.791 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:37:17.843 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:37:17.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:17.944 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:37:17.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:17.978 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:37:18.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:18.004 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T11:37:18.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T11:37:18.122 INFO:teuthology.task.internal:Duration was 2056.091952 seconds 2024-04-20T11:37:18.122 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T11:37:18.145 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T11:37:18.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:37:18.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:37:18.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:37:18.194 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:37:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:37:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:37:18.787 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T11:37:18.787 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T11:37:18.787 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:37:18.851 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-04-20T11:37:18.851 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:37:18.890 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T11:37:18.890 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:37:18.912 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T11:37:18.912 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:37:18.920 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:37:18.932 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:37:19.008 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T11:37:19.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:37:19.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:37:19.016 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:37:19.515 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T11:37:19.526 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T11:37:19.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:37:19.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:37:19.639 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:37:19.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T11:37:19.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:37:19.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:37:19.686 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:37:19.707 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-04-20T11:37:19.728 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-04-20T11:37:19.782 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-20T11:37:19.828 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:37:19.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:19.879 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:37:19.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:19.894 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:37:19.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:37:19.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T11:37:19.921 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T11:37:19.922 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi044 2024-04-20T11:37:19.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:37:20.030 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi078 2024-04-20T11:37:20.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:37:20.063 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664510/remote/smithi165 2024-04-20T11:37:20.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:37:20.097 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T11:37:20.097 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:37:20.105 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:37:20.107 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:37:20.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T11:37:20.170 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T11:37:20.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T11:37:20.181 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T11:37:20.182 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:37:20.189 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:37:20.192 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:37:20.204 INFO:teuthology.orchestra.run.smithi078.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:37 /home/ubuntu/cephtest 2024-04-20T11:37:20.206 INFO:teuthology.orchestra.run.smithi165.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:37 /home/ubuntu/cephtest 2024-04-20T11:37:20.241 INFO:teuthology.orchestra.run.smithi044.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:37 /home/ubuntu/cephtest 2024-04-20T11:37:20.244 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T11:37:20.258 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T11:37:20.285 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} duration: 2056.091952085495 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-20T11:37:20.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T11:37:20.371 INFO:teuthology.run:pass